Al Roth's
Game Theory, Experimental Economics, and Market Design Page

Updated 4/16/14 Check out my Market Design Blog
My Stanford front page with contact info, etc.

This is a still very preliminary update of the page I will display at stanford...

(It's been a while since I comprehensively updated this page, but I've kept updating my Market Design Blog (and the most exciting posts are probably this one on my students on the job market this year, and those tagged Nobel, see also the 2012 Nobel Memorial Prize in Economics. ).

I am the Craig and Susan McCaw Professor of Economics at Stanford (and the Gund Professor Emeritus of Economics and Business Administration at Harvard). My new address is Dept. of Economics, Stanford University, Stanford CA 94305, my email is alroth "at" stanford.edu.

My research is in game theory, experimental economics, and market design (for which game theory, experimentation, and computation are complementary tools).

Here is my Spring 2012 course in Experimental Economics (Ec 2040/HBS 4160). [Updated 1/23/12: it will be updated continuously without further announcement here, as new slides, schedules etc. become available.]

Here is my Fall 2011 Market Design course (supplemented by a Market Design Blog). We also may continue to have an occasional seminar on matching and market design. [Updated 8/24/11.]

The Behavioral and Experimental Economics Workshop (Tuesday Seminar) [Updated 2/7/12]

Job market candidates: one postdoc, two students whose main adviser I was privileged to be, and two students who I helped to advise on papers of mutual interest:

Follow the link above for more on Clayton Featherstone (also on the Stanford job market page here);Eduardo Azevedo, and Jacob Leshno; and Yuichiro Kamada, and Katie Baldiga. Update: they all got jobs:) Check out the updates at the bottom of this blog post: Five Harvard candidates for the Economics job market this year (2011-12) [Updated 5/29/12].

You can scroll through this page, or use the links in the table of contents. (Also, many of my papers and books can be found in downloadable format in chronological order, rather than by topic (including my previously out of print 1979 book Axiomatic Models of Bargaining.)

 

Table of Contents

  • Experimental Economics [Updated 2/23/08: Roth, A.E., "On the Early History of Experimental Economics," Francesco Guala's History of Experimental Economics.]

  •  

    Some very short introductory articles (and press)

    Game-theoretic models of bargaining. permission to webpublish

    Two-sided matching models. permission to webpublish

    Experimental economics (revised for 2003). permission to webpublish

    Is Economics a Science? (Of course it is...) (two short letters to The Economist, one by me, one by Charlie Plott)

    Matching and Allocation in Medicine and Health Care (transcript of a talk at the National Academy of Engineering)

    For an account of my collaboration with Keith Murnighan, a social psychologist with whom I've written a dozen papers, see "Some of the Ancient History of Experimental Economics and Social Psychology: Reminiscences and Analysis of a Fruitful Collaboration,"

    Here's an August 2010 article about me in Forbes that focuses on school choice in NYC. (For those of you who find that articles lose something in the original, you can read it here in Hebrew.)

    The National Science Foundation has asked for 2000 word 'white papers' on challenges worth exploring over the next decade: here's mine, on Market Design.

    The NSF published an interview with me about market design (Oct. 15, 2010): Economist Finds Best Matches for Students and Schools

    Leon Neyfakh at the Boston Globe wrote a profile of me and market design (April 3, 2011): The Matchmaker: The Harvard economist who stopped just studying the world and began trying to fix it

    The Chronicle of Higher Education takes an interest in the job market for economists: Stanford Lures Alvin Roth and 2 Other Economists From Harvard

    Return to Table of Contents

    Short expository papers

    For some general exhortation regarding game theory as a part of empirical economics, see my article by that name in the Economic Journal: Game theory as a part of empirical economics.

    For some thoughts on how game theory, experimental economics, and computation will form the toolbox for the design of markets and other economic environments, see Game Theory as a Tool for Market Design (in pdf format).

    For some discussion of why the experimental evidence from individual choice experiments is interpreted differently by many psychologists and many economists, see Individual rationality as a useful approximation.

    For an account of the history of experimental economics from 1930 to 1960, see Early History of Experimental Economics.

    For some methodological exhortation (on how we conduct and report experiments) see "Let's Keep the Con Out of Experimental Econ.". (see also Conduct in Science)

    For a discussion of the Nash equilibrium, see my short paper with Charlie Holt in PNAS, "The Nash Equilibrium: A Perspective"

    Niederle, Muriel, Alvin E. Roth and Tayfun Sonmez, ''Matching and Market Design'', The New Palgrave Dictionary of Economics. 2nd Edition. Eds. Steven N. Durlauf and Lawrence E. Blume, 2008.

    Erev, Ido, and Alvin E. Roth, ''Multi-agent learning and the descriptive value of simple models,'' Artificial Intelligence, 171, 2007, 423-428.

    Roth, Alvin E. ''The Art of Designing Markets,'' Harvard Business Review , October, 2007, 118-126.

    Roth, Alvin E. "Is Experimental Economics Living Up to Its Promise?, in Frechette, Guillaume and Andrew Schotter (editors), in The Methods of Modern Experimental Economics, Oxford University Press, forthcoming.

    Roth, Alvin E. "In 100 Years," January 2012, (for a volume of predictions by economists, In 100 Years, edited by Ignacio Palacios-Huerta, MIT Press, forthcoming)

    Return to Table of Contents

    Video and Audio: Lectures and Radio Interviews

    Video-taped survey lectures:

    All of a sudden people are putting lectures on the web: here are some overlapping overviews of market design I recently gave:

    Rosenthal Memorial Lecture at Boston University: What have we learned from market design? April 27, 2007. (about an hour and a half, covers kidney exchange, and school choice in New York and Boston, see if you can find the photo that seems to be of Bart Lipman when I talk about repugnance of some transactions).

    Market failures and Market Design, Google, October 11, 2007. (about an hour, covers the labor market designs for gastroenterologists, orthopaedic surgeons, and others, and kidney exchange.)

    Yahoo! Big Thinker Series: Market Design: October 15, 2007. (about an hour, covers kidney exchange and school choice, and repugnant transactions.)

    Nancy L. Schwartz Memorial Lecture: I gave the 2010 lecture, here is the video (about 1 and a quarter hours)

    Random Graph Models in Kidney Exchange - Theoretical Developments and Practical Challenges (the movie), presented at MSR New England, July 25, 2012. The video runs an hour and a half, including the questions at the end.

     

     

    Radio interviews and podcasts:

    Here's a radio interview I did about game theory on The Brian Lehrer Show on WNYC, New York Public Radio, 10/11/2005 (on the occasion of the Aumann/Schelling Nobel Prize): Listen ; Download MP3

    You can listen to a 90-second show about kidney exchange (from 2005) on the NSF-sponsored radio broadcast Imagine That.

    My paper on Repugnant Markets formed the basis for a BBC Radio 4 broadcast in July 2007: here's the half-hour long recording .

    Here's a radio interview about market design, 11/14/07, from public radio station WOSU in Columbus Ohio (about 1 hour).

    Robin Young of NPR's Here and Now interviewed me about repugnant markets on March 25, 2008 in a piece called The "Yuck!" Factor (15 minutes)

    Here's a 15 minute audio interview on repugnance as a constraint on market design, conducted by Romesh Vaitilingam at the 2008 AEA meetings in New Orleans.

    Here's a 10 minute interview on school choice, in which Chad Aldeman interviews me and Atila Abudlkadiroglu: "A Closer Look at Mandatory School Choice" .

    Here's a 4 minute interview on Radio Scotland from August 2011, on kidney sales, in which Sarah Toom talks to me about the debate there about whether payments to living kidney donors should be allowed, and other ways of increasing transplants (including kidney exchange).

    The March 2012 issue of the AMA journal Virtual Mentor is a special issue on Organ Transplantation.
    It contains a two part podcast of an interview with me about kidney exchange (about ten minutes each):
    http://www.soundprescribing.org/podcast/ethics-talk-kidney-patient-donor-exchanges-part-one.mp3
    http://www.soundprescribing.org/podcast/ethics-talk-kidney-patient-donor-exchanges-part-two.mp3

    Return to Table of Contents

    Slide Presentations

    Repugnant Transactions (2008 Kahneman Lecture, IAREP World Meeting at LUISS, Rome, September 4.)

    On March 15, 2010 I gave a 15 minute presentation as part of a Congressional Briefing on the usefulness of Economics (particularly economic research supported by the NSF). Here are my slides: Improved Markets for Doctors, Organ Transplants and School Choice.

    Return to Table of Contents

    Market Design

    Economic institutions evolve, but they are also designed. Market makers and regulators, as well as entrepreneurs and managers, have always been involved to some extent in the design of economic institutions.

    Game theory is the part of economics concerned with the detailed rules and procedures of economic institutions. But only relatively recently have the depth and breadth of robust game theoretic knowledge been sufficient so that game theorists can with some confidence offer practical advice on institutional design.

    As robust knowledge accumulates, game theorists are increasingly able to offer advice on the design of economic institutions in environments which may still be too complex to prove theorems about. That is, the process of giving advice is not the same as looking in Econometrica for the right theorem, but neither is it entirely unrelated. This is natural: just as chemical engineering is related to but different from chemistry, we can expect "microeconomic engineering" and economic design to be different from but related to game theory. (The final paragraph of my 1991 essay Game theory as a part of empirical economics, reads as follows.)

    "... in the long term, the real test of our success will be not merely how well we understand the general principles that govern economic interactions, but how well we can bring this knowledge to bear on practical questions of microeconomic engineering, to design appropriate mechanisms for price formation (as in different kinds of auction), dispute resolution, executive compensation, market organization, etc. To do this we'll need to learn more about the various kinds of frictions that enter economic environments as a function of size and complexity, about which properties of these environments are robust and which are fragile, and about which kinds of environments facilitate which kinds of learning. Just as chemical engineers are called upon not merely to understand the principles that govern chemical plants, but to design them, and just as physicians aim not merely to understand the biological causes of disease, but their treatment and prevention, a measure of the success of microeconomics will be the extent to which it becomes the source of practical advice, solidly grounded in well tested theory, on designing the institutions through which we interact with one another."

    It has been gratifying, in the years since I wrote that, to see market design begin to emerge as one of the subdisciplines of economics. As game theorists are increasingly asked to answer design questions, we need to pay attention to good examples of economic design, to start building a sensible, practice oriented literature. This page is meant to take a step in that direction, by pointing out some recent examples.

    In the years since my 1991 essay I have gotten to design (or redesign, or advise on the design) of a number of market and quasi-market allocation systems, from the National Resident Matching Program (that puts American doctors into their first jobs, to school choice systems now operating in New York City and Boston, to the New England Program for Kidney Exchange, to the market for Gastroenterology Fellows, and the market for new Ph.D. economists. In the course of this, my colleagues and I have learned a lot about design, and also about the variety of market failures that create the need for new markets to be designed.

    A lot of market design has turned out to involve game theoretic models of matching. Here's a link to a good book on the subject:

    I've now written several

    Surveys of Market Design:

    A lot of market design has involved deferred acceptance algorithms, that build on the famous 1962 paper by Gale and Shapley that introduced the marriage and college admissions problems and stable matching generally. Here's a survey I prepared in honor of David Gale's 85th birthday:

    Below is my 2007 Hahn Lecture at the Royal Economic Society :

    For a more specialized survey (listed below under Gastroenterology) see my paper with Muriel Niederle, ''The Effects of a Central Clearinghouse on Job placement, Wages, and Hiring Practice'', in Labor Market Intermediation, David Autor, Editor, NBER, forthcoming.

    Return to Table of Contents

    Medical and health care labor markets

    I started studying the labor market for American doctors in the early 1980's, and was asked to direct its redesign in 1995. The Roth-Peranson (1999) algorithm that resulted is now used to organize a number of medical and (mostly) health care markets. The following are organized through or with the support of Elliott Peranson's company National Matching Services: Postdoctoral Dental Residencies in the United States, Psychology Internships in the United States and Canada, Neuropsychology Residencies in the United States and Canada, Osteopathic Internships in the United States, Pharmacy Practice Residencies in the United States, Articling Positions with Law Firms in Alberta, Canada, Medical Residencies in the United States (NRMP), Medical Residencies in Canada (CaRMS).

    National Resident Matching Program and related medical labor markets

    An anti-trust suit against the NRMP and numerous other defendants was brought in 2002 by 16 law firms on behalf of 3 former residents seeking to represent the class of all former residents (and naming as defendants a class including all hospitals that employ residents). It was dismissed on August 12, 2004 in an Opinion, Order & Judgment by Judge Paul L. Friedman. Here is the original lawsuit (later amended in various ways in different filings) and Orley Ashenfelter's expert report for the plaintiffs. Here is the defendants' website (via the WaybackMachine). The theory of the complaint is that a match holds down wages for residents and fellows. Jeremy Bulow and Jon Levin have a paper in the June 2006 AER Matching and Price Competition providing some logical support for this possibility, by comparing impersonal prices to perfectly competitive prices at which each worker is paid his marginal product. Vince Crawford has a followup paper forthcoming in JEBO, The Flexible-Salary Match: A Proposal to Increase the Salary Flexibility of the National Resident Matching Program observing that personalized prices can be incorporated in a match. Muriel Niederle's and my JAMA paper "Relationship Between Wages and Presence of a Match in Medical Fellowships," showed that in fact there is no difference in wages between medicine subspecialties that use a match and those that don't, and our work (see below) on the collapse of the gastroenterology match showed that, following that collapse, wages did not rise (and many adverse consequences for applicants transpired). Muriel Neiderle has a December 2007AER paper more carefully modeling the form that ordered contracts take in the NRMP, which shows that competitive equilibrium is not hindered. Fuhito Kojima has a June 2007 AER paper showing that impersonal prices may not compress wages as in Bulow and Levin when firms can hire more than one worker.. Congress passed legislation clarifying that the NRMP is a marketplace and does not violate antitrust laws. (This was attached to a larger bill, the Pension Funding Equity Act of 2004 Public Law 108-218; the relevant section is quite short and easy to read, it is SEC. 207. CONFIRMATION OF ANTITRUST STATUS OF GRADUATE MEDICAL RESIDENT MATCHING PROGRAMS. This legislation led directly to the dismissal of the case. (In June 2006 the appellate court upheld the dismissal, and on January 8, the Supreme Court denied plaintiffs' petition to hear an appeal of the dismissal.).

     

  • Kojima, Fuhito, Parag A. Pathak, and Alvin E. Roth, " Matching with Couples: Stability and Incentives in Large Markets," working paper, May, 2010.
  • Comment on the NRMP’s “Supplemental Offer and Acceptance Program” Proposed to Replace the Post-Match Scramble by Peter A. Coles, Clayton R. Featherstone, John William Hatfield, Fuhito Kojima, Scott Duke Kominers, Muriel Niederle, Parag A. Pathak, and Alvin E. Roth.
  • Gastroenterologists, Orthopaedic surgeons (from centralized to decentralized and back again)

    Muriel Niederle and I have examined the failure of the Gastroenterology Fellowship market in the late 1990's, it's consequences, and what would be needed to get it back on its feet. (This helped us learn a lot about what clearinghouses accomplish.) We helped organize the centralized match that was successfully started in June 2006, and the policies that govern the transition to the match.

    Here's the resolution about the conduct of the match passed in 2006 by the four main Gastroenterology professional societies.

    Orthopaedic Surgeons:

    Harner, Christopher D., Anil S. Ranawat, Muriel Niederle, Alvin E. Roth, Peter J. Stern, Shepard R. Hurwitz, William Levine, G. Paul DeRosa, Serena S. Hu, "Current State of Fellowship Hiring: Is a universal match necessary? Is it possible?," Journal of Bone and Joint Surgery, 90, 2008,1375-1384.

    Return to Table of Contents

    Kidney Exchange (aka "Kidney Paired Donation" or Paired Kidney Donation)

    After Tayfun Sonmez and Utku Unver and I wrote our first paper on kidney exchange (inspired by the pioneering surgeons who did the first handful of these, in Maryland, New England, and Ohio), we contacted Drs. Frank Delmonico and Susan Saidman of the Massachusetts General Hospital. With them we helped found the New England Program for Kidney Exchange (NEPKE) in 2005. We have also been running matches for Drs. Steve Woodle and Michael Rees and their colleagues in the Paired Donation Consortium they started in Ohio, and more recently for the Alliance for Paired Donation. There is now a movement towards a national exchange...

    Papers

    News reports:

    There have been several news stories about the work on implementing kidney exchange in New England that Sonmez and Unver and I are doing with local surgeons and tissue typing experts, especially Frank Delmonico and Susan Saidman. The Boston Globe ran a front page story called Cross-donor system planned for region's kidney patients, June 5, 2004, and The Wall Street Journal ran a story on page B1 titled "Easing the Kidney Shortage,," June 17, 2004. The SIAM News ran a slightly more technical story. The National Science Foundation has a story highlighting the basic research underlying the organization of kidney exchange, Kidney Exchange: A Life-Saving Application of Matching Theory. You can listen to a 90-second show about kidney exchange (from 2005)on the NSF-sponsored radio broadcast Imagine That. The organization we helped found is the New England Program for Kidney Exchange (NEPKE).The Boston Globe ran a story called Giving Life Despite Limits on 3/14/06 emphasizing the aspect of matching strangers. Dubner and Levitt have a column on kidney transplants in the 7/9/2006 NY Times Sunday Magazine, and some related links on their Freakonomics page. Quite apart from the complexities of matching, there are moving human stories behind each exchange. Here is a story with a photo of the six participants (3 donors, 3 recipients) in an undirected (altruistic) donor chain matched through NEPKE, when they all met in March 2007, a month after their successful, simultaneous surgeries. The Pittsburgh Post Gazette ran a pair of stories in May 2007 focusing on the software being developed to implement kidney exchange by Tuomas Sandholm (and here is a more detailed story about Sandholm's work, in a CMU magazine), which now replaces the earlier software written by Utku Unver in the matches run for the Alliance for Paired Donation. The AMA ran a story in their Jan.28, 2008 issue of amednews.com on kidney markets, repugnance, and kidney exchange. Tim Harford (the Undercover Economist) has a story on the 6-way exchange run at Hopkins in April 2008, following up his related article in the Financial Times, "Stakes in kidneys." Here's a June 16, 2008 article (in German) in the Handelsblatt (and here's the automated google translation:) Roth et al. (2006) proposed that chains initiated by non-directed donors need not have only simultaneous surgeries, and this has come to pass, here's a WSJ report Daisy Chains of Kidney Donations. Here's a story from Miller-McCune magazine: Making a Market for Kidneys. (For some reason, many of these stories mention me without naming my colleagues, Utku Unver and Tayfun Sonmez by name. This is a longstanding source of frustration, which I make a point of mentioning to the reporters, but, in this case, to no avail. I wish I knew how to solve that problem...:(fortunately some news accounts of our NEJM article on NEAD chains redress the imbalance). The NEJM story drew a lot of press; here's an account of some of it. (While I think of it, here's a collection of my blog posts on kidney exchange.) Here's a story from Carnegie Mellon University, focusing on the work of computer scientists David Abraham, Avrim Blum and Tuomas Sandholm on developing an algorithm for large numbers of patients and donors. Virginia Postrel has an informative article in the July 2009 Atlantic. Haaretz has an article (in Hebrew) on kidney exchange, featuring the work of Itai Ashlagi.

    Kidney paired donation and the law:

    Because the National Organ Transplant Act (NOTA) of 1984 prohibits ''any person to knowingly acquire, receive, or otherwise transfer any human organ for valuable consideration for use in human transplantation..." questions have sometimes arisen about the legal status of kidney exchange, and of the possible future ability of UNOS or some other organization created in part by the NOTA to organize it. UNOS published a legal opinion that kidney exchange is legal under the NOTA as written, and legislation to amend the NOTA to make this explicit was introduced in the House and Senate in January and February of 2007, and became the `Charlie W. Norwood Living Organ Donation Act' [Public Law 110-144] in December, 2007. (In late March 2007--better late than never-- the U.S. Department of Justice finally issued a memo saying that in fact kidney exchange is legal under the NOTA.) The Norwood Act makes clear that "human organ paired donation" includes not just 2-way exchanges, but larger ones as well (the "paired" in "paired donation" here is interpreted as referring to the patient-donor pairs, and avoids the need to use the word "exchange").

    Progress on a national paired kidney donation clearinghouse:

    Following the recent changes in the legal landscape, a national kidney exchange clearinghouse is being considered by the United Network for Organ Sharing (UNOS), the non-profit agency that already administers the national Organ Procurement and Transplantation Network (OPTN) for deceased donors. On February 4, 2008, they hosted a meeting to hear proposals on how such a clearinghouse might be organized (here is the agenda, and here are the slides that Utku Unver and I discussed in the first presentation, and a follow-up summary memorandum).

    Slide presentations:

    Photos:

    Here are two photos of a kidney exchange I observed in January 2006 (I am the one in the yellow gown, keeping my hands behind my back...) kidney prep, transplant.

    Links:

    New England Program for Kidney Exchange(NEPKE)

    Nepke's description and bibliography of kidney exchange

    Alliance for Paired Donation

     

    Other organ transplantation issues

  • Leider, Stephen and Alvin E. Roth, ''Kidneys for sale: Who disapproves, and why?'' American Journal of Transplantation , 10 (May), 2010, 1221-1227.

  • Kessler, Judd B. and Alvin E. Roth, '' Organ Allocation Policy and the Decision to Donate,'' American Economic Review, forthcoming.

  •  


    School choice mechanisms

    In May 2003 the Director of Strategic Planning at the New York City Dept. of Education contacted me for advice on designing a centralized high school admission process for the 90,000+ students who enter 9th grade in NYC each year. Parag Pathak and I undertook the task, and quickly started talking to Atila Abdulkadiroglu and Tayfun Sonmez, who had written a paper on school matching that appeared in the June 2003 AER. In September 2003 the Boston Globe carried a story on that paper, mentioning the resident matching program as a possible alternative design, that led to us all meeting with Boston Public Schools in October 2003 about redesigning the Boston school choice system. Neil Dorosin (who used to work at NYCDOE) founded the non-profit Institute for Innovation in Public School Choice (IIPSC) to promote modern school choice technology. IIPSC's projects include helping design and implement school choice systems in New Orleans, Denver and Washington DC.

    New York City

    Abdulkadiroglu, Atila , Parag A. Pathak, and Alvin E. Roth, "The New York City High School Match," American Economic Review, Papers and Proceedings, 95,2, May, 2005, 364-367.

    Abdulkadiroglu, Atila , Parag A. Pathak, and Alvin E. Roth, "Strategy-proofness versus Efficiency in Matching with Indifferences: Redesigning the NYC High School Match,'' American Economic Review, 99, 5, Dec. 2009, pp1954-1978. And here are the AER links at which you can access the Appendix and Download the Data Software

    Boston

    The new school choice mechanisms went into operation in New York City in 2003 (for students entering high school in 2004), and in Boston in 2006 (for 2007 entry into grades K, 6, and 9). In April, 2006 Harvard hosted a conference on Designing Choice, for schools officials and others interested in choice from around the country. Here are two articles written for a broad audience interested in education and mathematics, respectively. MATCHMAKING: ENABLING MANDATORY PUBLIC SCHOOL CHOICE IN NEW YORK AND BOSTON , By Thomas Toch and Chad Aldeman, and School Choice by Joseph Malkevitch. Here's an August 2010 article about me in Forbes that focuses on school choice in NYC. Here's a May 2012 article about Parag Pathak that focuses on his work on schools and school choice: Game theory in the real world. Here is a self-updating link to my blog posts on school choice.



    Decentralized Markets, and market failures--congestion, unraveling, market culture

    Repugnant Markets:

    The paper on repugnant markets has been popular in the blogosphere and the press, e.g. here, and here, and here, and (earlier) here...even though I refrained from calling it "Ick-onomics"). It also formed the basis for a BBC Radio 4 broadcast on Repugnant Markets in July, 2007. Here's the transcript, (in which I seem to say "you know" a lot) and here's the half-hour long audio recording (in which you can judge for yourself), and some related BBC news stories here and here. And here's a related story in the Financial Times on kidney exchange by Tim Harford, who conducted the BBC interviews (and who also thinks about buying babies). Here's an interview on repugnance at HBS Working knowledge (their story includes a video of the North American Wife Carrying Championships). Here's a WSJ online discussion between Julio Elias and me of markets for kidneys, repugnance, and how kidney exchange seems not to arouse repugnance. The AMA ran a story in their Jan.28, 2008 issue of amednews.com on kidney markets, repugnance, and kidney exchange. Here's an April '08 Freakonomics blog connecting the discussion to the market for kidneys in Iran. There was an American Enterprise Institute symposium on Repugnance as a constraint on markets (with video and audio links) on 1/16/08: in Washington DC, the commentators were an interesting and diverse group: Sally Satel, Paul Bloom, and Michael Novak. It was followed by a Jan 31 NY Times article: Economists Dissect the 'Yuck' Factor. That article in turn was followed by this Freakonomics column: Repugnance Revisited, or: Are Economists Really 'Evil'? (I guess it's a tossup: evil or dismal?) Robin Young of NPR's Here and Now interviewed me about repugnant markets on March 25, 2008 in a piece called The "Yuck!" Factor (15 minutes). Here's a 15 minute interview on repugnance as a constraint on market design. On my market design blog I've found myself writing about repugnance quite a bit: here is a (self updating) link to all my posts on repugnant transactions (including this one on the Opposite of repugnance: Protected transactions , which got a nice plug here.) My paper "In 100 Years" includes some speculation on transactions whose repugnance may change one way or another in the future (and it got a nice plug here--apparently Stephen Dubner has an attraction to repugnance...:)

    Market for law clerks:

    The WSJ Law Blog ran an item on this paper that drew some interesting comments, as did the followups here and here (where they touch on issues Muriel Niederle and I discuss about Designing Rules for Offers and Acceptances.

    Here are the dates and rules in the Federal Judges Law Clerk Hiring Plan for 2012, but the plan, which has been on it's last legs for a surprisingly long time, may finally be dead: see my blog post The Federal Law Clerk Hiring Plan is pretty much over .

    AEA market for economists

    I am the chair of the American Economic Association's Ad Hoc Committee on the Job Market, whose other members are John Cawley, Peter Coles, Philip Levine, Muriel Niederle, and John Siegfried. We have been charged with making recommendations to improve the efficiency of the market for new economists.

    The first of our recommendations to be put into practice, in the Spring of 2006, is the Economics Job Scramble, a web page on which applicants and employers could indicate their continued availability as of late March. The second, which went online November 20, 2006, is that the AEA will facilitate Signaling for Interviews in the Economics Job Market, to allow applicants to send up to two signals to employers with whom they would like to interview. Here is some description and advice.(See also Greg Mankiw's blog on the subject, and the January 8, 2007 Wall Street Journal story "Job Hunting Takes a Line from Dating" that was reprinted in various non-subscription-required places as "Economists Learn Matchmaker Role".)

    Here's a report on the first few years of Committee activity, with a preliminary assessment of the new signaling and scramble facilities:


    eCommerce

    There's a growing body of work about online reputation mechanisms. See e.g. Chris Dellarocas, Axel Ockenfels, Paul Resnick, and the Reputations Research Network.

    Some illuminating attention has recently been paid to search-engine ad auctions: see the papers by Edelman, Ostrovsky, and Schwarz, and by Hal Varian.

    Here's Amy Greenwald's essay Game Theory and the Design of Electronic Markets.



    Other economic design (maybe not always of markets)

    We haven't hit on the unconstrained optimum terminology for our new field. I tried to spark the use of the term "design economics" in my 2002 manifesto , which might have more naturally included all the things that economists can help design (e.g. marketplaces, contracts, organizations, laws, treaties...). But languages are like economies (and both are like oceans to the extent that you can't hold back the tide), so I'm cheerfully resigned to having all sorts of economic design included under the heading of market design. I guess we're just taking a very broad view of what constitutes a market...:)

     


    Other Market Design Links

    Researchers in matching and market design: (game theorists, experimenters and computer scientists interested in matching, auctions and market design) Alphabetically by last name:

    Atila Abdulkadiroglu | David Abraham | Jose Alcalde | Ahmet Alkan | Itai Ashlagi | Susan Athey (Clark Medal winner, 2007) | Larry Ausubel | Chris Avery | Ken Binmore |Peter Biro |Anna Bogomolnaia | Eric Budish | Jeremy Bulow | Estelle Cantillon |Yiling Chen | Yan Chen | Kim-Sau Chung | Peter Coles| Vincent Conitzer | Peter Cramton | Vincent Crawford | Chris Dellarocas | Faye Diamantoudi | Ettore Damiano | Ben Edelman | Lars Ehlers | Federico Echenique | Aytek Erdil | Haluk Ergin | Itay Fainmesser | Clayton Featherstone | Guillaume Frechette | Li Gan | Daniel Grosu | Ernan Haruvy | John Hatfield | Nicole Immorlica | Rob Irving | Onur Kesten | Bettina Klaus | Paul Klemperer | Flip Klijn | Fuhito Kojima |Scott Kominers | Hideo Konishi | John Ledyard | Robin Lee | Jon Levin | Li, Hao | Jeffrey K. MacKie-Mason | Mohamad Mahdian | Mihai Manea | Jordi Masso | David Manlove | R. Preston McAfee | John McMillan | Paul Milgrom | Eiichi Miyagawa | Benny Moldovanu | Herve Moulin | Muriel Niederle | Noam Nisan | Axel Ockenfels | Michael Ostrovsky | Szilvia Papai | David Parkes | Parag Pathak | David Perez-Castrillo | Charlie Plott | Marek Pycia | Paul Resnick | Antonio Romero-Medina | Amir Ronen | Tuomas Sandholm| Michael Schwarz | Jay Sethuraman | Sven Seuken | Robert Shiller | Yoav Shoham| Tayfun Sonmez | Marilda Sotomayor | Wing Suen | Utku Unver | Hal Varian | Rakesh Vohra | Robert Wilson | Licun Xue |

     

    The European network for research on matching practices in education and early labour markets has a list of matching theorists and market designers in Europe, as well as some material on matching institutions in Europe.

     

    The computational game theory group at Microsoft Research has posted some papers on design.

    At the University of Michigan there is a research group on Incentive-Centered Design that includes a joint graduate program with Wayne State University. (I'll give a seminar there in September.)

    Penn State has a Center for the Study of Auctions, Procurements and Competition Policy (and their site includes a variety of data sets among other things).

    The 2007 Nobel Prize for "foundations of mechanism design theory" was awarded to Leonid Hurwicz, Eric S. Maskin, and Roger B. Myerson. Here are the very short, short, and longer announcements from the Nobel committee, and the video. While much of mechanism design theory is fairly abstract, it's easy to see how market designers can think of ourselves as doing "applied mechanism design," particularly when we think about fundamental concepts like incentive compatibility.

    Other markets (designed by other market designers:)

    In 1993, Congress authorized the Federal Communications Commission (FCC) to auction off licenses for radio spectrum (which had previously been allocated by bureaucratic means.) This was the birth of auction design as a big activity for economists, as many auction theorists and others got involved. The FCC maintains a lot of information on these auctions on their Wireless Telecommunications Bureau Auctions Home Page, including some background information, and some interesting uses of experimental economics, on their Experiments, Papers and Studies page, and some older papers from the Combinatorial Bidding conference in 2000. Other countries followed suit, making auction design an international activity. Here is the UK's Radiocommunication Agency's spectrum auction page.

    Many of the most prominent auction designers involved in spectrum and electricity auctions formed the firm Market Design Inc, which has a long and varied collection of auction markets they have designed or advised and a bibliography of papers (mostly from the 1990's). Some of the economists involved have formed other companies to conduct auctions, notably Auctionomics and Power Auctions LLC .

    Economic Principals has a nice article about Paul Milgrom and his book Putting Auction Theory to Work, in the context of designing spectrum auctions.

    Peter Cramton, Yoav Shoham and Richard Steinberg have edited a volume on Combinatorial Auctions.

    You can download Paul Klemperer's book Auctions: Theory and Practice from his home page.

    Vijay Krishna has a textbook called Auction Theory.

    Scottish doctors needed to be scheduled as well as matched, and The Scottish PRHO Allocations Scheme describes a recently implemented algorithm for doing so, designed by Rob Irving at the University of Glasgow. (Earlier incarnations of this and other British markets are discussed in Roth, A.E. "A Natural Experiment in the Organization of Entry Level Labor Markets: Regional Markets for New Physicians and Surgeons in the U.K.," American Economic Review, vol. 81, June 1991, 415-440. Irving and David Manlove have a page on their work on Stable Matching Algorithms.

    Peter Biro has a page of matching applications from around the world.

     

    Michael Kremer has been doing on a lot of work on the design of markets for vaccines.

    In October 2007 I hosted a conference organized by Ecotrust on Market Design for Limited Access Programs in U.S. Fisheries.

     

    Financial markets are a rich source of market design, and I particularly admire the work of Robert Shiller: His web pages on what he calls Financial Democracy , and its associated references and links , are well worth looking at.

     

    Ad Search Auctions: There's been a lot of work on ad auctions, spurred in part by the 2007 AER paper by Ben Edelman, Michael Ostrovsky, and Michael Schwarz. Here's a paper by Susan Athey and Glenn Ellison that looks at ad auctions as an integral part of search.

    Here are some papers presented at a July 2008 Workshop on Ad Auctions, and here's one by Muthu Muthukrishnan.


    Class notes and materials:

    Susan Athey teaches an undergrad class on market design at Harvard: here is her syllabus.

    Peter Bossaerts has a course called Designing Market-Based Solutions to Allocation and Communication Problems at Ecole Polytechnique Federale de Lausanne.

    Peter Cramton teaches a market design class at Maryland.

    Fuhito Kojima taught a Matching and Market Design class at Yale, and more recently taught this one at Stanford. (He also posts some short courses he's given.)

    Kate Larson has a course on Electronic Market Design at Waterloo.

    Noam Nisan teaches a class called Foundations of Electronic Commerce.

    David Parkes has a class on Computational Mechanism Design

    Tuomas Sandholm teaches Foundations of Electronic Marketplaces at CMU

    Leigh Tesfatsion maintains pages on Electricity restructuring and market design.

    Markus Walzl has a course on Mechanism and Market Design.

    (And here's another link to my market design class at Harvard.)

     

    Thomas Kittsteiner and Axel Ockenfels have a review paper: Market Design: A Selective Review. Zeitschrift f�r Betriebswirtschaft, Special Issue 5 (2006), 121-143.

    Software:

    Greg Barron has put online the Excel implementation of the (Gale-Shapley) Deferred Acceptance Algorithm he built for the IMF's Economist Program.

     

    Conferences:

    NBER Market Design Group Conference, October 8 and 9, 2010.

    Return to Table of Contents

    Experimental Economics

    The 2002 Nobel Prize in Economics was awarded to Danny Kahneman and Vernon Smith.

    Here are two papers on the early history of experimental economics (and its close relation to game theory):

    Roth, A.E., "On the Early History of Experimental Economics," Journal of the History of Economic Thought, 15, Fall 1993, 184-209.

    Francesco Guala's brief History of Experimental Economics.



    The Handbook of Experimental Economics, Volume 1
    John H. Kagel and Alvin E. Roth, editors,
    Princeton University Press, 1995. Paperback edition, Fall 1997.

    Table of Contents
    (click on chapter headings for detailed contents)
    Preface
    1. Introduction to Experimental Economics by Alvin E. Roth
    2. Public Goods: A Survey of Experimental Research by John O. Ledyard
    3. Coordination Problems by Jack Ochs
    4. Bargaining Experiments by Alvin E. Roth
    Bibliography of bargaining experiments.
    5. Industrial Organization: A Survey of Laboratory Research by Charles A. Holt
    6. Experimental Asset Markets: A Survey by Shyam Sunder
    7. Auctions: A Survey of Experimental Research by John H. Kagel
    8. Individual Decision Making by Colin Camerer

    Reviews (from the hardcover bookjacket) by Ken Binmore, Robert Gibbons, Daniel Kahneman, Richard Thaler, and Ariel Rubinstein, and (from the paperback bookjacket) by Katerina Sherstyuk and F. van Winden.

    Kenneth C. Williams of Michigan State University wrote the following review in the American Political Science Review, Sept. 1996, Volume 90, pp632-633. Permission to web publish

    Ordering information from PUP for U.S. and overseas customers.

    The Handbook of Experimental Economics, Volume 2
    John H. Kagel and Alvin E. Roth, editors, in preparation

    Announcing the Handbook of Experimental Economics Vol 2. to be published by Princeton University Press; J. Kagel and A. Roth (eds).

    We are pleased to announce the plans for volume 2 of the Handbook of Experimental Economics. The idea is to cover topics that have seen substantial growth since (or were not covered in) the publication of vol 1 as well as to update several of the more active subject areas. As with vol 1 the emphasis will be on series of experiments showing the back and forth that whittles down what we do not know.

    The list of contributors and authors is listed below (with links to chapters as they become ready for public review and comment).

    1. Intro: the last ten+ years (Roth)

    3. Political economy (Palfrey)

    4.Voluntary Giving to Public Goods by Lise Vesterlund (updated February 10, 2011, taken down for revision)

    6. Learning and the Economics of Small Decisions, by Ido Erev and Ernan Haruvy updated, February 1, 2012

    7. Field Experiments: "Psychology and Economics in the Field" (TBD)

    8. Neuroeconomics by Colin Camerer, Jonathan Cohen, Ernst Fehr, Paul Glimcher and David Laibson, updated November 2011

    9. Other-regarding preferences: A Selective Survey of Experimental Results by David J. Cooper and John H. Kagel.

    10. Auctions: A Survey of Experimental Research, 1995 – 2008 by John H. Kagel and Dan Levin

    11. Macroeconomics: A Survey of Laboratory Research, by John Duffy

    12.Experiments in Market Design (Jan 2012 draft) (Roth)

    Penultimate versions of the chapter contributions will be presented on July 14-16, 2007 at the Stony Brook Game Theory festival. We invite interested members of the research community to attend the Game Theory festival which will include many more sessions than just this one (see the Stony Brook web site for topics to be covered). We also invite members of the research community to send suggestions to the authors of the relevant chapters.



     

    Prediction

    Together with Ido Erev, I am interested in quantitative prediction, and experimental behavior is a good domain on which to explore ideas. The following paper grew out of our work on game theory:

     

  • Erev, Ido, Alvin E. Roth, Robert L. Slonim, and Greg Barron, '' Learning and equilibrium as useful approximations: accuracy of prediction on randomly selected constant sum games ,'' Economic Theory , special issue: Behavioral Game Theory Symposium, 33, 2007, 29-51.
  •  

    More recently, Ido instigated the first of what will likely be several prediction tournaments. Here's the report of the first one.

    And here's the introduction to the second and third, which, along with the competition winners and other related papers will appear from time to time online in : Games, Special Issue on Predicting Behavior in Games

     

    Return to Table of Contents

    Other labs and experimenters (with miscellany at the end)

    The Laboratoire d'Economie Experimentale de Montpellier maintains a long list of  The experimental labs in the world.

    Below is a less comprehensive list of experimenters at some of those labs, alphabetically by University.

    The Scottish Experimental Economics Laboratory at the University of Aberdeen (where you can find Jurgen Bracht, Miguel Costa-Gomes, Erika Seki, and Joe Swierzbinski.)

    The UAA Experimental Economics Laboratory at the University of Alaska, Anchorage.

    Center for Research in Experimental Economics and Political Decision-Making at the University of Amsterdam.

    Juan Camilo Cardenas at Universidad de los Andes.

    Todd Cherry, David Dickenson, and Tanga M. McDaniel at Appalachian State University.

    Economic Science Lab at the University of Arizona., directed by Martin Dufwenberg

    David Reiley is at the University of Arizona and Yahoo! (He notes: "I am the economist formerly known as David Lucking-Reiley.")

    A list of Australian experimenters is on the web site of the ARC Economic Design Network

    Ananish Chaudhuri at the University of Auckland, NZ

    Brit Grosskopf is now at the University of Birmingham

    Laboratory of Experimental Economics at the University of Bonn, founded by Reinhard Selten, directed by Armin Falk.

    There's now an active experimental group at Brown University.

    A stellar group of diverse experimenters at the University of California, Berkeley, runs the experimental social science lab called XLab

    The Laboratory for Experimental Economics and Political Science at Cal Tech is directed by Charlie Plott. Tom Palfrey and David Levine (at UCLA) direct the California Social Science Experimental Laboratory at UCLA. John Ledyard has a number of his experimental papers up on his page, including several addressed towards questions of market design.

    Gary Charness at the University of California, Santa Barbara has links to quite a few of his papers.

    Dan Friedman and his colleagues at the University of California, Santa Cruz, have some pages on Learning and Experimental Economics.

    Jim Andreoni, Vince Crawford and Uri Gneezy are at the University of California, San Diego.

    Glenn Harrison is at the University of Central Florida

    Andreas Ortmann is at Charles University.

    Axel Ockenfels is at the University of Cologne.

    Dick Thaler and George Wu are at the University of Chicago's School of Business. John List has recently moved to Chicago's economics department, from the University of Maryland, where his Field Experiments site still resides.

    Dan Ariely at Duke University and the Center for Advanced Hindsight has some really novel experiments.

    There is an active program in experimental economics at the University of East Anglia, including two of the field's pioneers, Graham Loomes and Robert Sugden.

    Florida State has a big group of experimenters, including David Cooper, Mark Isaac, Svetlana Pevnitskaya, and Tim Salmon.

    Oliver Kirchcamp is at Friedrich-Schiller-Universit�t Jena.

    Georgia State University's Experimental Economics Center is growing.

    Here at Harvard there are experimenters (both in the lab and field) all over: when I get organized I'll link to some more of their web pages: Nava Ashraf, Greg Barron, Eyal Ert, David Laibson, Sendhil Mullainathan, ... David Laibson has written a letter to incoming Harvard graduate students listing Harvard faculty with behavioral/experimental interests.

    Hewlitt Packard has a Game theory and experimental economics research group and a Social Computing Lab.

    The Center for Experimental Business Research at the Hong Kong University of Science and Technology has an elaborate site with many links. See also the personal page of the lab's director, Rami Zwick

    On Sunday, 25 June 1995, I had the privilege of participating in the inauguration of the new experimental lab at the Hebrew University of Jerusalem--RatioLab. The directors are Professors Gary Bornstein, Eyal Winter, and Shmuel Zamir.

    Jordi Brandtsis at the Institute for Economic Analysis in Barcelona.

    Bob Forsythe is one of the founders of the very active experimental group at the University of Iowa.

    The Center for Experimental Economics at Leopold-Franzens University of Innsbruck.

    Georg Weizsäcker is at the London School of Economics.

    The experimental lab at the University of Mannheim was the host to the ESA conference in June '98.

    There's an experimental economics program at the University of Massachusetts at Amherst.

    Werner Guth directs the Max Planck Institute of Economics in Jena.

    Jim Engle-Warnick is at McGill, in Montreal, where there is an active inter-University lab called Cirano.

    Experimental Economics Laboratory at McMaster University is directed by Stuart Mestelman and Andrew Muller.

    The Mississippi Experimental Research Laboratory is at 'Ole Miss' directed by Mark Van Boening.

    Monash University has an active experimental economics group that includes Klaus Abbink, Nick Feltovich, Lata Gangadharan, Phil Grossman, Andreas Leibbrandt, Vai-Lam Mui, Birendra Rai, and Anmol Ratan.

    Experimental Economics Research Program at the University of New Mexico: Philip Ganderton, David Brookshire, Stuart Burness.

    Ben Greiner is at University of New South Wales in Sydney.

    Andy Schotter and Guillaume Frechette are at NYU.

    The Centre for Decision Research and Experimental Economics at U. Nottingham is directed by Chris Starmer. Robin Cubitt, Simon G�chter and Martin Sefton are among the experimenters there.

    John Kagel, Dan Levin, Lixin Ye and (most recently)Luke Coffman, are part of the active program of Experimental Economics Ohio State University.

    Tatsuyoshi Saijo at Osaka University makes some of his papers available, in English and Japanese.

    At Penn State, the active laboratory for economic management and auctionsis the home of Gary Bolton, Elena Katok, Anthony Kwasnica, and Abdullah Yavas.

    Andreas Blume, John Duffy, Jack Ochs, and Lise Vesterlund contribute to making Pittsburgh a center of experimental economics.

    The Universitat Pompeu Fabra in Barcelona has the Laboratori d'Economia EXperimental, which has become one of the most visible in the world, with a big group of affiliated faculty including my former postdoc Rosemarie Nagel, and Antoni Bosch, who links to this poem.

    Tim Cason and is keeping the experimental tradition alive at Purdue.

    Alistair Munro and Dirk Engelmann are at Royal Holloway, University of London.
    The University of Siena has an Experimental Economics Laboratory

    Muriel Niederle is at Stanford.

    Pablo Guillen and Bob Slonim are at the University of Sydney.

    Ido Erev is at the Technion in Haifa.

    John Van Huyck is among the experimenters at Texas A&M University.

    Catherine Eckel and Ernan Haruvy are experimenters at the University of Texas at Dallas

    CentER at Tilburg University has an experimental facility called Centerlab. Jan Potters has some of his experimental papers up on the web.

    The Computable and Experimental economics lab at the University of Trento has a page describing their activities.

    The Laboratorio de Investigación en Economía Experimental is a joint project of two universities in Valencia.

    Charlie Holt at the University of Virginia is a leading experimenter, and his web site has material both on his experimental research, and on his uses of experiments in teaching economics.

    Douglas Davis and Laura Razzolini are experimenters at Virginia Commonwealth University.

    Ananish Chaudhuri at Washington State University has a page for his work and his class on Behavioral Economics.

    Jim Andreoni at the University of Wisconsin has some of his papers and (soon to come) data up on the web. (Check out some of his academic children.)

    Owen Phillips and Jason Shogren are at the University of Wyoming.

    The Centre for Experimental Economics at the University of York. is directed by Miguel Costa-Gomes. John Hey founded the lab there.

    Ernst Fehr and Bruno Frey and Urs Fischbacher have web pages representing the active research program at the University of Zurich.

    For names and addresses of experimental economists, a good source is the directory maintained by the Economic Science Association. The endearingly named European Network for the Development of Experimental Economics and its Application to Research on Institutions and Individual Decision Making has contact information for many European experimenters, as does the Gesellschaft für Experimentelle Wirtschaftsforschung (Society for Research in Experimental Economics), including a big list of European labs, and a survey (in English) of experimental software and advice by Oliver Kirchcamp.. A group of like-minded psychologists is the Society for Judgment and Decision Making

    A page on The Economic Behavior of Children by Bill Harbaugh and Kate Krause contains some experiments involving children, and a bibliography.

    A page on Experiments on families is maintained by Alistair Munro.

    There is growing interest in experimental methods among researchers in Operations Management.

    Return to Table of Contents

    Things to participate in:

    The Technion Prediction Tournament has been concluded, for simple models that will predict choice of a riskless alternative versus a simple lottery, under one of three conditions (1. Description; 2. Experience sampling; 3. Experience repeated.) Here's a working paper on the competition results: A choice prediction competition, for choices from experience and from description A choice prediction competition, for choices from experience and from description

     

    Conference: 3rd LeeX International Conference on Theoretical and Experimental Macroeconomics in Universitat Pompeu Fabra. June 18-19, 2012. Keynote speakers are: Shyam Sunder, Yale University, and Ramón Marimón, European University Institute. The conference organizers are: John Duffy, University of Pittsburgh, Frank Heinemann, Technical University of Berlin, Rosemarie Nagel , ICREA and Universitat Pompeu Fabra. Deadline Friday, 16 March, 2012.

    Summer school:  Fifth Barcelona LeeX Experimental Economics Summer School in
    Macroeconomics, BLESS-M-2012
    , June 11-15, 2012 in Universitat Pompeu Fabra. The summer school organizers and lecturers are: John Duffy, University of Pittsburgh Frank Heinemann, Technical University of Berlin, Rosemarie Nagel, ICREA and Universitat Pompeu Fabra, and guest lecturers are Charles Noussair, Tilburg University, and  Shyam Sunder, Yale University. Deadline Sunday, 1 April 2012.

    Experiments at Harvard, experiments on the web, tournaments, conferences

    Sign up here for economics experiments at Harvard's Computer Lab for Experimental Research

    Experiments on the web

    Iowa electronic markets. trade a variety of contracts, both financial and political, which change over time. One of the research interests of this long running set of investigations is the ability of markets to forecast events. (See also the related Austrian Political Stock Markets in Vienna, and UBC Election Stock Markets in Canada.) Conferences

    Experiments in the Classroom

    Moblab makes available free software for teaching economics with experiments. (I'm one of their advisors.)

    Charlie Holt the veteran experimenter at the U. of Virginia is also a leader in bringing experimental games into the classroom, and has some thoughts and resources on the subject here

    Here is John Hey's class Introduction to Experimental Economics.

    Martin Shubik at Yale has a 2001 paper on the use of "teaching games"

    Shyam Sunder has some material from his class in Experimental Economics at Yale SOM.

    A newsletter devoted to bringing experiments into the classroom for instructional purposes, Classroom Expernomics was published for several years by Greg Delemeester at Marietta College in Ohio, and John Neral at Frostburgh State University in Maryland.

    The Fall 1993 issue of the Journal of Economic Education is concerned with classroom experiments.

    A more recent article in that Journal is Asker, John, Brit Grosskopf, C. Nicholas McKinney, Muriel Niederle, Alvin E. Roth and Georg Weizs�cker, "Teaching auction strategy using experiments administered via the Internet," Journal of Economic Education, 35, 4, Fall 2004, 330-342.

    The Royal Economic Society had a session on Experimental Economics for Education in the Classroom and Beyond.

    A nice description of studying experimental economics, from Experiments with Economic Principles by Ted Bergstrom and John Miller, is

    "Taking a course in experimental economics is a little like going to dinner at a cannibal's house. Sometimes you will be the diner, sometimes you will be part of the dinner, sometimes both." Their page describes the book, and includes links to some courses that use it.

    Marko Grobelnik , Robert A. Miller and Vesna Prasnikar have posted some downloadable software (for matrix games and extensive form games) for running classroom experiments.

    Return to Table of Contents

    Experimental Philosophy, and Philosophy of Experimental Economics and Game Theory

    Here are some links to philosophers who do experiments, to philosophers who think about game theory, and to philosophers of science who think about experimental economics and game theory.

    Cristina Bicchieri

    Francesco Guala (whose links include a list of other philosophers of economics and a Philosophy of Experimental Economics Bibliography)

    Daniel Hausman

    Joshua Knobe (now at Yale) has an Experimental Philosophy page and coordinates the Experiments in Philosophy blog (hosted by Psychology Today).

    Thomas Nadelhoffer coordinates the Experimental Philosophy blog.

    Brian Skyrms

    Appiah's December 2007 address "Experimental Philosophy"

     

    An article in Prospect Magazine summarizes some of the discussion about the place of experiments in Philosophy: Philosophy's great experiment. (I'm reminded of early experimental economics...) Speaking of which, here's a conference in March 2010 in Kyoto called How and why economists and philosophers do experiments, with distinguished speakers from both disciplines.

     

    Here are some critiques of experimental economics [Updated 2/12/10] that I initially compiled for my class in Fall 2005, and have sporadically updated, including contributions by Gul and Pesendorfer, Gale, Levitt and List, Shaked, and Rubinstein, along with some replies, and some critiques of earlier vintage. (I am old enough to remember when no one felt moved to write criticisms of experimental econ:-)

    Return to Table of Contents

    Conduct of Science

    In the general scientific community, particularly in the highly competitive (for credit, for grants, etc.) biological sciences, there has started to be a good deal of discussion about what constitutes improper conduct in (and of) science. Not surprisingly, it turns out to be hard to set simple rules for complex matters. But the occasional hair-raising abuse makes it clear that we have an obligation to at least think about these issues (and make sure that our graduate students are aware of them).

    The following links are discussions of scientific conduct. While the examples aren't drawn from economics, some of the issues (such as the allocation of credit among researchers) will be relevant even to theorists, and many more will be relevant to experimenters and other empirical types.
    On Being A Scientist: Responsible Conduct In Research Committee on Science, Engineering, and Public Policy, of the National Academy of Sciences, National Academy of Engineering, and Institute of Medicine.
    The National Academies Press also has online an older book Responsible Science, Volume I: Ensuring the Integrity of the Research Process (1992)
    See also the editorial Ambiguity in the Practice of Science by Frederick Grinnell, in the 19 April 1996 issue of Science.
    David Goodstein of Cal Tech points out that Scientific Misconduct as it is practiced is less likely to be outright fraud than misconduct as a referee, etc.
    See an annotated bibliography of scientific misconduct, compiled for psychologists.

    The Newsletters of the U.S. Department of Health and Human Services Office of Research Integrity have chilling "Case Summaries" on research misconduct at the end of (it seems) every issue.

    A bigger issue than scientific misconduct is how scientists should avoid fooling themselves. This is one of the important uses of good experimental design. A famous early lecture on erroneous results in the physical sciences has been transcribed as Langmuir's talk on Pathological Science (December 18, 1953).

    The WSJ ran a nice book review of Books on Scientific Fraud, covering the (fraudulent) work of Paul Kammerer, in Arthur Koestler's book "The Case of the Midwife Toad;" the plagiarist "Elias Alsabati, who stole dozens of papers by other scholars and republished them as his own," in the book "Betrayers of the Truth" by William Broad and Nicholas Wade; cold fusion in the book "Voodoo Science" by Robert Park, and various fraudulent bits of medical science in "The Patchwork Mouse" by Joseph Hixson and "The Threat and the Glory" by P.B. Medawar. (Medicine has a particularly problematic incentive structure, and seems to have more than its share of scientific misconduct, perhaps because the stakes are so high.) A book review Physics and Pixie Dust, of the 2009 book " PLASTIC FANTASTIC: How the Biggest Fraud in Physics Shook the Scientific World," by Eugenie S. Reich covers the prolific fraud of the (supposed) physicist Jan Hendrik Schön, a researcher at Bell Laboratories who apparently fabricated dozens of papers reporting astonishing results in a short time. (" At Schön's peak, he was submitting 4 or 5 articles per month, most of them going to top journals like Science and Nature . ")

    See also my paper, "Let's Keep the Con Out of Experimental Econ.."

    Return to Table of Contents

    Game Theory

    Learning, and cognitive game theory




    Other game theory servers:

    Other pages on which game theory intersects with experiments

    Gary Bolton and Elena Katok at Penn State.
    Yan Chen at the University of Michigan.
    David Cooper at Florida State
    Vincent Crawford at UCSD.
    Martin Dufwenberg, University of Arizona.
    John Duffy at Pitt.
    Drew Fudenberg at Harvard.
    David Levine at UCLA and Washington U.
    John Morgan at Berkeley.
    Matt Rabin at Berkeley, already has some of his papers on procrastination posted, and on fairness.
    Bob Slonim at Case Western Reserve University.
    Dale Stahl at the University of Texas.
    Axel Ockenfels at the University of Cologne.
    Mark Walker and John Wooders, both at the University of Arizona.
    Bill Zame at UCLA.

    Ken Binmore is director of a project called Economic Learning and Social Evolution.

    The Center For Rationality at the Hebrew University of Jerusalem , including their experimental lab, RatioLab, and "Beehave" , the bee behavior laboratory, opened in 1994, which uses bumblebees foraging on artificial flowers as a model system to study interactive decision-making in animals.


    Return to Table of Contents

    Game Theory Journals

    The International Journal of Game Theory has a home page with a link to forthcoming articles.

    Games and Economic Behavior has a web page from which you can see tables of contents.

    The International Game Theory Review has a web page from which you can download papers.

    The journal Experimental Economics also has a web page.

    A new online game theory journal called Games made a 2010 appearance.

    Return to Table of Contents

    Evolution, evolutionary game theory, learning, and dynamics

    Greg Barron at Harvard Business School and Ido Erev at the Technion are doing some very exciting work on learning, and have put up some software on Greg's page.

    Ted Bergstrom's page has some papers on evolutionary models, and on matching models of the economics of the family.

    Speaking of evolution, Carl Bergstrom is a biologist interested in game theoretic aspects of evolution, particularly Signalling Theory and Animal Communication.

    Josef Hofbauer of the University of Vienna provides links to many of his papers.

    John Miller is a longtime student of adaptive models.

    Martin Nowak heads Harvard's Program for Evolutionary Dynamics.

    Arthur Robson has moved to Simon Fraser University.

    Avner Shaked at Bonn explores a variety of approaches.

    Karl Sigmund at the University of Vienna.makes many of his papers available.

    Leigh Tesfatsion maintains a page on Agent-Based Computational Economics concerning evolutionary and artificial-life models of economic environments, and pages on Electricity restructuring and auction design.

    VirtualLabs in evolutionary game theory has some tutorials.

    The Minority Game's Web Page is a collection of papers, mostly by physicists, on adaptive behavior in a market entry game (you have to choose "in" or "out" and you get a higher payoff if you are in the minority.) (This seems to have been carried out quite independently of the experimental work on market entry using similar games, but looks worth careful study.) It has been assembled by Damien Challet of the Institut de Physique Théorique, Université de Fribourg.

    Return to Table of Contents

    Game theory in computer science

    Computer scientists are doing more and more game theory, and economists and computer scientists have a lot to learn from each other. Here are some CS sites (in addition to those linked to above, under market design, namely: David Abraham , Vincent Conitzer, Daniel Grosu, Nicole Immorlica, Rob Irving, Mohamad Mahdian, David Manlove, Noam Nisan, David Parkes, Amir Ronen, Tuomas Sandholm, Yoav Shoham ).

    The Algorithmic Game Theory Group at Microsoft Research

    Avrim Blum is at CMU.

    The Economics Group of the Electrical Engineering and Computer Science Department at Northwestern.

    Joan Feigenbaum's class on Economics and Computation at Yale.

    Amy Greenwald's class on Game-Theoretic Artificial Intelligence at Brown.

    Joe Halpern has an encyclopedia entry on Computer science and game theory: A brief survey

    Adam Tauman Kalai's Spring 2008 Game Theory and Computer Science class at Georgia Tech.

    Michael Kearns has notes from a 2003 class on Computational Game Theory at Penn

    Nimrod Megiddo at IBM Almaden

    Noam Nisan's blog, Algorithmic Game Theory

    Christos H. Papadimitriou's seminar on Algorithmic Aspects of Game Theory at Berkeley.

    Tim Roughgarden's course on Algorithmic Game Theory at Stanford, a June 2008 survey paper, and his course on Foundations of Sponsored Search.

    Aaron Roth is at CMU.

    Eva Tardos' Algorithmic Game Theory class at Cornell.

    Moshe Tennenholtz at the Technion.

    Vijay Vazirani at Georgia Tech.

    There's an edited volume on Algorithmic Game Theory (with a nice foreword by Papadimitriou) [the preface to the book reveals that the full text is online at http://www.cambridge.org/journals/nisan/default.asp, using username: agt1user and password: camb2agt]

    Chandra Chekuri at the University of Illinois has a nice set of course notes and links for Algorithmic Game Theory

    David Dowe's chess and games page includes many links to work on game playing programs.

    Helger Lipmaa maintains a list of links that emphasize game theory and cryptography.

    Fr�d�rick Asselin maintains a list of researchers in Game Theory or Economics and Computer Science

    Muthu Muthukrishnan has a blog that often touches on search auction design and related topics.

     

    There's a Prize in Game Theory and Computer Science (in honor of Ehud Kalai)

     

    There are some agent-based competitions:

    The Project on Market Based Control of Complex Computational Systems runs a CAT Tournament (or TAC Market Design Tournament) meant to be the opposite of the TAC (Trading-Agent Competition). In the TAC, the organizers create a market and the competitors create trading agents, while in the CAT the organizers create trading agents and competitors develop "specialist" marketplaces for them to trade in.

    Return to Table of Contents

    Game theory in the classroom

    (This section is very much a work in progress--suggestions are welcome.)

    Martin Osborne at the University of Toronto, has, among other interesting items, corrections to his two textbooks coauthored with Ariel Rubinstein, and to his undergraduate textbook.

    Ariel Rubinstein, at Tel-Aviv and NYU, has initiated a site for Didactic Web-Based Experiments in GAME THEORY, at which teachers may post exercises in which their students can participate.

    Adam Brandenburger (who has posted teaching notes on game theory) and Barry Nalebuff have a page for their book about business and game theory, Co-opetition.

    There are online lecture notes and other resources at Gametheory.net

    Kenneth N. Prestwich at the College of the Holy Cross, Worcester, MA, has made available his course material on Evolution and Game Theory, including interactive Java applets. He writes that the site is "aimed primarily at undergraduates with a serious interest in animal behavior and evolution."

    Arthur Caplan at Utah State University has some software for presenting the geometry of transferable utility games.

    A set of links to Online Text and Notes in Game Theory is provided by the UK's Economics Network.

    Jerry Kelly at Syracuse posts lecture notes and related reading for his microeconomics/ game theory class.

    Game Theory with Professor Ben Polak at Yale.

    (And, for those who missed them at the top of the page, my Fall 2006 course Experimental Economics and Spring 2008 Market Design.)

    Return to Table of Contents

    Other servers devoted to game theory (including some blogs)

    For Paul Walker's outline History of game theory see history of game theory (updated to reflect the 2005 Nobel award to Bob Aumann and Tom Schelling).

    Harold Kuhn's Introduction to the Commemorative Edition of von Neumann and Morgenstern's Theory of Games and Economic Behavior.

    Combinatorial Game Theory: Some game theory with connections to operations research and computer science can be found at Combinatorial Game Theory maintained by David Eppstein at UC Irvine. Aaron Siegel offers a Combinatorial game suite of software tools.

    International Society of Dynamic Games, with issues of their newsletter, and links to some members and officers at Helsinki University of Technology and elsewhere. Tamer Basar includes some links to the prefaces and contents of the books he has written and edited.

    There is a POOL Listing of recent working papers on game theory from around the world (to which you can contribute your own),

    The Coalition Theory Network is devoted to research on coalition formation, and has links to a newsletter and conferences.

    Mikhael Shor at Vanderbilt maintains a wide ranging site called Game Theory.net that includes lecture notes (to which you are invited to contribute), and reviews of books and movies with a game theory connection.

    Ted Turocy at UT Austin and Andy McLennan, make available some downloadable software called Gambit, for finding various kinds of equilibria. (This was a project begun with the late Dick McKelvey at Cal Tech.)

    Holger Ingmar Meinhardt of the University of Karlsruhe has posted software for finding a variety of solutions to TU Games.

    There is an active Seville Game Theory Group

    There are now some game theory blogs, or at least blogs run by game theorists:

    Algorithmic Game Theory, Noam Nisan's blog.

    Cheap Talk, by Jeff Ely and Sandeep Baliga

    Leisure of the Theory Class by Eran Shmaya and Rakesh Vohra

    Market Design by Peter Coles and me (Al Roth:)

    Return to Table of Contents

    Other game theorists, and miscellaneous game theory connections

    Sylvia Nasar of the NY Times wrote a biography of John Nash called "A Beautiful Mind," that was subsequently made into a movie, on the occasion of his sharing the Nobel Memorial Prize in Economic Science in 1994 with John Harsanyi and Reinhard Selten. The Nobel link includes brief autobiographies of all three laureates. There is a PBS special and accompanying web page on Nash's life, called A Brilliant Madness. Harold Kuhn and Sylvia Nasar have edited a volume called The Essential John Nash (Here is Kuhn's Preface)

    Here is a brief tribute to Selten in my foreword to two volumes of Selten's collected works.

    Robert Aumann and Thomas Schelling shared the Nobel Memorial Prize in Economic Science in 2005. (Aumann's publications are available on the web).

    Yet another giant is Bob Wilson, whose students put together a festschrift in honor of his 65th birthday in May of 2002: Game Theory in the Tradition of Bob Wilson, edited by Bengt Holmstrom, Paul Milgrom, and Alvin E. Roth.

    A number of game theory papers and abstracts are available from the webpages of the
    Center for mathematical studies in economics and management science at Northwestern, and from the
    CentER for Economic Research at Tilburg.

    There is a book on Game Theory and the Law by Baird, Gertner, and Picker.

    The Game Theory Society posts links to members.

    The International Society of Dynamic Games (ISDG) posts links to conferences past and future.

    Some game theory papers can be found on the home pages of
    Luca Anderlini at (the other) Cambridge.
    Susan Athey at Harvard (the winner of the 2007 Clark Medal). In addition to posting some of her academic research, she has some advice on applying to grad schools.
    Ian Ayres at the Yale Law School (who employs game theory to argue about (among other things) discrimination and diversity, including applications to the FCC auction, and to a field experiment concerning bargaining in the automobile market).

    Salvador Barbera is one of the pillars of economics in Barcelona.
    Pierpaolo Battigalli at the European University Institute has a number of his papers posted in pdf format.
    Jesús Mario Bilbao at the University of Seville, has posted versions of both preprints and reprints, in various formats and languages.

    Adam Brandenburger at NYU Stern.
    Michael Chwe (an economist at Chicago).
    Peter Cramton at Maryland has posted lots of his papers on auctions and bargaining, as well as a good collection of auction and other links.
    John Hillas at Auckland.
    Matt Jackson is at Stanford.
    Paul Klemperer, the Edgeworth Professor of Economics at Oxford, posts some of his papers and books.
    Bettina Klaus is interested in fair allocation and matching procedures.
    Daphne Koller(a Stanford computer scientist, some of whose work is in game theory.)
    Vijay Krishna is at Penn State.
    Bart Lipman at Boston University has posted some of his papers on knowledge, and bounded rationality.
    George Mailath at Penn has quite a few interesting papers posted.
    Leslie Marx at Rochester has some of her papers posted.
    Andy McLennan at the University of Queensland has some of his papers on the web, along with some downloadable software for finding various kinds of equilibria.
    Nimrod Megiddo is at the IBM research center, and does work on the interface of computer science and game theory.
    Alexander Mehlmann's page exhibits his sense of humor and some of his work, including links to several of his books in both German and English.
    stephen morris, now at Princeton, is prolific.
    Rebecca Morton at NYU applies game theory to political science.
    Abhinay Muthoo at Essex has started a page that includes some information about his forthcoming book on bargaining.
    Robert Nau, at Duke, has some papers on common beliefs and correlated equilibria.
    Zvika Neeman, in Economics at BU, has posted papers on a wide variety of game-theoretic topics.
    Yaw Nyarko is at NYU.
    Barry O'Neill, at UCLA uses game theory in innovative ways in political science.
    Michael Ostrovsky, who works on matching and many other things, is at Stanford.
    Motty Perry at the Hebrew University of Jerusalem has some of his recent papers posted.
    Eric Rasmusen, at the Indiana University School of Business, has some papers, abstracts, and material from his textbook.
    Ariel Rubinstein has a very nice page.
    Larry Samuelson is now at Yale.
    David Schmeidler of Tel Aviv and Ohio State has some of his papers posted.
    Marciano Siniscalchi is at Princeton has posted a nice group of his papers on rationalizability and related subjects.
    Hyun Song Shin is at Oxford.
    Lones Smith at Michigan has posted some of his papers, a research manifesto, and some music...
    Ran Spiegler is at University College London.
    William Thomson, at the University of Rochester, has put the abstracts of a number of his papers up on the web.
    Eric van Damme has some downloadable papers linked to his publication list.

    Bernhard von Stengel at the London School of Economics has many papers on computational game theory.
    Joel Watson at UCSD has some of his papers on bargaining, reputation, and cooperation in web friendly form.

    Shmuel Zamir at the Hebrew University is the last game theorist I know in alphabetical order (but one of the first game theorists I knew).

    The 21st International Conference on Game Theory at Stony Brook University from July 12-16, 2010, is in honor of Bob Aumann's 80th birthday, and will have a day of talks in Honor of Shmuel Zamir's 70th birthday.


    A page on the work of the sociologist Jon Elster is maintained by Hans Olav Melberg. Some of Elster's work should be of interest to game theorists; I like Elster's work on institutions of local justice, and wrote a book review of his first book on the subject.

    An interesting page on Machine Learning in Games is maintained by Jay Scott at Drexel.

    There's even a band named Game Theory, and a Game Theory Guild for World of Warcraft players. And here's a game theory cartoon, about Nagel's guessing/beauty contest game: a famous experiment in game theory .

    Return to Table of Contents

    Bibliographies

    Bibliography of matching and market design. This bibliography began life as the bibliography of Roth, A.E. and M. Sotomayor, "Two-Sided Matching: A Study in Game-Theoretic Modeling and Analysis," Econometric Society Monograph Series, Cambridge University Press, 1990. (Winner of the 1990 Lanchester Prize.) Paperback edition, 1992. I've updated it sporadically (most recently 9/1/09), and would be glad to hear of missing references.

    Paul Klemperer has posted a pdf version of his auction bibliography .

    Bibliography of bargaining experiments. This bibliography began life as the bibliography of (my) Chapter 4 of The Handbook of Experimental Economics, John H. Kagel and Alvin E. Roth, editors, Princeton University Press, 1995. I have since updated it somewhat, and would be glad to hear of further updates I should make.

    Bibliography of Learning in Games . This bibliography is particularly incomplete, as the literature is growing fast. Please email me with missing items.

    Economic and Game Theory Bibliography compiled and maintained by David Levine at UCLA.

    Vince Crawford's course links have his reading lists.

    A bibliography of individual choice behavior and decision theory, compiled by Peter Wakker (who has also posted a Word version).

    A bibliography on Combinatorial Games with a nice introduction has been posted by Aviezri Fraenkel at the Weizmann Institute.

    Sergiu Hart has a bibliography on Shapley value and related values.

    Jerry Kelly has a bibliography on Social Choice

    Duke University Libraries now has a collection of my old papers: Inventory of the Alvin Roth Papers, 1960s-2000

    Return to Table of Contents

    Curriculum vitae, and consulting services
    Alvin E. Roth

    CV in pdf format. (Many of my papers are downloadable in pdf format here).

    Consulting services.

    In case we ever have to meet at an airport, here's my photo, taken by my son and photographer Aaron (who is also the photographer who took the picture of me that appeared on page C1 of the NY Times on April 8, 1998). (And here's another photo, taken in my usual attire:-)

    Return to Table of Contents

    Job Market Candidates 2011-12

    (updated 5/29/12)

    This year has been a crash course for me, since I am helping introduce five exceptional young economists to the job market: one postdoc, two students whose main adviser I was privileged to be, and two students who I helped to advise on papers of mutual interest. Here are links to blog posts on some of their work, disaggregated below.

    Eduardo Azevedo, see blog posts here and here.

    Katie Baldiga, see blog post here.

    Clayton Featherstone (also on the Stanford job market page here), blog post here.

    Yuichiro Kamada, see blog post here.
    Jacob Leshno, blog post here and here.

    Update: they all got good job offers:) To see which ones they accepted, check out the updates at the bottom of this blog post: Five Harvard candidates for the Economics job market this year (2011-12)

    Return to Table of Contents

    History of this page:

    This page was initially constructed in June, 1995 by my (then 11 year old) computer guru, Aaron Roth, who in those days did pages for a modest emolument. (It turns out old versions of the page, which give a window into how my work has developed over the years, are available courtesy of the remarkable Wayback Machine. Here are some snapshots of my page at Pitt before 1998, and at Harvard since then.) For design from a Human Factors perspective (and another page of Aaron's design), see, Roth Cognitive Engineering, and here's a new Roth Cognitive Engineering website for those looking for the work of Dr. Emilie Roth.


     

    al_roth"at"harvard.edu