On a university campus where productive research is the norm and students’ academic and professional pursuits are diverse and highly competitive, some degree of interaction between the research world and student life seems inevitable. While this manifests itself in a number of ways, one such instance was particularly noticeable to students who passed from the first to the second year of the master’s degree over the summer.
An opportunity for an experiment presented itself as students applied to their M2 track of choice and the directors of these degrees sought to piece together a program consisting of the strongest candidates possible. Historically, students who successfully passed the M1 year have been guaranteed a place in one of the seven M2 tracks; in most years, students ranked their top two options, and the M2 directors, working from these rankings and the capacity and prerequisite constraints for each program, negotiated and bargained until every student had a place in a program. As became evident to all parties involved, this allocation procedure induced a large amount of strategic and provincial behavior on the part of students and M2 directors.[1] At the urging of the administration and with the aid of some external funding, two TSE researchers, Yinghua He and Thierry Magnac, assisted by Christophe Lévêque and Adrian Torchiana, set out to design an experiment and examine the results of a number of allocation mechanisms, in turn, implementing the one which elicited a stable matching and the highest overall welfare amongst students and M2 directors. The literature covering the fields of mechanism design and school choice is rich and varied.[2] The high profile nature of matching programs like the National Resident Matching Program (NRMP) and the New York City High School Match (NYCHSM) program, and the awarding of the Nobel Prize to Roth and Shapley in 2012 has firmly placed mechanism and market design into the public conscious as something that economists regularly get right. Ultimately, if the NRMP can fill 96.4% of 29,000 residency openings from a pool of over 34,000 applicants, and the NYCHSM can match over 90,000 students to more than 700 high school programs with 83-85% of students ending up in one of their top 5 choices, mechanism design is alive and well.[3] For TSE, with only 150 students, 7 programs and a handful of prerequisite constraints for admission to certain programs, theory could be expected to have no problems transitioning into practice. The TSE experiment collected student preference rankings under four mechanism schemes: the student-proposing Deferred-Acceptance (DA) Mechanism (where students rank-ordered the seven M2 programs), the Boston Mechanism (again, with students ordering all programs), the DA mechanism with students applying to only their top four programs (thus, potentially reducing review costs), and the DA mechanism with students applying to their top three programs, with the opportunity to apply for further programs at the cost of writing a letter of motivation for each additional program (again, potentially reducing review costs).[4] Students were told that one mechanism would be chosen randomly, and that the results from this mechanism would be used in the allocation. Initially, this multistep procedure was met with criticism by a large number of students and some M2 directors. Some found the procedure needlessly complex and confusing, while others thought experimentation was being performed at the expense of student welfare. However, it seemed that as people became more comfortable with the theory driving the experiment, they saw the present and future value in improving the mechanism. The DA mechanism is known as a strategy-proof mechanism, meaning that one side of the market (here, students) can optimize their welfare by honestly revealing their rankings.[5] On the other hand, the Boston mechanism has been shown to be manipulable, meaning it may reward strategic misrepresentation of preferences.[6] Similarly, due to choice restrictions placed on students by the DA mechanisms with constraints and motivation letter costs, the mechanisms may provide incentives for students to rank-order program preferences strategically. Thus, it would be expected that sophisticated students provide different rankings for different mechanism procedures if there are strategic advantages to doing so. Ideally, if the correct mechanism is chosen, a stable matching - one with no potential Pareto improvements (on the student side, but also on the program side) – is found, and students end up in a program that corresponds to their motivation and academic interests, thus improving academic performance and therefore boosting student welfare, net of costs induced by the mechanisms. On the other side of the market, directors were encouraged to rank all 150 students. Due to review costs, however, it was deemed more practical to supply each director with a “focus group” of around 50 students that had a high likelihood of being assigned to their program. These sets of students were generated algorithmically from a preliminary test round of the mechanism based on grades, program constraints, and preliminary student preference reports. Thus, each director did not need to review and rank each of the 150 students participating in the allocation, but instead, benefitted from a reduced burden of review costs. In aggregate, review costs can be massively expensive in the sense of time lost- this development signifies a large welfare gain on the part of M2 directors. Once these two sets of preferences were constructed and submitted, an algorithm designed and executed by the research team was run, and results for each of the four mechanisms were obtained. The overall results of the experiment were in some ways expected, and in others, rather surprising. Based on the stability of the match, TSE Director Jean-Philippe Lesne’s preference for a non-strategic mechanism, and the criteria used to analyse student’s welfare ex-post[7], the standard DA mechanism was deemed to be the best mechanism. Overall, 91% of students were placed in their 1st choice program, 7% placed in their 2nd choice program, and the remaining 2% placed in their 3rd choice program. In comparing these results with the outcomes from the other 3 mechanisms, marginally different percentages were obtained in terms of allocation to each person’s 1st, 2nd and 3rd choice programs, but in no mechanism did a student find himself allocated to their 4th choice program. Perhaps more surprisingly, the results showed a greater than 90% correlation between students’ preference rankings under the different mechanisms. Most students did not utilize strategic behavior when ranking under the Boston and constrained DA mechanisms. With the knowledge that all participants were fairly sophisticated[8], and well aware of the rules of the mechanism and the potential advantages of ranking strategically under manipulable mechanisms, the reason for this high level of correlation is not superficially evident. Perhaps the time and effort cost of calculating the optimal strategy for each mechanism was deemed too high for students. Perhaps in some cases, the strict ordinal student preferences captured by the ranking did not adequately measure the intensity of students’ preferences.[9] Perhaps the fierce competition for a couple of programs and nearly open-ended admission to others meant that students were willing to take a risk and state true preferences under the Boston mechanism.[10] In any case, strategic play was quite rare, and ultimately, with the utilization of the DA mechanism, proved irrelevant. Looking forward, with the positive feedback about allocation results that the administration has received this year from students and faculty alike, students should expect a similar DA mechanism to be used for allocation next year. Citing the proposed changes to M2 doctoral track admission for next year, Mr. Lesne tentatively believes that a two-step procedure will be used either on the student side or the program side, whose aim would be to allocate first students applying to the doctoral M2 (ecomath), then use a DA mechanism to allocate the remaining students to the six professional M2 programs. M1 students should anticipate a meeting to discuss the allocation procedures towards the beginning of the spring term. Footnotes: [1] Consider, for example, a weak student who is most interested in gaining acceptance to the EMO master, one of the more competitive programs. Being required to submit only 2 preferences, by listing ECOMATH- the most competitive M2 track- as his top choice (for which he will surely be rejected on the basis of grades), and listing EMO as his 2nd choice, he has thus put the M2 directors in an awkward position, where if the student is not admitted to the EMO, the remaining directors have little further information as to the student’s academic preferences. Thus the EMO director can admit him (despite his weaker grades) or the student can be randomly (or grudgingly, following bargaining/negotiating amongst directors) allocated to one of the 5 remaining programs, both of which are suboptimal, inefficient results. [2] For a short overview of mechanism design, see the excellent Nobel Corner summaries written by Yinghua He, Michel Le Breton and Jerome Renault in the third issue of the TSEconomist. For a more detailed review, see Pathak, 2011: http://economics.mit.edu/files/6390 [3] For this year’s report on the NRMP, see: http://www.nrmp.org/2013-results-and-data-book-press-release/. For more information on the NYCHSM program, see: http://schools.nyc.gov/NR/rdonlyres/58C1D35F-33DB-44AA-9C2A-896E000BD404/0/2013CitywideFairOverviewSession_Posted.pdf [4] Relevant descriptions of the mechanisms’ timings and processes can be found in the chapter titled “School Choice”, written by Abdulkadiroglu in the recently published (2013) Handbook of Market Design. [5] Roth, 1982, MoOR. [6] Abdulkadiroglu and Sonmez, 2003, AER. [7] In this case, the two main criteria used seem to have been a minimisation of students allocated to their 3rd and 4th best preferences and a maximisation of students allocated to their 1st best preference (assuming preferences under the truth-inducing DA mechanism). [8] Given their status as graduate students in economics and probable exposure to some degree of mechanism design. [9] For example, consider the student who would only stay for the M2 at TSE if they were admitted to a particular program. Otherwise, they are essentially indifferent between the other six programs. In this case, strict ordinal measures do not account for the intensity (or here, indifference) of this student’s preferences. Perfect correlation of his preferences across the 4 mechanisms makes perfect sense. [10] Here, imagine a student whose top 3 preferences are respectively, EMO (highly competitive), PPD (less competitive) and ECOMATH (highly competitive). Under the Boston mechanism, a student might fear listing these true preferences if he believes he will be rejected by the EMO in the 1st round, that there is a chance he will miss out on a place in the PPD in the 2nd round, and that he will certainly have missed out on an ECOMATH place in the 3rd round, thus meaning he will be allocated to his 4th round choice. At TSE, however, this seems a scenario of low probability due to course sizes and prerequisite thresholds for less competitive programs, and thus, the student will probably simply play their truthful strategy.
0 Commentaires
Laisser une réponse. |
Archives
Octobre 2016
Categories
Tout
|