Updated June 21, 2021. The article was updated to correct minor grammatical errors, and technical details were added by Tyler York and Justin Pincar, founders of Achievable.
On Wednesday, February 12, 2020, the United States Medical Licensing Exam (USMLE) shocked the medical community by announcing that score reporting for Step 1 would change from “a three-digit numeric score to reporting only a pass/fail outcome.” The USMLE’s decree is likely to have profound implications on medical education and the residency selection process in the years to come. In this article, we will review the upcoming changes to the Step examinations and explore how they came about, discuss the reactions of advocates and opponents of the change to Step 1’s scoring system, and consider the implications of the USMLE’s decision.
What exactly will change
In addition to the change to Step 1’s scoring system, the USMLE announced two more changes to its testing program. Examinees will now be allowed to take each exam in the Step series up to four times, down from six, to achieve a passing score. Additionally, all examinees who wish to take Step 2 Clinical Skills (CS) will now be required to have first passed Step 1. Of note, the three-digit scoring systems of Step 2 Clinical Knowledge (CK) and Step 3, and the pass/fail scoring system for Step 2 CS, will remain unchanged.
How this happened
The number of applications residency programs receive has soared in recent years. Between 2007 and 2018, for example, the number of programs an average residency applicant applied to has increased by 88%, from 32 to 60. Already stretched thin by their clinical duties, many residency program directors turned to screening tools to help sift through the deluge of applications. Step 1, scored on a standardized three-digit scale and most commonly taken during the second or third year of medical school (before residency applications are submitted), took on an increasingly important and outsized role in the residency selection process.
As Step 1 grew in prominence, critics pointed to its impact on medical schools, its effects on medical student wellness, and concerns about its clinical relevance to argue for a course correction. In response to these growing concerns, the USMLE convened the Invitational Conference on USMLE Scoring (InCUS) in March 2019 to consider changes to its three-digit scoring system. The USMLE subsequently released several vague policy recommendations from InCUS in June and sought public comment in October. Four months later, the USMLE announced the aforementioned changes to its testing regime.
An uncertain timeline
The exact timeframe for the transition from a three-digit to a pass/fail scoring system is unclear. However, the new scoring regime is, at minimum, 22 months away. In its announcement, the USMLE noted that the transition to pass/fail “will take effect no earlier than January 1, 2022, with further details to follow this year.”
As a result, current second-year medical students who choose to complete medical school without breaks will not be scored under the new pass/fail system. However, current first-year medical students who choose to take Step 1 after their third year of medical school may be assessed on a pass/fail basis. This state of affairs will be clarified after the USMLE outlines its plans in more depth later this year.
The other two new policies, while also operating on uncertain timelines, are likely to come into effect sooner. The USMLE will start limiting test takers to four attempts on each exam in the Step series “no earlier than January 1, 2021” and will require examinees to have completed Step 1 before Step 2 CS “no earlier than March 1, 2021.”
How people are reacting to the change in Step 1’s scoring system
The reaction to the change in Step 1’s scoring system has unsurprisingly been mixed. A lively debate about the pros and cons of this change has popped up on the SDN Forums. An open online poll on Medscape, which as of the time this article was written had recorded 3924 votes, found 47% of respondents in favor of the change to a pass/fail scoring system and 39% opposed. The most commonly cited concerns about the policy change were the possibility that “pressure will increase on Step 2 and Step 3 exams” (28% of respondents), worries about a “disadvantage to DO and international medical graduates” (22%), and a “disadvantage to graduates from less prestigious medical schools” (21%), and worries that “basic science knowledge will become deemphasized in medical school curricula” (14%).
Interestingly, support in the poll for a pass/fail scoring system was stronger among physicians than medical students. A majority of physicians (51%), in fact, supported the change, while 35% did not. In contrast, more medical students opposed (44%) than supported the change (41%).
Shortly after the decision was announced, long-time advocates of a pass/fail Step 1 expressed joy over the news. Dr. Shannon McNamara, an Assistant Professor of Emergency Medicine at New York University, wrote “Step 1 goes pass/fail! This is HUGE news. I’m so glad to see the USMLE getting on board with the many years of [medical education] research advocating for this change.” Proponents of the status quo, however, were disappointed. Dr. Ganesh Rao, the Residency Program Director for the Department of Neurosurgery at Baylor College of Medicine in Houston, wrote that “This is a terrible decision and will disadvantage good students at lesser-known schools. Of course, the USMLE consulted no one about this. Another step in the dumbing down of our profession.”
Others reacted more philosophically. “This change is complex, in origin and in effect. I don’t believe anyone truly knows how it will impact the medical school experience or the [M]atch process. However, it has certainly made a lot of educators wonder what the best process is to evaluate a medical student,” explained Dr. Adaira Landry, the Assistant Residency Program Director for the Department of Emergency Medicine at Brigham and Women’s Hospital in Boston. Dr. Bryan Carmody, an Associate Residency Program Director for the Department of Pediatrics at Eastern Virginia Medical School, noted that “Simply getting rid of three-digit scores doesn’t improve medical education. And it doesn’t make residency selection any better, either. It does give us the opportunity to make changes. And the importance of that should not be understated.”
The anticipated near-term effects of a change to pass/fail
At this point, the implications of these changes are not entirely clear. However, it is likely that the emphasis that medical students, medical schools, the test preparation industry, and residency programs place on Step 1 will diminish once the exam is graded on a pass/fail basis.
Preparation for Step 1 would change drastically, as students would now be simply concerned with passing in a time-efficient way rather than maximizing their score. Courses like Achievable, which uses adaptive learning to optimize memorization and study time, will become more popular as students adjust their goals.
In this new era, residency programs are likely to shift their focus from a pass/fail Step 1 to a scored Step 2 CK as a screening and applicant stratification tool. Medical schools and the test preparation industry, in response, may reorient their curricula and testing materials away from the basic sciences to better prepare students for the more clinically-focused Step 2 CK. As a result, all of the anxiety and stress that medical students previously experienced preparing for Step 1 is likely to shift to Step 2 CK.
Some have argued that changing Step 1’s three-digit scoring system may harm students from “lower-ranked academic institutions” and international medical schools by eliminating an opportunity to stand out in the residency selection process. This, in turn, might place even more value on the Medical College Admission Test (MCAT), which plays a large role in determining the fate of medical school applicants. However, there are reasons to suspect that these concerns will prove to be unfounded, at least in the near future. As it stands, the medical school a residency applicant attends already plays a large role in the ultimate fate of his or her application. Moreover, as discussed above, a scored and more prominent Step 2 CK will still provide applicants from less prestigious institutions an opportunity to stand out with high scores.
The changes the USMLE announced last month will usher in a new era in medical education in America. Observers will be watching closely to track their effects on medical schools and medical students in the years to come.
Kunal Sindhu, MD, is an assistant professor in the Department of Radiation Oncology at the Icahn School of Medicine at Mount Sinai and New York Proton Center. Dr. Sindhu specializes in treating cancers of the head, neck, and central nervous system.