Medical

Step 1 Becomes Pass/Fail: What are the Impacts?

On Wednesday, February 12, 2020, the United States Medical Licensing Exam (USMLE) shocked the medical community by announcing that score reporting for Step 1 would change from “a three-digit numeric score to reporting only a pass/fail outcome.” The USMLE’s decree is likely to have profound implications on medical education and the residency selection process in the years to come. In this article, we will review the upcoming changes to the Step examinations and explore how they came about, discuss the reactions of advocates and opponents of the change to Step 1’s scoring system, and consider the implications of the USMLE’s decision. 

What exactly will change

In addition to the change to Step 1’s scoring system, the USMLE announced two more changes to its testing program. Examinees will now be allowed to take each exam in the Step series up to four times, down from six, to achieve a passing score. Additionally, all examinees who wish to take Step 2 Clinical Skills (CS) will now be required to have first passed Step 1. Of note, the three-digit scoring systems of Step 2 Clinical Knowledge (CK) and Step 3, and the pass/fail scoring system for Step 2 CS, will remain unchanged. 

How this happened

The number of applications residency programs receive has soared in recent years. Between 2007 and 2018, for example, the number of programs an average residency applicant applied to increased by 88%, from 32 to 60. Already stretched thin by their clinical duties, many residency program directors turned to screening tools to help sift through the deluge of applications. Step 1, scored on a standardized three-digit scale and most commonly taken during the second or third year of medical school (before residency applications are submitted), took on an increasingly important and outsized role in the residency selection process.

About the Ads

As Step 1 grew in prominence, critics pointed to its impact on medical schools, its effects on medical student wellness, and concerns about its clinical relevance to argue for a course correction. In response to these growing concerns, the USMLE convened the Invitational Conference on USMLE Scoring (InCUS) in March 2019 to consider changes to its three-digit scoring system. The USMLE subsequently released several vague policy recommendations from InCUS in June and sought public comment in October. Four months later, the USMLE announced the aforementioned changes to its testing regime. 

An uncertain timeline

The exact timeframe for the transition from a three-digit to a pass/fail scoring system is unclear. However, the new scoring regime is, at minimum, 22 months away. In its announcement, the USMLE noted that the transition to pass/fail “will take effect no earlier than January 1, 2022 with further details to follow this year.”

As a result, current second-year medical students who choose to complete medical school without breaks will not be scored under the new pass/fail system. However, current first-year medical students who choose to take Step 1 after their third year of medical school may be assessed on a pass/fail basis. This state of affairs will be clarified after the USMLE outlines its plans in more depth later this year. 

The other two new policies, while also operating on uncertain timelines, are likely to come into effect sooner. The USMLE will start limiting test takers to four attempts on each exam in the Step series “no earlier than January 1, 2021” and will require examinees to have completed Step 1 before Step 2 CS “no earlier than March 1, 2021.”

How people are reacting to the change in Step 1’s scoring system

The reaction to the change in Step 1’s scoring system has unsurprisingly been mixed. A lively debate about the pros and cons of this change has popped up on the SDN Forums. An open online poll on Medscape, which as of the time this article was written had recorded 3924 votes, found 47% of respondents in favor of the change to a pass/fail scoring system and 39% opposed. The most commonly cited concerns about the policy change were the possibility that “pressure will increase on the Step 2 and Step 3 exam” (28% of respondents), worries about a “disadvantage to DO and international medical graduates” (22%) and a “disadvantage to graduates from less prestigious medical schools” (21%), and worries that “basic science knowledge will become deemphasized in medical school curricula” (14%).

Interestingly, support in the poll for a pass/fail scoring system was stronger among physicians than medical students. A majority of physicians (51%), in fact, supported the change, while 35% did not. In contrast, more medical students opposed (44%) than supported the change (41%). 

Shortly after the decision was announced, long-time advocates of a pass/fail Step 1 expressed joy over the news. Dr. Shannon McNamara, an Assistant Professor of Emergency Medicine at New York University, wrote “Step 1 goes pass/fail! This is HUGE news. I’m so glad to see the USMLE getting on board with the many years of [medical education] research advocating for this change.” Proponents of the status quo, however, were disappointed. Dr. Ganesh Rao, the Residency Program Director for the Department of Neurosurgery at Baylor College of Medicine in Houston, wrote that “This is a terrible decision and will disadvantage good students at lesser known schools. Of course the USMLE consulted no one about this. Another step in the dumbing down of our profession.”

Others reacted more philosophically. “This change is complex, in origin and in effect. I don’t believe anyone truly knows how it will impact the medical school experience or the [M]atch process. However, it has certainly made a lot of educators wonder what the best process is to evaluate a medical student,” explained Dr. Adaira Landry, the Assistant Residency Program Director for the Department of Emergency Medicine at Brigham and Women’s Hospital in Boston. Dr. Bryan Carmody, an Associate Residency Program Director for the Department of Pediatrics at Eastern Virginia Medical School, noted that “Simply getting rid of three digit scores doesn’t improve medical education. And it doesn’t make residency selection any better, either. It does give us the opportunity to make changes. And the importance of that should not be understated.” 

The anticipated near-term effects of a change to pass/fail

At this point, the implications of these changes are not entirely clear. However, it is likely that the emphasis that medical students, medical schools, the test preparation industry, and residency programs place on Step 1 will diminish once the exam is graded on a pass/fail basis.

In this new era, residency programs are likely to shift their focus from a pass/fail Step 1 to a scored Step 2 CK as a screening and applicant stratification tool. Medical schools and the test preparation industry, in response, may reorient their curricula and testing materials away from the basic sciences to better prepare students for the more clinically-focused Step 2 CK. As a result, all of the anxiety and stress that medical students previously experienced preparing for Step 1 is likely to shift to Step 2 CK.

Some have argued that changing Step 1’s three-digit scoring system may harm students from “lower-ranked academic institutions” and international medical schools by eliminating an opportunity to stand out in the residency selection process. This, in turn, might place even more value on the Medical College Admission Test (MCAT), which plays a large role in determining the fate of medical school applicants. However, there are reasons to suspect that these concerns will prove to be unfounded, at least in the near future. As it stands, the medical school a residency applicant attends already plays a large role in the ultimate fate of his or her application. Moreover, as discussed above, a scored and more prominent Step 2 CK will still provide applicants from less prestigious institutions an opportunity to stand out with high scores. 

The changes the USMLE announced last month will usher in a new era in medical education in America. Observers will be watching closely to track their effects on medical schools and medical students in the years to come.

About the Ads
K
Kunal Sindhu is a resident physician in New York City. You can follow him on Twitter @sindhu_kunal. Kunal Sindhu is a resident physician in New York City. You can follow him on Twitter @sindhu_kunal.
Since the announcement of P/F for Step 1, some believe that Step 2CK will eventually become P/F too. As for now, I want to believe the main reason behind this change is to combat the rapidly increasing anxiety/stress over Step 1 and to emphasize student wellness. I also acknowledge the other reasons behind this change being the unjustifiably outsized role Step 1 took in residency applications and the potential freeing up of student time to redirect focus on school curriculum, research, etc. If these are the intentions, then I admire the bold move towards P/F.

However, more and more people are growing concerned that the shift of all this unhealthy anxiety/stress will go toward Step 2CK, since the report explicitly states Step 2CK will remain scored. Since this is such a natural and easy transition, it makes me think that we will surely enter "Step 2 mania" and that the governing bodies are knowingly treating the symptoms of "Step 1 mania" without focusing attention on the cause.

I have to conclude that the actual intention is not alleviate the extreme anxiety/stress that came with Step 1, but it is rather to knowingly shift it toward Step 2CK since this exam has more clinical relevance. In simpler words, we understand students are in an unhealthy state with "Step 1 mania," but at least "Step 2 mania" will produce "better doctors."
The USMLE examined a patient with a high-grade fever and a cough. They prescribed tessalon perles and called it a day.
A lot of exams in Europe and over the world are P/F and it seems to have worked very well, although the system of getting a residency is quite different. Step 1 also has a lot of clinically irrelevant aspects for the practicing physician. Knowing where NADPH goes in a cycle does not seem to be the best thing to spend time on to learn. But positives/negatives as well I guess. But instead of now looking at Step 1 for a screen, they will now look to Step 2 instead, which is much more clinically relevant. Isn't that better? So I'm curious to see why it is "bad" for IMGs or for some students. “Disadvantage to graduates from less prestigious medical schools”. How come?
M