QBanks Before Step 1: Bomb the Questions, Ace the Test
Created July 18, 2016 by Jason Ryan, MD
Here’s a scenario I am asked about a lot: It’s a month before Step 1 and a student is doing lousy on UWorld questions. Or sometimes a student tells me they just took a NBME practice test and dropped from their previous score. Step 1 is getting closer and panic sets in. What should the student do? Bump their test date out a few weeks? Find a new resource? Maybe just quit medicine all together?
This scenario happens to everyone to some degree as Step 1 nears. And although the panic reaction is understandable, poor performance on practice questions before Step 1 is actually a very good thing in many cases. There are three important reasons not to worry too much about wrong answers to practice questions, which I will explain in this post.
In general, we think more about questions we get wrong than those we get right. When we answer incorrectly, it bugs us. It’s frustrating. Plus we have to work to understand why we were wrong. All of this make questions—and the topics they cover—stick in our minds. This means wrong answers in the weeks before Step 1 more easily become knowledge at your fingertips during the exam. When you are acing 95% of practice questions, you won’t retain as much information.
I my experience, students who perform best on Step 1 are often the ones who don’t panic over—and sometimes even enjoy— questions they get wrong in Qbanks. They are looking for a tough Qbank to really challenge them. They know difficult questions will stick in their mind. The wrong answers don’t bother them because they know it’s all just practice.
Practice questions are not necessarily predictive of Step 1 scores. No valid relationship has ever been established between any practice test and USMLE scores. Some websites offer to translate your practice test scores into Step 1 scores, but these correlations are based on self-reporting by students. This is unreliable and only includes the tiny sliver of students who choose to report their scores. How many in the class ahead of you revealed their scores? How many showed you an authentic score report sheet? Probably not a lot. When so many students keep their scores private, no correlation will be accurate.
Some students will say their actual Step 1 test contained questions similar to a particular Qbank or NBME practice test. This is undoubtedly true since Qbanks, NBME exams, and Step 1 cover the same concepts. However, don’t be misled into thinking that “similar” questions on Step 1 mean your performance on practice questions is necessarily predictive. It is not.
Obviously, students who correctly answer practice questions are more likely to correctly answer Step 1 questions, but practice questions are not destiny. I have advised many students over my career. I have seen stellar Step 1 scores by students who performed poorly on Qbanks and NBME practice exams. I have also seen underperformance by students who did well on practice tests.
Practice questions are different from exam questions. The NBME publishes guidelines for medical schools to use in writing USMLE board-style questions for classes. You can read them here. The guidelines list common errors in question writing. You will see these mistakes made routinely in many of the available QBanks. For any question you get wrong in a Qbank, there is a good chance it was written in a manner unacceptable to the NBME. The content may appear on your exam, but not the question.
The NBME practice tests are more representative of the question style of Step 1. These tests generally do not make writing errors since they are approved by NBME committees and follow the guidelines. But NBME practice questions are retired and will not show up on your exam.
The bottom line is this: Any practice question you get wrong is unlikely to be repeated exactly on your exam. The key is to remember the concepts from questions you use for practice. And when you answer incorrectly, concepts are more likely to stick.
If you are struggling with practice questions leading up to Step 1, don’t panic. Review what you’ve done over the past year. If your foundation of knowledge is strong and you’ve done well in school, you will be fine. Those wrong answers don’t necessarily mean a bad score is in your future. In fact, they may turn out to be the key to a top performance.
About the Author
Dr. Jason Ryan trained in internal medicine and cardiology at Harvard’s Beth Israel Deaconess Medical Center where he also served as a chief resident. In addition to his MD, he holds a Master of Public Health degree and a Bachelor of Science degree in Chemical Engineering. He is a faculty member at the University of Connecticut School of Medicine in Farmington, Connecticut and the creator of Boards and Beyond.