Menu Icon Search
Close Search

How to Prepare for Multiple Mini-Interviews, Part 2

Created February 6, 2013 by Jeremiah Fleenor, MD, MBA
Share

 

In part 1 of this two part series we looked at some of the reasons why ADCOMs (admissions committes) are searching for a new way to assess an applicant’s personality. The correlation between an applicant’s GPA and their future success in the didactical components of medical school is well established. The new frontier is a more fair and predictive way to evaluate an applicant’s character, ethics, and communication skills. That evaluation tool seems to be found in the multiple mini-interview (MMI).

The Multiple Mini-Interview

Some logistical orientation of the MMI is necessary to better appreciate the data that drives its increasing popularity. Briefly, the MMI is a type of admissions assessment tool where an applicant is shuttled through many (between 6-12) different stations or rooms where he or she is asked to perform a certain task. The task might be to break some bad news to a friend or to role play telling a patient you made a mistake while you were operating on him, discuss a current event that has a strong ethical element, or even engage in a traditional medical school type of interview. Each station may last 6-10 minutes. Once the applicant has completed all of the stations, individual scores are assigned and a composite score may be given based on all of the evaluators’ input. This then becomes your interview score and is weighed according to the university’s policies along with your academic record and other factors.

Before getting into any more of the specifics of the MMI or how to prepare for this type of interview I’d like to go into the thinking of ADCOMs to even entertain a new type of interview format. This will shed some light on the motives of admissions committees and give you some inside information.

Reliability, Context, and Validity

There are many reasons why the traditional medical school interview does such a poor job of assessing personality and future clinical success. Let’s briefly look at each.

Reliability

When speaking about the reliability of the medical school interview the issue at hand can be summed up with the question, “Would two interviewers give similar scores after interviewing the same applicant (presumably one right after the other)?”. Reliability for the medical school interview has been reported to range from .14 – .95. That’s a huge range. It’s been hypothesized that this is due to an interviewer having access to the applicant’s academic record and the other interviewer not having that information, and/or power differences between members of the ADCOM and their resultant influences and/or the interviewer’s background and expectations.

One study showed that 56% of the difference between interview ratings has to do with interviewer variability. Meaning, it has nothing to do with the applicant. Not only is that frustrating for the applicant, but it’s unethical as well.

Context

Much research has shown that success or the ability to communicate effectively in one area does not mean that the same person can do so in another area. Just because you can talk about the effects of the printing press on Western civilization doesn’t mean that you can speak clearly about the role of the magnetic compass on sea travel. Meaning, you may communicate very well during interviews and convince the ADCOMs of your great personality but this doesn’t mean that you will communicate and behave accordingly in future clinical practice.

The converse of this would be true as well. You may not excel in the pressure cooker setting of a medical school interview but in time would be able to show yourself to be a fine physician who’s caring, compassionate, and very conscientious. However, if you’re not freed from the bias of context, you may never have the chance to prove the interviewer wrong.

Validity

This was discussed in part 1 of this series but deserves some repeating. The traditional medical school interview is about as good as flipping a coin when it comes to predicting how well an applicant will do on future clinical tests. When sticking to the strict measures of validity, the traditional interview is nearly useless.

After reviewing all the data, it’s easy to see why more and more schools are looking for better ways to assess an applicant’s personality.

Can I Get Some Help?

I think any admissions committee member who was totally honest with himself wishes that he had a “personality GPA” for each applicant. ADCOMs hope to have a single measurable data point that correlates with success in the non-cognitive areas of medical education just like the real GPA correlates with success in the cognitive/didactic areas of medical education. Up to this point such a thing hasn’t existed but the MMI is showing some signs of providing the kind of predictive power that ADCOM’s are looking for.

In the same study just mentioned, the investigators found that the MMI was, by far, the best predictor of the applicants’ future success on a clinical examination. It was a better predictor of non-cognitive success than the medical school interview, the applicant’s personal statement or his or her GPA.

The good news is that this wasn’t just some random paper. The data has been confirmed in other publications.

A 2007 study showed that the MMI, when compared to the traditional interview, personal statement and GPA, best predicted higher scores on the major clinical tests during the medical students’ training. This same study showed that the MMI also best predicted success on the “non-cognitive domains” of Part 1 of the Canadian qualifying exam. This is a test that medical students must pass to become licensed physicians in Canada. It is similar to the United States Medical Licensure Exam (USMLE).

A 2009 study showed that a correlation between the MMI score and the number of stations passed in a structured clinical exam (part of the Canadian qualifying exam) was 0.43. The value rose to 0.65 when the sub-score of patient interaction was evaluated.

Substitution, Please

When looking at all of this data one wonders why all U.S. medical schools haven’t bitten on the MMI hook, line and sinker. Yet when you count up how many U.S. schools are using the MMI it’s shockingly low. According to the MSAR, only 9 U.S. schools list the “MMI” as their chosen interview format. Considering there are approximately 167 U.S. medical schools (MD and DO) that means only a little over 5% are using the MMI.

Why such a low number?

While this is only speculation on my part, I suspect there are several reasons for the low but rising use of the MMI.

First, the ground breaking paper on the use of the MMI for medical school admissions was published in 2004. That means it’s been less than a decade since this idea was brought to the attention of medical school admissions committees. To give you some perspective, the first U.S. medical school was opened in 1765. So eight years vs. the 247 years that medical education has been around in the United States isn’t actually that long. I would just give it some more time and I think the MMI’s use in medical schools will continue to rise.

Second, the majority of the research being done on the MMI is happening in other countries, namely Canada, the U.K. and Australia. For example, 11 of 17 Canadian schools recognized by the Association of American Medical Colleges (AAMC) use the MMI. I found very few U.S. based papers that were studying the MMI at their institutions. I suspect this too will change as more and more U.S. medical schools adopt the MMI and want to substantiate its validity here.

Third, there is always risk when changing from one format to another. There’s no guarantee that what has worked elsewhere will work here and I think universities are wrestling with this issue. In addition to the risk is the matter of cost. There is cost in testing a new interview format and then implementing it. In these tight economic times, expense issues weigh heavier and heavier. Furthermore, institutional cultures are hard to change. This is true in business as well as government and education. Cultural inertia is hard to overcome but I think increasing data and U.S. acceptance of the MMI will help turn the more entrenched universities.

Conclusion

Although the MMI is far from perfect, there seems to be more and more data that this type of admissions tool can help ADCOMs better predict how a medical student will perform when it comes to the non-cognitive or more personality based tests. This is significant in that these aspects of a physician’s overall nature significantly determine how he or she will behave and succeed in future clinical practice.

Certainly there are limitations to what the MMI can predict and how far out the predictions can safely be assumed to hold true. Thus far, it appears that the reliability and validity of the MMI to better predict success during medical school is solid. How well this admissions tool can predict success as a practicing clinician has yet to be determined. Regardless, the MMI is likely to be welcomed by more ADCOM’s that must make the difficult decision of who to admit into the competitive field of medicine.

If you have any questions please email Dr. Fleenorn, author of The Medical School Interview: Secrets and a System for Success 2nd ed., at [email protected].

References
Salvatori P. Reliability and validity of admissions tools used to select students for the health professions. Adv Health Sci Educ 2001; 6:159-75.

Eva K, et al. The Ability of the Multiple Mini-Interview to Predict Preclerkship Performance in Medical School. , Vol 79, No 10. October 2004.

Kulatungo-Moruzi C, Morman GR. Validity of admissions measures in predicting performance outcomes: the contribution of cognitive and non-cognitive dimensions. Teach Learn Med. 2002;14:34-42.

Eva K, et al. An Admissions OSCE: the multiple mini-interview. Medical Education 2004: 38: 314-326.

Reiter H, et al. Multiple Mini-Interviews predict clerkship and licensing examination performance Medical Education 2007; 41: 378-384.

Eva K, et al. Predictive validity of the multiple mini-interview for selection medical trainees. Medical Education 2009; 43: 767-775.

MSAR: Getting Started 2013-2014. Association of American Medical Colleges. Copyright 2012. Washington, D.C.

Medical School Admission Requirements (MSAR). Association of American Medical Colleges. Copyright 2011. Washington, D.C.

Harasym PH, et al. Reliability and validity of interviewers’ judgments of medical school candidates. Acad Med (Suppl) 1996; 71: 40-2.

// Share //

// Recent Articles //

improve your CARS score
  • Medical, +1 MORE
  • The Top 5 Ways to Improve Your CARS Score Today

  • Posted April 28, 2016 by Nick Zehner
  • For most pre-meds taking the MCAT, the CARS section proves to be one of the biggest obstacles standing between them and admission to the medical school of their dreams. The CARS section is a highly artificial environment, unlike any test you’ve ever taken before. It can be difficult to know where to begin and what...VIEW >
minorities and the mcat
  • Medical, +1 MORE
  • Minorities and the MCAT

  • Posted April 28, 2016 by Brian Wu
  • The MCAT looms large on the horizon of many would-be medical students – and there is a lot of anxiety over choosing preparation courses and books and in finding different ways to achieve the highest score possible. And there is good reason for this – a poor or mediocre MCAT score can close the doors...VIEW >
20160427_Chip_SS_118623172
  • The Dangerous Devolution of Physicians into Technicians

  • Posted April 27, 2016 by Amanda King, contributing writer for in-Training
  • Reposted from here with permission. As I sat in my institution’s white coat ceremony this past fall, I listened to our dean describe the process of selecting the newest batch of future doctors. I’m an MD/PhD student, so this is my fifth time hearing this speech, but the statistics still blow my mind: less than 3% acceptance...VIEW >
multiple MCAT scores
  • What the Adcom Sees (and Thinks) About Your Multiple MCAT Scores

  • Posted April 27, 2016 by Linda Abraham
  • MCAT History Back in the olden days (like prior to 2007), the MCAT was only offered a few times a year, and test-takers took the paper exam with a No. 2 pencil. There was also a restriction placed on the number of times you could take the exam in a single year, as well as...VIEW >
TPW_Office_SS_266115644-2
  • 5 Study Tips for the USMLE Step 1

  • Posted April 26, 2016 by Drs. Ted O'Connell, Ryan Pedigo, and Thomas Blair, authors of Crush Step 1: The Ultimate USMLE Step 1 Review
  • 1. Set a goal As the saying goes, “being begin with the end in mind.” Before you begin preparing for the USMLE Step 1, you should consider where you are with your knowledge base and your score, as well as what your goal target score is. To determine where you are starting from, you should...VIEW >
exam preparation

// Forums //