AudiologyDentalMedical

The Competency Manifesto: Part 3

Part 3: Competency Mirror, Not the Carnival Mirror

Did you ever like carnival mirrors?  It’s often funny to see how these mirrors exaggerate various body parts to make you look like you have a short body (dwarfism) or an enlarged head (megaloencephaly).
The distorted view is often as entertaining as the game of comparison obsessively played by many prehealth applicants.  Way too often we measure ourselves by the schools we attend, the grades we made, the research we’ve performed, the clinical experiences we’ve had, the trips we’ve taken, and the clubs we’ve joined.  While often there may be some who enjoy one-upping others in their achievements, the echo chamber effect often makes it hard for individuals to really see the impression they make to others in the admissions process, and it really is this difference that can doom applicants.
In the previous articles (Part 1 and Part 2), I outlined competencies that I have used to holistically evaluate applicants for health professions programs.  For this article, I give everyone an idea of where advisees think of their own competencies in general, and how it compares to solicited references and to interviewers.

Background

Very few of us are born with expertise in a competency.  Often a skill requires some degree of training, consistent practice and competitive application in order for one to become a true master of that skill.  While we are born with certain talents or perspectives that may assist us in our development of that skill, we usually rely on others to coach us or evaluate us to become better.  Our progressive development in this competency must also be frequently assessed using quizzes and exams.  For each competency, we find ways to scale the learning curve, and it is where we are on the curve that determines our competency in a skill set.
In the system used for my institutional evaluations, I have defined six levels of competency for each of the major criteria.

  1. Naïve/Newcomer: An individual exhibits little observable knowledge, skill, or sincere interest.
  2. Intermediate: An individual has received minimal but not sufficient training to exhibit a core set of knowledge, skills, or interest.
  3. Proficient: An individual has completed sufficient training (usually through a set of required classes) to reliably reproduce a core set of knowledge and skills, but must receive further training when confronted with situations where the training is applied.
  4. Confident: An individual is competent in an above-average set of knowledge and skills, and demonstrates appropriate confidence in adapting to new situations that test the skill set beyond what is normally covered in a typical set of classes.
  5. Master: An individual demonstrates consistent excellence in a set of knowledge and skills and can appropriately seek affirmation and criticism to independently develop new knowledge and skills.
  6. Expert: An individual has received external validation of superior quality in a set of knowledge and skills, and is considered an innovator, leader, or authority in that area.

You can see variations of this learning curve in so many activities whether it is the color of a belt by someone engaged in the martial arts or one’s passion for understanding the societal factors behind health care disparities.  An individual who has a passing interest through informal volunteering may simply be a novice or intermediate, but that interest may drive that person to take classes in sociology or history to reach a better understanding.  A master or expert can clearly see situations where the textbook cases learned in those classes can be applied.  The most important change in medical education is the belief that one’s development of clinical competencies must begin much earlier.  Thus, applicants must be able to recognize themselves as ready as possible in their personal and pre-professional competencies and articulate the learning curve they ascended for their evaluators.

About the Ads

Self-Assessment of Pre-health Advisees

Every year, as a requirement for access to pre-health advising, I require all my advisees to complete an annual survey and maintain their connections with my advising system.  For the last two years, I have asked each advisee to perform a self-assessment in the 10 major competencies that are used to evaluate each prospective applicant for a committee letter.  This self-assessment is also done in a final request for a committee letter (pre-application) performed the winter before an applicant begins to submit an application.
The pre-applicants then have separate one-on-one interviews with members of my committee while solicited references are independently collected.  While solicited references usually consist of professors, clinicians, and workplace or volunteer supervisors, interviewers are members of the University community with a vested interest in student development, health care, or education – but with whom the advisees mostly did not interact.  The interviewers thus share the perspective of the “common patient” and the general community who have an interest in our future health care workforce [http://med.stanford.edu/ism/2011/january/interview-0110.html ].
In analyzing the data from 2009-2010, I found some interesting shifts of competency perception among all my advisees.  Looking at the “academic foundation” competency, I expect that most incoming freshmen would declare themselves as naive in this area, while most seniors would be more “masters”.  Indeed 40 of the 70 freshmen rate themselves no higher than “intermediate” versus 8 of 72 seniors.  What interested me was that 8 of 36 sophomores rated their academic foundation competency no higher than intermediate, and not just because of the number of respondents dropped.  In contrast 17 of 64 juniors and 3 of 46 alumni claimed a naïve-to-intermediate competency with their academic foundation.  Apparently within a year, those freshmen who continued on as pre-health advisees believed they had a proficient-to-master competency of the academic foundation for a successful health professional career.  138 of 172 applicants claim themselves as master or expert in academic foundation.
What is somewhat surprising is the “intrapersonal intelligence” self-reported competency.  60% of the freshmen identified themselves as “confident” or better.  This proportion grew to 80% of sophomores, 88% of juniors, 93% of seniors, and 96% of alumni.  If I raised the bar to masters and experts, the proportions were 34% freshmen, 53% sophomores, 59% juniors, 75% seniors, 71% alumni, 80% postbacs, and 80% for applicants.  Interestingly, the greatest concern expressed by admissions committees seems to be the lack of apparent competency in the interpersonal domain (professionalism, integrity, ethics, and personal management) among applicants that are interviewed.

External Assessment of Pre-health Advisees

Solicited references and interviewers were also asked to rate the competencies of applicants they reviewed.  Solicited references tended to rate applicants very highly in all competencies, giving over 90% of the applicants as master or expert in all domains including academic foundation and intrapersonal intelligence.  There is a lot of debate on the usefulness of letters of recommendation in the admissions process, especially if the only letters written highly praise and maybe over-exaggerate the applicants.  By having a system where everyone is excellent, excellence becomes average, and these data further confirms other studies that show the minimal impact solicited letters may have in admissions decisions.
In contrast, interviewers rated 50% of potential applicants they interviewed as master/expert in each domain including academic foundation (82/193) and interpersonal intelligence (99/193).  In the end, 23 of 104 applicants were given one of the two highest levels of recommendation from the committee.
How the committee rating is used in admissions decisions depends highly on the school, so evidence needed to be compiled about the importance of a committee evaluation in the admissions process.  That will be discussed in a future article.

Post-Script: Applying Competencies to Emotional Intelligence

The leaders of medical education all recognize that the old set of “core coursework” in premedical education, established with the Flexner report over a century ago, is inadequate for the challenges of present-day and future medicine.  Many schools have begun to clearly articulate that the typical core coursework sequence (one year of general biology, physics, chemistry, and organic chemistry) must include core content which now includes at least a course in biochemistry and college math.  The mistake in evaluating applicants has been to ascribe mastery and expertise in the grades earned in the aggregate of classes one takes.  In the professional world, one maintains that expertise (and one’s license) through professional practice, external advice, reflection, and continued education.
The same analogy should be applied for emotional intelligence.  By our nature, all humans express and manage emotions to a level where we individually are comfortable.  But the competency learning curve dictates that a minimum set of concepts must be clearly understood to be proficient, and the variety of challenges that hone those skills further help one ascend the learning curve.  For many applicants, they feel they have already the broad experience to be considered a master of emotional intelligence: that we are all able to handle honest criticism, that we are resilient in the face of enormous challenge, and that we have a value system that stands up to the most challenging of situations.  In light of so much research indicating how medical students lose empathy and become more depressed in their clerkship years, I have to suspect that the compasses that guide us in emotional intelligence often need calibrating prior to entering medical school.
So ask yourself what external standards have calibrated your own emotional-intelligence compass towards a professional career.  How often do you seek advice from other professionals interested in your emotional intelligence (such as a psychological counselor)?  How often do you reflect on yourself and your experiences in both good and bad times, and have you taken advantage of your institution’s resources for health and wellness? In other words, how do you see your reflection in your mirror, and is it how others (especially patients you never met before) see you?

Example 1: Academic Foundation


Figure 1.  Academic Foundation: EY2011 Self-Assessment and Evaluation. Ratings from applicants, their professors from whom they solicited letters of recommendation, and independent committee interviewers on competency level for academic foundation among applicants for the entering class of 2011 (all health professions) on a percentage basis.  Note that independent committee interviewers distinguish the lower end of the competency curve better than applicants or solicited professors.  This has meaning for anyone going to interview at schools where they have no or minimal contact with their interviewer.


Table 1. Self-Evaluations and Assessments on Academic Foundation

Example 2: Emotional Intelligence


Figure 2.  Emotional Intelligence: EY 2011 Self-Assessment and Evaluation. Ratings from applicants, their professors from whom they solicited letters of recommendation, and independent committee interviewers on competency level for academic foundation among applicants for the entering class of 2011 (all health professions) on a percentage basis.  Note that independent committee interviewers distinguish the lower end of the competency curve better than applicants or solicited professors.  This has meaning for anyone going to interview at schools where they have no or minimal contact with their interviewer.

Table 2. Self-Evaluations and Assessments on Emotional Intelligence
Emil Chuck, Ph.D., is the Health Professions Advisor and Term Assistant Professor of Biology and Bioengineering at George Mason University.  He has worked with Kaplan Test Prep & Admissions as an admissions consultant, student advisor, and test prep instructor.  There are no conflicting relationships that are relevant or associated with the information in this article.
 

About the Ads
E
Emil Chuck, Ph.D., is the Health Professions Advisor and Term Assistant Professor of Biology and Bioengineering at George Mason University. He has worked with Kaplan Test Prep & Admissions as an admissions consultant, student advisor, and test prep instructor. There are no conflicting relation... Emil Chuck, Ph.D., is the Health Professions Advisor and Term Assistant Professor of Biology and Bioengineering at George Mason University. He has wo...
E
To back up the point: a followup news article: "Study Finds a Big Gap Between College Seniors' Real and Perceived Learning" just published.
http://chronicle.com/article/Study-Finds-a-Big-Gap-Between/127087
Z
  • Z
    Zack
  • April 16, 2011
I know when I was preparing to go to medical school my undergraduate school's advisors always stressed being a quality applicant and not just focusing on academic numbers and the number of accomplishments. They also told us to present ourselves openly and honestly to our letter writers. Basically they would tell us not to focus on the very things schools would be evaluating us on. Every student would focus on getting good numbers, doing research just to have something to put down and convincing letter writers (who really were not qualified to write LORs the way that a long time boss would be) that we were excellent. Honest students would seek quality but be ever aware that what mattered most was how it appeared to the committee.
This is not a bad reflection on students but a reflection of the fact that students have a goal and it is not the same as the goal of the admission committee. Each student's goal is to be accepted. People can argue about whether or not that should be their goal but it is. The admission committee's goal is to sift through the students and find the ones who they perceive will be successful in school and make great healthcare providers. The committees must make known the standards whereby an applicant is judged, if for no other reason than because they need the applicant to provide the information. In so far as a committee tells students the objective and subjective criteria, students will focus on appearing competitive according to those criteria. This is not because they are deliberately dishonest. It is because only a great fool would knowingly apply in a way that made them look mediocre. To do so would be to forfeit the goal of admittance. By definition, those who apply have not given up on their goal.
Schools will continue with this problem as long as students who apply intend on being accepted (which of course they always will) and as long as comprehensive objectivity is still illusive (which it always will be.) This wont be fixed by shifting the criteria upon which an applicant is judged. Students (even the most honest ones) will just change the focus of their appearance.
E
To Zack: the point of what I am writing is to say that the criteria shift is occurring or has occurred. While the standards still remain regarding the screening of applicants, the schools are more likely to consider applicants whose goals are not just about "getting accepted". Admissions committees are charged now to uphold their schools' missions and find people who can best make real the vision of the school when it comes to the impact in the community or in the profession. Sure the short-term goal for every applicant is to get an offer of acceptances, and each committee has a daunting task to find the few students they want to interview.
With competencies and the criteria mentioned previously, the focus is going to rely much more on how really prepared you are for the next step, about documenting your competency development (before during and after professional school), and knowing the true privilege and responsibility of being a health care provider. Instead of appeasing those who evaluate you (which I guess to a point will always be important anyway), aligning your own impression with that of how others perceive your abilities is going to be much more important in your success.

E