Pittsburgh offered Michael Fox everything he could want. Though the winters were blustery, he found the medical students warm, a faculty eager to teach, and numerous opportunities at the medical center, all important factors in his decision to attend the University of Pittsburgh School of Medicine.
And to a small extent, the school’s ranking in U.S. News & World Report also played a part.
“I knew that I wanted to go to a well-known and well-ranked medical school, and the U.S. News list seems to be the go-to guide for those factors,” says Fox, now a fourth-year.
The rankings are important, he adds, “especially if you care about reputation or want to go to a school that keeps as many doors open for your future as possible.”
Medical schools, and U.S. News, caution applicants against relying too much on the rankings. But for some applicants searching for a deciding factor between two offers, it often comes down to what the magazine thinks.
Each year, prospective medical students—and their parents—eagerly await the annual medical school rankings published by U.S. News. Schools covet high rankings and work hard to get them, as the top 10 get significant publicity. The magazine’s rankings of colleges and graduate and professional schools are the de facto standard for evaluation by potential applicants and potential employers of a school’s graduates. With its latest rankings available at newsstands everywhere, no other rankings have as wide a reach among the general public.
But as the rankings became cemented in public consciousness, they’ve catapulted far beyond the subjective conclusions of a general-interest news magazine. With essentially the only rankings around, “they are as close to a standardized, objective system we have that allows us to compare every school in the country,” Fox says.
Validity
Robert Morse is the director of data research for U.S. News and has worked there since 1976. He develops the methodologies and surveys for the “America’s Best Colleges” and “America’s Best Graduate Schools” annual rankings, which were introduced in the mid-’80s. Morse says that he is not an authority on medical education. While other professional societies have worked with the magazine on its methodology specific to their academic disciplines, those in medical education haven’t.
For decades, schools assailed the list as unscientific, questioning the methods used to judge them. From undergraduate schools to law schools, those methods have been considered unfair. Criteria include reputation among similar schools as a major component, and some schools have chosen not to participate in evaluating their peers. On its Web page listing medical schools alphabetically, the magazine footnoted 24, stating, “This school refused to return the survey.”
“We’re independent of the institutions and the accrediting bodies,” Morse says, “but we are outsiders that have been doing this for about two decades and developed the knowledge and expertise about the merits of schools, so the public trusts us for that.”
Fox, like others, didn’t delve into how U.S. News came to its conclusions. His assumption of validity was so certain that even a cursory look was enough. “I looked to see what number each school has next to its name, and that’s about it.” (This year, Pitt ranked 14th in research and 12th in primary care.)
One reason that many don’t wade into the methodology is because of its complexity. Understanding it takes time, which few students—and general readers—have. Even those readers who make the effort can lack the expertise to conclude whether the methods are sound.
This year’s graduate school evaluations were published in the May issue and are considered the 2011 rankings. Details of the methodology were included for graduate business and part-time law school rankings, but not for medical schools. The methodology, which differs for each discipline, and much more detailed ranking information is reserved for the magazine’s Web site and its separate college guide on newsstands. These also serve as reference guides, with basic information such as tuition, enrollment and contact information.
U.S. News surveyed the 126 allopathic medical schools accredited by the Liaison Committee on Medical Education (LCME) in 2008, plus the 20 osteopathic medical schools accredited by the American Osteopathic Association the same year. In total, 122 schools provided the data necessary to rank the quality of their research and primary care components.
The magazine uses a weighted average of indicators, including reputation among administrators of other medical schools, selectivity, student MCAT scores, GPA and ratio of faculty to students. Primary care includes the proportion of graduates entering primary care specialties.
On its view of rankings posted on its Web site, the Association of American Medical Colleges (AAMC), which represents all accredited U.S. and Canadian allopathic medical schools, cautions against over-relying on the rankings. Separately, AAMC president Dr. Darrell Kirch had some strong words to say about the rankings in a 2007 article, saying the rankings had “fundamental” flaws. The AAMC posts a widely referenced 2001 article from its journal Academic Medicine immensely critical of the rankings system. They “are ill-conceived; are unscientific; are conducted poorly; ignore medical school accreditation; [and] judge medical school quality from a narrow, elitist perspective,” wrote William McGaghie, a professor at Northwestern University Feinberg School of Medicine. McGaghie is an expert in medical education, from undergraduate to postgraduate levels.
Nearly a decade later, McGaghie remains skeptical. He has not looked into the issue for some time, he says, but because of the commercial aspect of the rankings, he can’t imagine they have become any more valid.
“This is not science at all, and has to do with marketing, selling magazines and making money,” McGaghie says, later adding, “It’s really hard to take it seriously without understanding the real reason it’s done, and the final reason continues to be commercial.”
Morse doesn’t really disagree. “U.S. News is not trying to conduct peer reviewed research and social science research. That is a misconception,” he says. “The two most misguided points are that U.S. News is claiming to be scientific. I think our rankings have at least some credibility.”
The ranking is intended to reach prospective students and parents. “They are the primary users and market, and they know about the MCAT and NIH and know about residencies,” Morse says. “The general public might not be statistically sophisticated and they are not the end users. But they trust U.S. News has developed the judgment to determine the important factors.”
McGaghie, on the other hand, feels the rankings are intertwined with a school’s wealth, prosperity and visibility. Schools without name recognition become second-class, he says. The top five medical schools this year were Harvard University; the University of Pennsylvania; Johns Hopkins University; the University of California, San Francisco; and Washington University in St. Louis.
He emphasizes that all the allopathic medical schools must meet strict LCME accreditation guidelines. “All of these schools have undergone a rigorous accreditation process, and their graduates are all called ‘doctor,’” McGaghie says. “It is a slap in the face of the accreditation process for U.S. medical schools.”
Mayo Medical School is one of the schools that stopped completing the peer assessment. (Incidentally, the Mayo Clinic does participate in the magazine’s rankings of best hospitals and is ranked second-best in the country in the most recent list.)
“Our decision was based on our view that important measures of quality medical education were not included in the process used to arrive at the rankings,” said Barbara Porter, administrator and assistant dean of student and academic affairs at Mayo, in a statement.
Porter acknowledged that applicants have used the rankings as the deciding factor in where to go to school.
“We do want students to have access to all of the information they need to make this very important decision and, therefore, we will likely re-evaluate our decision to participate in the rankings in the future.”
Lower-Ranked Schools Hurt
Fox absolutely thinks the list hurts lower-ranked schools in many ways. “With most students using it as the first place they go,” he says, “…a lower rank could influence the number of high-quality applications a school might receive.”
But Fox believes higher-ranked schools are better in many ways, and reputation counts. “We really all do get the same education,” he says of the general curriculum, but each school is unique in research exposure and alumni network. “These things, and others, are what will help to set you apart for residency applications and also help you better establish yourself in your future.”
Morse sees significant differences in the importance of the rankings to medical students than the ranking of other graduate programs to students in those disciplines. “[It] seems the rankings are less important to medical students to be successful in medicine. You don’t need to go to the very top school, but in law, it seems to be [important]. It’s important to go to the top research university if you want to be an M.D./Ph.D., but that’s not the case for the vast majority of [medical] students.”
To Dr. John Prescott, the AAMC’s chief academic officer, potential applicants should examine schools’ specific missions. “All are wonderful schools and have different missions,” Prescott says. “I tend to look at individual schools, and how well the school is accomplishing its mission.”
“What is most important,” he adds, “is visiting a school, talking to the faculty and students and, during the interview, seeing if they like you and you like them.”
Fox echoes Prescott, saying it takes many factors to determine the right school. The rankings fail to address the most important aspect for a prospective student, which is choosing the right fit for that student.
“At the end of the day, med school is about more than rankings, and about more than lectures and H&Ps and boards scores,” Fox says. “It’s about developing and maturing into a better person, who just so happens to be a doctor. You’ve got to pick the school that you think will allow you to fully develop and mature into the person and doctor that you want to be.”
Steve Woo is associate editor of The New Physician.