Just yesterday, the Wall Street Journal published its third annual ranking of business schools. With great fanfare and a separate 12-page section of the newspaper, the Journal declared that the University of Pennsylvania’s Wharton School of Business was “A New Winner.”
Great news, I suppose, for Wharton, which has long been an exceptional school and has often ranked No. 1 in other surveys, including the more authoritative BusinessWeek list. I need to make a confession here because I created the BW ranking back in 1988. So I have the rather dubious distinction of knowing too much about these rankings.
When the Journal got into the rankings game a couple of years ago, it adopted a methodology that was and remains significantly flawed. Here’s the inside skinny: The Journal ranks on the basis of surveys it sends to MBA recruiters. The newspaper gets its list of recruiters from the business schools.
Most major MBA recruiters, however, send alums back to their schools to interview candidates. The result: If you merely survey the recruiters who visit a particular school, you’re surveying an entirely biased person. Would a Wharton MBA who recruits Wharton students rate his or her school poorly? Would that person rate any school that is a direct competitor to Wharton and can therefore beat it in the poll? No way. In fact, what happens here is that any single company which recruits MBAs often gets numerous votes — each from the alum of the school they recruit from. If McKinsey & Co. sends out 50 recruiters a year, then McKinsey has the opporunity to get 50 votes in the Journal survey.
The Journal doesn’t rank the best business schools. It ranks the loyalty of a school’s graduates. Nothing more. Nothing less. It’s why Dartmouth’s Tuck School finished first in 2002 and 2001. Dartmouth’s small graduating classes and intimate educational setting make its alums more loyal than even Harvard or Stanford. So why did Wharton come out so well this time? Because in the last BusinessWeek survey, Wharton took a big dive. What the Journal is measuring is an alumni body of recruiters that have come to the school’s aid. It’s totally bogus.
This year, the paper reported that it surveyed 2,191 recruiters. Pretty impressive, right? Except there’s less than 300 companies in the world that actively recruit MBAs from enough schools to make a legitimate judgment about the quality of these institutions and the MBAs they graduate. If the Journal wants to do a legitimate survey of recruiters, it has to do what BusinessWeek does: Spend months finding out who is in charge of MBA recruitment and development at all the companies which make the market for these graduates and then survey one person in each organization for the company’s views. But that takes a lot more effort and a lot more work.
There’s one other annoying problem with the Journal‘s coverage, something that violates simple Journalism 101 in covering any poll or survey. The newspaper reports that it sent its online survey to 2,191 recruiters, and then later reports that its ranking is “based on the opinions of 2,191 MBA recruiters.” It’s not possible that the Journal had a 100% response rate. Whatever the response rate is, the Journal doesn’t even bother to report it. That is as bogus as the highly misleading result.
The bottom line: The Wall Street Journal’s methodology is as solid as Enron’s accounting.