Ranking confessions

Ranking Confessions from THE’s Deputy Editor

Some time ago THE announced a bit of a change in its approach to rankings and its previous league table partner joined up with US News and World Report

Inside Higher Ed carries an interesting piece from Phil Baty in which he admits to the failings in THE’s previous league table methodology:

I have a confession. The rankings of the world’s top universities that my magazine has been publishing for the past six years, and which have attracted enormous global attention, are not good enough. In fact, the surveys of reputation, which made up 40 percent of scores and which Times Higher Education until recently defended, had serious weaknesses. And it’s clear that our research measures favored the sciences over the humanities. We always knew that rankings had their limitations. No ranking can be definitive. No list of the strongest universities can capture all the intangible, life-changing and paradigm-shifting work that universities undertake. In fact, no ranking can even fully capture some of the basics of university activity – there are no globally comparable measures of teaching quality, for example.

There’s lots of interesting critique in here, including the particular problem of the ‘reputation survey’ element of the QS ranking which had a very high weighting but very small number of participants:

The reputation survey carried out by our former ranking partner attracted only a tiny number of respondents. In 2009, about 3,500 people provided their responses – a fraction of the many millions of scholars throughout the world. The sample was simply too small, and the weighting too high. So we’ve started again.

Many of the problems identified with the QS method will be challenging to address. It will be interesting to see how the new THE table pans out. And whether its previous ranking partner prospers with a new publisher.

One thought on “Ranking confessions

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s