“Topsy-turvy ranking” in social science teaching

Research which challenges some league table views of teaching quality

Times Higher Education has a piece on a detailed study of teaching of Sociology at a range of instututions which has some interesting results:

Teaching in universities that are usually ranked towards the bottom of higher education league tables is more consistently of a high standard than instruction at institutions towards the top of the rankings, a study has suggested.

The in-depth examination of pedagogical quality in sociology and related degrees at four different types of institution found that rankings were not a good guide to teaching quality or the “personal transformative” effect of an undergraduate education on students.

Researchers from the universities of Nottingham, Lancaster and Teesside interviewed students at four other unnamed institutions over the three years of their degree courses.They also surveyed 700 students, interviewed lecturers and observed teaching as well as analysing assignments and each department’s curriculum documents.

More details of the research are on the University of Nottingham website. The problems with league table and ‘student as consumer’ approaches are questioned and the deeper benefits of the students’ educational experiences highlighted:

These research findings have a number of significant policy implications that contradict approaches endorsed by government and higher education leaders following recommendations in last year’s Browne Report.

Principal investigator Dr Monica Mclean from The University of Nottingham’s School of Education worked with Dr Paul Ashwin from Lancaster University and Dr Andrea Abbas from Teesside University to evaluate courses. They interviewed students over the three years of their degree courses, surveyed over 700 students, interviewed lecturers, observed teaching and analysed assignments, each department’s curriculum documents, and national policy documents.

In their research, the team identified indictors of high quality learning outcomes and processes which are not accounted for in the measures currently used in higher education league tables (such as, staff: student ratios, money spent on library resources, or numbers of research students).
They found three broad outcomes of a high quality undergraduate social science education, which included both individual and social benefits. These were:

· enhanced academic and employability skills
· understanding of and empathy for a wider range of people
· a change in personal identity and an intention to change society for the better.

The extent to which students experienced each of these individual and social benefits was positively and significantly related to their levels of engagement with academic knowledge or mastery of their subject.

Students experienced engagement with academic knowledge as a process of personal transformation that required hard work to achieve. Research showed that facing the difficulty of acquiring knowledge makes it valuable and enjoyable. Experienced difficulty of the disciplinary knowledge was very similar across all institutions.

Differences in the quality of undergraduate education, as defined by the indicators above, did not reflect the institutions’ positions in higher education league tables. Scales used in the survey reveal the complexities of the experiences of a high quality undergraduate degree. They show that students at all surveyed HEIs grapple with – and value – the same kinds of knowledge and report achieving similar individual and social outcomes.

All of this shows that undergraduate education is a lot more complex and difficult to capture than league tables suggest and raises real questions about current Government higher education policies.

The Times: 2011 University League Table

2011 University Rankings published by The Times

Not huge changes here with the exception of the remarkable jump by Lancaster, up from outside the top 20 to a top 10 slot. Not entirely clear from the data why this should be.

1 Oxford (1)
2 Cambridge (2)
3 Imperial College (3)
4 St Andrews (4)
5 London School of Economics (7)
6 Durham (8)
7 University College London (5)
8 Warwick (6)
9 York (11)
10 Lancaster (23)
11 Edinburgh (14)
12 Exeter (9)
13 Bath (13)
14 Bristol (10)
15 Leicester (15=)
=16 Loughborough (17)
=16 King’s College London (12)
18 Sheffield (18)
19 Southampton (15=)
20 Nottingham (20)

Make up of the top 20 though is pretty similar to last week’s table in The Independent with one notable exception: there is no place in The Times for the University of Buckingham.

In the subject tables, Oxbridge is, as usual, dominant but there is more to say here:

But excellence in specific subjects is not confined to these old universities. Campus universities created in the 1960s are well represented. Indeed, Warwick and Loughborough lead more tables (three each) than any university outside Oxbridge. Loughborough is ahead of the field in its speciality of sports science, as well as in building and librarianship.Warwick is top for American studies, communication and media studies and drama, dance and cinematics. Nottingham is the only other university to top more than one table, sharing the lead in agriculture with Reading and also heading the ranking for pharmacy.

Full details and subject league tables are available via The Times Good University Guide. Unfortunately, in order to get to them you have to sign up to the new Times website.