Degree education ’more equal’ than league tables claim

A new study suggests a competitive market for university students which uses standardised information about courses to highlight differences in ‘quality’ is misleading. Treating potential and current university students simply as customers can overlook the ‘personal transformative’ effect of engaging with academic knowledge in undergraduate-level education.

Results of a major Economics and Social Research Council (ESRC) funded three-year study of sociology and related social sciences degrees reveal today that students have benefited in equivalent ways from studying the same or related subjects at four very different universities – half towards the top of conventional league tables, the rest near the bottom.

The Universities of Nottingham, Lancaster and Teesside investigated student experiences at four other unnamed universities. They will present outline findings from the ‘Pedagogic quality and inequality in university first degrees’ project at an event for policymakers at The Work Foundation, Westminster.

These research findings have a number of significant policy implications that contradict approaches endorsed by government and higher education leaders following recommendations in last year’s Browne Report.

This study of sociology and related social science degree courses considered separate universities. The team named these Prestige, Selective, Community and Diversity Universities to reflect their differing reputations.

In popular higher education rankings and league tables, the relevant departments at Prestige and Selective have regularly been ranked in the top third for research and teaching, whilst those at Community and Diversity were regularly rated in the bottom third.

Principal investigator Monica Mclean from The University of Nottingham’s School of Education worked with Paul Ashwin from Lancaster University and Andrea Abbas from Teesside University to evaluate courses. They ed students over the three years of their degree courses, surveyed over 700 students, ed lecturers, observed teaching and analysed assignments, each department’s curriculum documents, and national policy documents.

In their research, the team identified indictors of high quality learning outcomes and processes which are not accounted for in the measures currently used in higher education league tables (such as, staff: student ratios, money spent on library resources, or numbers of research students).
They found three broad outcomes of a high quality undergraduate social science education, which included both individual and social benefits. These were:

• enhanced academic and employability skills
• understanding of and empathy for a wider range of people
• a change in personal identity and an intention to change society for the better.

The extent to which students experienced each of these individual and social benefits was positively and significantly related to their levels of engagement with academic knowledge or mastery of their subject.

Students experienced engagement with academic knowledge as a process of personal transformation that required hard work to achieve. Research showed that facing the difficulty of acquiring knowledge makes it valuable and enjoyable. Experienced difficulty of the disciplinary knowledge was very similar across all institutions.

Differences in the quality of undergraduate education, as defined by the indicators above, did not reflect the institutions’ positions in higher education league tables. Scales used in the survey reveal the complexities of the experiences of a high quality undergraduate degree. They show that students at all surveyed HEIs grapple with – and value – the same kinds of knowledge and report achieving similar individual and social outcomes.

Monica Mclean from Nottingham, who led the study, said: “Many people have criticised broad league table rankings of universities. We sought to look beyond the measures used, asking about what students really value, about the use to individuals of studying for a degree (other than getting a job afterwards), and about how society benefits from social science graduates.

“Our findings challenge ‘good university’ versus ‘bad university’ stereotypes. There are huge differences in public imaginations of the quality differences between universities. Our study indicates this is largely mitigated once you find out what students are learning and the personal transformations they experience.”

Paul Ashwin, Senior Lecturer in the Department for Educational Research at Lancaster University, said: “Most recent policy documents convey the belief that quality resides in creating a competitive market for students by giving them standardised information about the courses offered so that they can make a choice as informed consumers.

“Our research raises questions about how appropriate it is to portray students simply as customers and consumers of higher education when they are engaged in a process of personal transformation.”

According to Andrea Abbas, Principal Lecturer in Sociology at the Teesside University: “This emphasis on assuring quality through competition for students obscures a quality enhancement approach that emphasises teacher development and student engagement, which our findings suggest are more likely to result in high quality learning outcomes”.

“Our findings suggest that the acquisition of employability skills is related to the level of students’ engagement with academic knowledge.”

Students who reported that they received high quality teaching had higher levels of engagement with academic knowledge and reported achieving higher levels of the three broad outcomes.

An emphasis on ‘diversity’ and choice of degree type for different kinds of student in recent policy implies different kinds of knowledge should be offered to traditional and non-traditional students. The researchers conclude that this emphasis could act as a barrier to increased social equality by reducing non-traditional students’ access to transformative knowledge.

Aspects of high quality teaching identified in the research included studying a degree course in which:

• students experience good relations with their tutors, who are interesting and enthusiastic, encourage effort and give feedback which advises students how to improve ;
• high quality academically-focused discussions are stimulated in well-designed courses, and students are able to talk to their tutors and other students about their work;
• the relevance of the knowledge under discussion is made explicit;
• and, students are supported on their courses and by the wider university to overcome obstacles to studying.

These aspects have been identified as important by previous studies but this research demonstrated that when the four universities were compared statistically, significant differences between institutions proved that they do well on different measures of quality and that the rankings are not what would be expected based on university league table positions.

Table: Universities Rankings on Selected Scales (*institutions in bold have significantly higher scores)

Prestige, Community, Selective, Diversity

data also showed that whilst students were aware of the status of their universities in current rankings, this is quite distinct from their own experiences of quality. The team investigated this further using scales related to the processes and outcomes of a good quality university education as described above.

The event for policymakers at The Work Foundation in London is a dissemination event, the first of two. The second, aimed at academics and higher education leaders, will take place at The East Midlands Conference Centre at The University of Nottingham’s University Park Campus on January 27 2012.