Higher Ed data – way too much information

Tackling the surfeit of data

I’ve written before here about Higher Education regulation (see for example this general commentary and this post on information provision) and the excess of information provision available to prospective students.

It’s pleasing therefore to see that HEFCE is undertaking a review of providing information about higher education. The aims of the review are set out as follows:

The review will aim to ensure that:

  • wherever possible, the different elements of the provision of information fall within a coherent framework, across UK institutions
  • we gather sound evidence to help us form the future information
  • the outcomes of different mechanisms suit the issues they are designed to address
  • information is usable and accessible, and that we are able to make the best use of technology to facilitate this in the future.

The review will reflect on how much this area of our work costs the public purse. It will also consider the role of a range of organisations in providing independent, contextualised, robust, comparable and usable information.

unistats latin

There’s plenty more where this came from

The review will look at the purpose and use of NSS results, at the Unistats site and the Key Information Set data as well as the Destination of Leavers from Higher Education (Delhe) survey. It is also going to examine how this data is used by prospective students. If all goes well this should be an extremely valuable piece of work and will, it is to be hoped, result in a significant reduction in the quantity of data collected and published (and the bureaucratic burden on universities) in favour of an improvement in the quality of information available to applicants.

A long way to go but let’s hope that the group overseeing the work, the Higher Education Public Information Steering Group (HEPISG, from which acronym I’m afraid I still derive puerile amusement) will do its job well and we will see some real change in this area.

Advertisements

The College Scorecard: KIS-ing in the USA

President Obama announces a higher education initiative which looks a bit familiar.

scorecard

The College Scorecard as it is known has attracted some mixed reviews since it was announced by the President. The Chronicle of Higher Education notes that it is intended to help prospective students but that it doesn’t perhaps do all that is claimed:

In his State of the Union address earlier this month, President Obama announced the release of the College Scorecard, a project he first proposed in a speech at the University of Michigan at Ann Arbor last year. The interactive online tool provides comparable information on college affordability and value, to help students and families figure out “where you can get the most bang for your education buck,” the president said.

Most interestingly it bears some quite striking similarities to the Key Information Set, recently established in England, ostensibly to aid student decision-making about higher education choices:

graduate-employment

Graduate salary information

finance

Financial information

And as an earlier post on Unistats and KIS noted, that isn’t necessarily a good thing.

Unistats and KIS – just too much information?


Unistats – now with added KIS – has launched

The all new Unistats site has launched:

Unistats is the official site that allows you to search for and compare data and information on university and college courses from across the UK. The site draws together comparable information on those areas that students have identified as important in making decisions about what and where to study. The items that students thought were most useful have been included in a Key Information Set (KIS), which can be found on the Overview tab for each course.

The site draws on the following official data on higher education courses:

  • Student satisfaction from the National Student Survey
  • Student destinations on finishing their course from the Destinations of Leavers from Higher Education survey
  • How the course is taught and study patterns
  • How the course is assessed
  • Course accreditation
  • Course costs (such as tuition fees and accommodation)

There is a mass of information here and, as this screenshot shows, data is presented in a handy tabular form:

However, we do have a problem. As previous posts have noted there really is just too much data here and across the various university, HE sector information and league table websites. The launch recently of the new Which? University site (about which I posted here recently) added to the mess and the Unistats upgrade just serves to make the picture even more complicated for applicants.

There is no information deficit in HE. We do not need more and better course comparison websites. What we do need is fewer new websites and more and better guidance for prospective students.

The Imperfect University: More and more regulation

More Regulatory Woes

A recent speech by the Universities Minister focused on his apparent desire to reduce regulation for institutions: “We are in a government that understands the value of autonomy,” Mr Willetts said.

Mr Willetts talked about the possibility of reducing data collection requirements as well as the likelihood of universities escaping some EU regulation following the shift to majority private funding as a result of the new fees regime.

Both of these would be welcome steps. The burden of excessive regulation is a significant problem for universities. It might be thought that as the size of the public contribution to higher education funding declined this would be reflected in a reduction in red tape. Unfortunately, despite the rhetoric, universities are left with the feeling that the weight of regulation tends to keep growing rather than reducing.

So will the reduction in public funding really mean less government involvement in university affairs? Sadly, no. Rather than cuts in government funding to HE resulting in a bonfire of red tape, there is a whole host of new or augmented regulations, including:

  • The new fee arrangements which institutions are all deeply engaged in preparing for the implementation of at present
  • Changed student number controls, with the uncertainties of AAB+ and bidding for the 20,000 students at the margin
  • The move to more comprehensive annual access agreements with OFFA
  • The changed financial memorandum between institutions and HEFCE
  • New visa arrangements for Tier 2, staff, and Tier 4, students, together with monitoring arrangements for the latter
  • The Key Information Set (KIS) which will require all universities to provide more information to prospective students
  • The proposed introduction of a Student Charter
  • The idea of extended transcript information for all graduates through the Higher Education Achievement Record
  • The revised Quality Assurance Agency institutional review method
  • The increased burden of Freedom of Information requests
  • Developments in the work of the Office of the Independent Adjudicator
  • Charities regulation.

This is only part of the picture. The cost to the sector of compliance with this regulatory framework is significant. And this really is not what universities need right now. Part of the burden is data collection and the Information Landscape Project, announced by the Minister in the same speech, is intended to address this:

The BIS White Paper ‘Students at the Heart of the System’ discusses a new regulatory framework for HE in England and explicitly tasks HESA, working with the Higher Education Funding Council for England, the Higher Education Better Regulation Group and the Information Standards Board for education and skills, to redesign the information landscape in order to arrive at a new system that meets the needs of a wider group of users; reduces the duplication that currently exists and results in timelier and more relevant data.

Will this make a difference? We will see. Universities have been here before though and have yet to see a real impact on regulation.

Policy, Regulation and Lies?

So, how big is the accountability ‘burden’ on institutions? Looking back over the past dozen years there are several attempts to measure and to reduce or improve regulation. In 2004 PA Consulting, following up an earlier report on accountability costs, investigated changes in accountability costs as perceived by the universities they contacted. The study cited a number of changes since 2000 including the introduction of a revised QAA framework and reduced requirements from HEFCE in terms of bidding, tendering and consultations. The paper concluded, on the basis of its survey, that the accountability cost to the sector had declined over the four year period to the equivalent of £188m (at 2000 prices), a reduction of 25%. However, it is also noted that this was the equivalent to the ‘annual income of two large universities’ (at that time) and therefore ‘a cause for continued attention’. Something of an understatement, especially given the ‘slightly disconcerting’ range of additional accountability requirements identified in the report. These findings were commented on favourably by the (then) Minister for Lifelong Learning, Further and Higher Education in his presentation of the Government response to the Interim Report of the Better Regulation Review Group (in 2004).

The Better Regulation Review Group was succeeded by the Higher Education Regulation Review Group (HERRG). HERRG was established to ‘introduce a stage of informed scrutiny into the policy making process’. The impact of both groups was, arguably, negligible. It is difficult to muster much confidence that the latest incarnations of this kind of task force will be any more beneficial in terms of reducing the bureaucratic burden.

There is always a gap between policy as formulated and as implemented. The simplistic nature of the instrumental interpretation of relationship between policy and evaluation is rational but disconnected from reality. Unfortunately though, evaluation is often seen in this way, as the provision of information to policy makers or stakeholders who prefer simply described and preferably numerical outcomes. The disappointment at the lack of measurable output inevitably leads to a desire for further regulation in order to deliver greater confidence in the results of policy. The consequences of this though can be inimical to the intended ends of the original policy and corrosive of trust.

Victims of our own success

The role of HE in creating the professional labour force of the UK public sector is one example of how successful universities have been and how vital is their role in contemporary society. Given the importance of universities then the desire for intervention should not come as a surprise but the interventionist approach which characterises regulation in HE is also part of wider trends and a diminution of an historical trust which no longer appears to offer adequate reassurance of the quality and standard of HE provision. This decline in trust results in heavy transaction costs and, associated with the move towards controlling institutions via contract and regulation, matters have to become more explicit rather than implicit. It is difficult though to see this reduction in trust in institutions as anything other than a long term and irreversible trend. The very nature of the national quality assurance framework, for example, would seem to reinforce this and it is difficult to imagine that the historical basis of trust can be reconstructed.

Regulations - lots of them

Other angles

Some other interesting views: Martin Wolf [1] has argued that universities are, in effect, a nationalised industry which accepted their financial dependence on the state with the foundation of the UGC in 1919. The government has asked institutions to assure quality without providing the funds needed to ensure it and the less generously the government funds, the more it interferes with the universities. That has certainly been the experience in the last few years. Salter and Tapper [2] suggest that, whereas Robbins saw total institutional freedom as necessary for the efficient operation of universities, in the 1980s the principle of HE regulation was established to ensure the efficient use of public funds and then, having developed this far, specific accountability arrangements were needed.

The costs of regulation of HE outweigh the extremely limited benefits. Indeed the Better Regulation Task Force report says that the HE sector has ‘earned the right to more autonomy’ and that multiple audits, excessive data requirements and over-restrictive funding are ‘symptomatic of a lack of trust between government and the HE sector’.[3] The Chairman of the Task Force observes that ‘there is no evidence that the sector is particularly prone to management or financial failings or failures to deliver on academic performance’. The statement suggests that earned autonomy is a key theme for government which wants to see it applied more in HE. The report refers to the PA Consulting study (quoted above) which estimates the accountability burden for HE to be £250m per annum and notes that the National Audit Office regards HE as a low risk sector in terms of fraud and malpractice.

The changes made in HE regulation in the first decade or so of the 21st Century have not resulted in the reduction in burden and cost which is called for. The claimed 25% reduction in the burden of regulation between 2000 and 2004 seems extremely modest in the light of the above (incomplete) list of regulatory interventions which includes a number of new requirements replacing the eliminated accountability demands.

Private HE institutions, which are expected to increase in number under the new fee regime, will benefit from significantly lighter regulation. However, for everyone else there remains the seemingly iron law that as government funding declines the volume and range of government regulation inevitably increases. So, less money and ever greater constraints on how it can be spent. No matter what the Minister may say.

It could be worse?

Things may be difficult in universities but they could be an awful lot worse according to a recent story in the TES about the appearance of Geoff Russell, chief executive of the Skills Funding Agency (SFA), before the Public Accounts Committee which was considering how bureaucracy should be reduced in the FE sector:

Mr Russell’s retirement may be imminent but, as the hearing made clear, few expect the red tape that has stifled the sector for so long to be untangled any time soon. The committee was informed that, for every £5 that goes to FE, £1 is swallowed up by bureaucracy. The biggest difficulties, MPs heard, are caused by colleges having to deal with very different data obligations and regulations from several agencies simultaneously. A general FE college offering higher education provision, Mr Lang explained, has to accommodate the requirements of four different funding bodies – the Department for Education, the Young People’s Learning Agency, the SFA and the Higher Education Funding Council for England – as well as policy directives from the Department for Business, Innovation and Skills (BIS).

That’s pretty expensive and works out at more than £150 per student according to the National Audit Office. Things may be bad for the universities but at least they aren’t this bad. So, should we feel lucky?


What is to be done?

There has been only one substantive change in the last decade to the mass of regulation loaded onto institutions: the ending of subject-based inspection of universities (Subject Review or TQA as it was originally known). Whilst undoubtedly welcome and beneficial, the gain the ending of such inspections represented has more than been replaced by other forms of regulation. So the burden continues to grow. One step forward, two steps back.

What then are universities to do? In order to respond to the ever increasing burden of regulation I would suggest that the following steps are worthy of consideration:

  • Don’t put too much faith in Ministerial rhetoric when it comes to reducing bureaucracy;
  • When working out how to deal with all of the different regulatory demands try to sort what really matters from what is less important – assuring quality and standards, and complying with the demands of the QAA is pretty important as is legal compliance with health and safety, employment and equality legislation but other regulations may be less significant;
  • Protect core activities, ie teaching and research, from the impact of regulation as far as is possible;
  • In most cases, go for minimal compliance rather than ‘gold-plating’ of procedures to respond to regulation – this often means steering away from what may be seen as ‘best practice’;
  • Object to new regulations wherever possible – work with others for greater effect;
  • Don’t be deflected from delivering the agreed university strategy by regulation.

Will we see a reduction in the regulatory burden as Mr Willetts claims? We might, but it is unlikely to make a big difference. My advice? Don’t hold your breath.

________________________________________

[1] Wolf, M (2002), ‘How to save the British Universities’, Singer and Friedlander Lecture, delivered on September 26 2002 at Magdalen College, Oxford

[2] Salter, B and Tapper, T (2000), ‘The politics of Governance in Higher Education: the Case of Quality Assurance’, Political Studies, 48(1), pp66-87.

[3] Better Regulation Task Force (22 July 2002), Press release for Higher Education: Easing the burden, London: Cabinet Office.

Too much data?

Will more data help prospective students?

Richard Partington, writing in THE, expresses concern about the ‘data overload’ which the Key Information Set (KIS) will deliver. He notes that the provision of information to applicants via the KIS is intended to work in a similar way to price comparison websites such as those offering car insurance. And that this, despite what Ministers might think, is not necessarily a good thing:

But what really worries me, is how the data will be “innovatively presented” by the third-party providers whom the government envisages will advise applicants. Comparing universities and courses is already really difficult. Unless students are lucky enough to be supported by excellent careers advisers, they struggle to make sense of substantially incomparable information regarding course content, teaching, learning, costs and support. The problem has arguably been exacerbated by newspaper league tables that seek to distinguish themselves by weighting data differently, or including additional delineators – sometimes of comical spuriousness. The impossibility of comparing like with like will only get worse under the new arrangements. Try, for example, comparing the fee-waiver, bursary and scholarship packages of Oxford and Cambridge. Both are, I believe, strong and broadly similar. But they look very different.

An earlier post noted similar issues around the provision of advice to students in the new system. It seems to me to be quite likely therefore that the excessive provision of detailed but not necessarily meaningfully comparable data will, as Partington suggests, baffle rather than enlighten students.

A currently pretty much empty site shows an example of what the KIS data will look like and it’s easy to see how seductive this might be for those looking for a cheap solution to the provision of advice to prospective students

Enlightening or baffling? We’ll have to wait and see.

Regulation, Regulation, Regulation

More Regulatory Woes

In University Governance: Questions for a New Era, Professor Malcolm Gillies looks at a whole set of issues around university governance. A previous post noted his suggestion about a greater involvement of alumni but he suggests that they will become more important than the state, at least in governance terms, because of the change in balance of funding from public to private, ie from government to graduate:

42. State denial:  The withdrawal of the state as chief funding agent of higher education creates new balances in governance authority.  The body which can be most expected to fill that space is the alumni, as they now become the chief funding agent of most English universities in direct replacement of that state interest, through their decades-long repayment of state-provided loans.  The alumni also have the greatest, life-long stake in the institution’s reputation and its protection.  They understand the institution’s symbolic value.

But will the reduction in public funding really mean less government involvement in university affairs? Sadly, no. Rather than cuts in government funding to HE resulting in a bonfire of red tape, there is a whole host of new or augmented regulations, including:

  • The new fee arrangements which institutions are all deeply engaged in considering at present
  • Student number controls, which may well change in the light of fee developments
  • The move to more comprehensive annual access agreements with OFFA
  • The changed financial memorandum between institutions and HEFCE
  • New visa arrangements for Tier 2, staff, and Tier 4, students, together with monitoring arrangements for the latter
  • The Key Information Set (KIS) which will require all universities to provide more information to prospective students
  • The proposed introduction of a Student Charter
  • The idea of extended transcript information for all graduates through the Higher Education Achievement Record
  • The revised Quality Assurance Agency institutional review method
  • The increased burden of Freedom of Information requests
  • Developments in the work of the Office of the Independent Adjudicator

Private HE institutions, which are expected to increase in number under the new fee regime, benefit from significantly lighter  regulation. However, for everyone else there remains the seemingly iron law that as government funding declines the volume and range of government regulation inevitably increases. So, less money and ever greater constraints on how it can be spent.

A new ranking for China’s universities

According to a recent report on a ranking by the China University Alumni Association, Shanghai’s universities rank second in the country behind Beijing in educating future billionaires:

The ranking analyzed the educational background of nearly 2,500 of China’s billionaire based on five domestic and overseas rich lists between 1999 and 2010.

“The report aims to encourage college students to set up their own enterprises and provide guidance to them,” said Zhao Deguo, editor in chief of the association’s website.

Peking University, Tsinghua University and Zhejiang University took the top three spots in the ranking with 79, 70 and 66 billionaire alumni respectively.

Shanghai’s Fudan University was in the fourth place with 46 billionaires. Jiao Tong University and East China Normal University made up the Shanghai top three with 25 and nine billionaires respectively.

It’s a little cruder than the UK’s graduate employment survey but does at least take a long term view. Let’s hope that no-one decides it would be good to include this in the new Key Information Set.

A more detailed commentary on the report appears here.

.

Providing information that helps students with HE choices

New consultation on providing information that helps students make the right higher education choices

HEFCE has launched a consultation on information for prospective students:

Schools, colleges, universities, student unions and a wide range of other bodies are being asked to comment on the information that higher education (HE) providers publish to help prospective students choose the course and institution that are best for them.

They are invited to respond to a consultation being conducted by HEFCE, Universities UK and GuildHE. The consultation mainly concerns a proposed Key Information Set (KIS) which all publicly funded HE providers in England and Northern Ireland would be required to publish for each course on their web-sites.

The press release continues:

The consultation is informed by the results of research commissioned by HEFCE, and undertaken by Oakleigh Consulting and Staffordshire University, which identified the information current and prospective students identified as ‘very useful’. This mostly relates to costs, satisfaction and employability. Information about the fees for each course will also be included.

The intention is that information will be presented in a standardised format on each university and college web-site, looking similar for all courses at all institutions, thus making the information potentially more useful, comparable and accessible. Discussions are also taking place about how the information can be linked to the UCAS web-site.

But do prospective students really need more information? And is this kind of standardised set of data really going to help inform decisions. Or will most students turn to other sources such as The Times League Table rather than this sort of information. Guess we’ll find out.