Irish Education head warns of greater government intervention in higher education

by Brian Owens

*****
This from Dick Ahlstrom, our Irish correspondent. Could it be the shape of things to come in other countries? -- BO
*****

The days of limited government regulation of higher education are drawing to a close, and higher education sector may see its income linked to national economic priorities, according to the chief executive of the Higher Education Authority.

Tom Boland was speaking on 12 June at an international colloquium on higher education at the Dublin Institute of Technology. He was expressing his personal views rather than the official line of the HEA, he told delegates.

He said that the days of low-key regulation by government were coming to an end. Lack of controls had “given us unnecessary and inefficient duplication in programme provision. It has give us mission creep, inflexible staffing structures and practices and it has given us a fragmented system of institutions which to a very great extent stand apart and aloof from each other,” he told delegates.

He suggested there was a case for significant change in the sector, where university income was coupled to national economic priorities.

The current situation could not remain, however, he said. “We cannot afford this mode of operation and organisation any more. We cannot afford the cost and we cannot afford the implications for quality,” he stated.

The sector needed to place a greater emphasis on collaboration and coherence. This would make it possible to build up a critical mass of expertise in particular disciplines. It would take a “shrewd use of resources” but was necessary if Ireland were to have any research impact internationally.

“Collectively our higher education institutions represent a very valuable national resource and it is imperative for Ireland’s economic and social development that its full potential, across all institutions of all types, be realised,” he stated.

Obama's FutureGen revival draws mixed reviews

by Colin Macilwain

*******
This from Rebecca Trager, our Washington correspondent. The progress of FutureGen will be watched closely by those interested in the global potential of carbon capture and storage -- CFM
*******

The Obama administration has reversed another prominent Bush-era science and technology policy decision by reviving the FutureGen clean coal project. But many doubt the worth of the project, and reaction to the decision has been split.

Federal funding for FutureGen, which originally aimed to build a near zero-emissions coal-fueled power plant in Mattoon, Illinois that uses carbon capture and storage (CCS), was pulled in January 2008 by the former Bush administration because its estimated cost had almost doubled, from $950 million to $1.8 billion.

Things became nasty when the administration was accused of deliberately sabotaging the project because Illinois was selected as the site for FutureGen, rather than applicants from elsewhere – including Bush’s home state of Texas.

More recently, Congress’ non-partisan Government Accountability Office (GAO) concluded that the Bush administration had overstated FutureGen’s likely pricetag. The project’s costs had only risen to $1.3 billion, GAO said. Concurrently, the House of Representatives’ Science and Technology Committee analyzed government documents and accused Bush’s Department of Energy (DOE) of purposely using erroneous figures to derail FutureGen.

The saga is at over for now. DOE’s new director, Steven Chu, announced on June 12 that the department will soon restart FutureGen’s preliminary design activities, and update its cost estimates.

The department anticipates contributing over $1 billion to the effort, and obtaining at least $400 million from industry partners. A decision about the project’s future will come in early 2010.

The administration’s action is unsurprising since Obama represented Illinois in the Senate, and was one of several lawmakers from that state that protested the cancellation of FutureGen.

Illinois politicians – both Republican and Democratic – applaud DOE’s about-face, calling it a “historic moment” for the state and country. They characterize the FutureGen project as having great scientific promise and economic benefits. But others, including many environmental groups, are less enthusiastic.
Greenpeace, however, has protested against FutureGen by putting a fake billboard on the plant’s future site that called it a “$2 billion dollar hole in the ground”. City officials reportedly made them remove it.

CCS is an unproven technology and it won’t be commercially viable before 2020, Greenpeace states. Money spent on CCS means that clean alternatives like wave, wind and solar power get short-changed, the NGO warns. The organization says CCS keeps the hope of coal as an energy source alive, even when it is not needed.

Chu himself has made negative statements about coal and its environmental impact, calling it his “worst nightmare” in December 2008.

The fact that FutureGen’s target for cutting carbon emissions has already been dramatically reduced – from 90 percent to 60 percent – has drawn fire from the project’s critics.

FutureGen’s intent is to test technologies that could later be deployed commercially. But the non-profit Clean Air Task Force says the new 60 percent threshold will achieve levels that are no higher, and maybe even lower, than several commercial projects already at advanced stages of development.  

During these tough economic times, FutureGen’s potential to generate over 10,000 jobs makes it very politically attractive. But overly eager politicians should not cloud questions about its technical merit.

DOE press release
http://www.energy.gov/news2009/7454.htm

Wales to get chief scientific adviser

by Laura Hood

Wales is to have its own chief scientific adviser for the first time, the Assembly government has announced.

First minister Rhodri Morgan, who also acts as the Welsh science minister, has agreed that the assembly government would benefit form the advice of a top-level scientist as it continues its own science strategy, independently of Whitehall.

The decision follows a 2008 review, which was commissioned by Morgan as a first step to assessing whether or not an independent CSA would be needed.

Chris Pollock, the former director of the Institute of Grassland and Environmental Research in Aberystwyth and author of this report, has been widely tipped for the role.

Morgan was expected to make a decision on appointing a CSA as early as last December, but reportedly delayed the announcement over funding worries. However, now apparently looking to a more financially stable future, Morgan said in a statement that appointing a CSA would help efforts to “encourage the knowledge, skills and enterprise to strengthen businesses in Wales ahead of the global economic upturn”.

The adviser will report directly to Morgan, making recommendations on promoting science, technology, engineering and maths in Wales, and on general policy making.

The Welsh CSA will be expected to act as a link between UK CSA John Beddington and the Welsh government.

The assembly will begin the recruitment process for the CSA position immediately.

Edinburgh and Aberdeen win in Scottish funding settlement

by Colin Macilwain

The Scottish Funding Council has announced university funding allocations that reward Edinburgh and Aberdeen universities for their strong performance in the 2008 Research Assessment Exercise.

But Strathclyde and Stirling universities, which did badly in the RAE, receive reductions in their overall funding — and sharp cuts in the research support component of it — under the settlement, which is released today in a letter to the universities from SFC chief executive Mark Batho.

"This is the start of a brave new world for Scottish higher education," said Batho, noting that the settlement is the first to reflect a new funding structure agreed last year with the Scottish government, whereby universities receive their support from a general fund and a new, ‘horizon fund', which is supposed to reflect the government's strategic priorities.

SFC allocations to support research activity are adjusted sharply in the settlement, mainly as a result of the RAE results. Aberdeen's total research support soars by 22 per cent, to £22 million, for example, and that of the University of Edinburgh, Scotland's largest research university, increases by 10.7 per cent, to £75.7m.

Stephen Logan, senior vice principal of the University of Aberdeen, said that the outcome was "fantastic" for the university and reflected a long-term strategy to strengthen research there. Five years ago, the university was reorganized from 38 departments in six faculties to twelve schools grouped in three colleges, and it is now embarking on a ten-year, £200m construction programme. "This gives us great confidence that our strategy is working", Logan said.

In a statement, Timothy O'Shea, principle of the University of Edinburgh, said: "Since 2002 our objective has been to improve the quality of our research at a faster rate than the upper quartile of the Russell Group of universities - and we have achieved that."

David Gani, head of policy at the SFC, said that the increases for stronger universities such as Edinburgh reflected their RAE performance, rather than any strategy of concentrating resources. "Edinburgh has done well because it increased the volume and quality" of its RAE submission, he said.

The University of Glasgow, Scotland's second largest, will receive £47.3m in research support, up by 1.1 per cent, or less than the expected rate of inflation.

Overall, the SFC's university budget grows by 3.4 per cent this year. Of that, a small fraction is yet to be allocated to institutions, meaning that the total allocation revealed today is up by 2.9 per cent  — close to the likely rate of inflation — from last year.

But research support for Strathclyde falls under the settlement by 5.3 per cent, to £20.4m. And at the University of Stirling, it plunges by 16.8 per cent, to £8.1m. These reductions would have been even sharper but for the SFC's decision to provide an extra £1.3m to Strathclyde, and £1.5m to Stirling, for this year only, to soften the transition. "Our transition problem is not a large one," says Batho. "We had two institutions that needed some level of additional level of support."

Andy Mitchell, a spokesman for Stirling, said that after the RAE outcome, the drop in the research allocation was "disappointing but not unexpected". "The 2008 RAE is already out of date," he said, noting that it was based on work published up to 2006. "We have moved on since then and we have a strategy in place, with our sights set on the next assessment."

Total funding for each university, including support for teaching, is subject to slower change — but still shows pronounced differences between winners and losers in the RAE. Edinburgh's total grant support from the SFC is up 5.3 per cent to £186.5m, and Aberdeen's is up 7.2 per cent to £90.4m.

But Stirling's total allocation falls by 1.6 per cent and Strathclyde's by 0.3 per cent. With inflation further eroding the value of these grants, both institutions face cuts this year. Mitchell said that Stirling was still working on its detailed response, but had no plans to close departments.

As was the case in England, some ‘new' universities benefit handsomely from the SFC settlement, after their better-than-expected performance in the RAE. Research support for Robert Gordon University in Aberdeen grows by 79 per cent to £2.7m, the University of the West of Scotland is up 30 per cent to £1.36m, and Abertay University in Dundee up by 27 per cent to £947,000. However when teaching and other grants are added in, these institutions each receive only small increases in their overall allocations.

Link: Scottish Funding Council Circulars

Link: SFC grants for 2009-10 and percentage changes since 2008-09 (PDF)

Puzzling risks to critical excellence

by Roy Anderson

From the demise of well-known high street chains to the collapse of sterling, one topic dominates the news agenda these days, and the outlook is unrelentingly tough. Amidst the economic gloom, however, there is encouraging evidence of UK strength in one key area. When it comes to the source of new ideas, discoveries and innovations–our world-class universities–the UK more than holds its own, despite spending less of its GDP to support them than all but one other G8 nation.

The 2008 Research Assessment Exercise proved that the UK's top universities are doing research that is world-leading and internationally renowned. These universities will play a vital role in the eventual economic recovery and in maintaining the UK's ongoing competitive edge.

When it comes to industrial R&D, companies still do the development side but the majority of innovation now comes from the best universities. In fact, the UK is second only to the US in research excellence and, with just 1 per cent of the world's population, produces 9 per cent of publications and accounts for 12 per cent of citations. Universities are finding ways to improve healthcare and produce new non-carbon emitting sources of power. And they are spinning out their technical innovations into small and medium size companies, creating jobs and helping to make the economy competitive.

This Government has a good record on support for university research. During recent years, it has increased funding well ahead of inflation. In his annual grant letter last month to the Higher Education Funding Council for England, universities secretary John Denham stressed the need for maintaining this high level of funding for those institutions that carry out the largest volumes of world-class research. In particular, he singled out the importance of science, technology, engineering and mathematics disciplines.

I am therefore puzzled by signs that HEFCE may, at this most critical time, reduce the funding allocated to the UK's world-class universities (RF, 4/02/09, p1).

In the 2001 RAE, around one third of university research received the highest 5* rating. The changes for the 2008 exercise meant that 150 out of 159 institutions received the highest 4* rating for some of their research. The available funds will inevitably be spread more thinly, reducing the amount that will go to those universities with a critical mass of excellence.

That may contradict Denham's stated intention "to maintain high levels of funding for those institutions with the largest volumes of world-class research". And begs the question of why we have a dual funding system if HEFCE cannot fund it appropriately.

The RAE 2008 clearly shows that pockets of excellence exist across the UK. But solitary islands of excellence are no good if you are trying to solve global problems. For that you need concentrated multidisciplinary excellence, sustained by long term funding. It took the best engineers, physicists and computer scientists in the world to design and build CERN; the development of what became the World Wide Web is just one result of the creative problem-solving triggered by the research at CERN. The scale of the challenges that face the planet make it very unlikely that their solutions will arise in places where a critical mass of researchers is not assembled.

Advocates of thinly spread funding have argued that increasing funding for those pockets of excellence need not threaten the top universities, as they will be able to make up the lost funding from other sources.

However, this argument looks frail at present. The global economic outlook is such that external funders have rising pressures on their own resources. Without appropriate support and funding, we will end up with a mediocre system of higher education, and the UK will have an international reputation that reflects that.

In this economic climate, no sensible person or business would make an investment unless the benefits of doing so were clearly demonstrated. The UK’s leading universities have repeatedly proven their worth over successive RAEs, in the impact of their highly-rated research and in world rankings, with four in the world’s top ten. This excellence is good for the UK and for the world, and must be preserved.
Roy Anderson is Rector of Imperial College London.

Don't like the result? Get over it!

by Les Ebdon

There are those who want to retreat from the conclusions of the 2008 Research Assessment Exercise, to change the rules of the game, or who yearn for the comfort of the old binary divide. They should take note of Barack Obama’s comment when he took the presidential pen in his left hand for his first public signature and declared: "I am a leftie. Get over it".

The RAE 2008 demonstrated, beyond doubt, that universities with very little research investment in the past could compete with world-leading and high-quality research elsewhere. High quality is everywhere. Let’s celebrate, and get over it, and not rewrite the rules to protect the historically privileged at the expense of others for whom proper recognition, along with public research funding, has been (to borrow another Obama phrase) "a long time coming".

Nor should we fall into the trap of calling for universities to be partitioned into the equivalent of the old grammar, technical and secondary modern school system, with differential funding regimes and pecking orders. It is now even more obvious that no university can claim to lead all of the research agenda, and nor should one.

The Higher Education Funding Council of England’s decision, confirmed last week, to fund excellence wherever it is found, is welcome. The algorithm (7:3:1:0, for allocating funding for 4*:3*:2*:1* activity) remains highly selective but recognises the need to fund world-leading research at all institutions. More puzzling is the decision to limit funding of postgraduate research students to the larger groups. There seems to be no evidence for this. Given the growing shortages of supply of PhDs in many areas, this particular announcement seems perverse and harmful (RF, 4/02/09, p1).

All UK universities are committed to the Bologna Declaration, which confirms that all universities should be centres of teaching, research and knowledge transfer. UK universities should be united in their support of the declaration by backing the distribution of research funding in a way that recognises the excellence that the RAE 2008 has identified.

There is, of course, another good reason why excellence should be funded wherever it has been found. Undergraduate students have a right to expect

research-informed teaching wherever they study. To imply that postgraduate provision should be limited to a few universities pays little regard to reality and to the need for more postgraduate study in many of the newer disciplines and in multidisciplinary areas. It also ignores the fact that countries such as Sweden, Germany and Finland, which are the UK’s key competitors when it comes to innovation, all have a higher percentage of doctoral graduates than the UK.

Both government and funding council have indicated that the new quality-related funding regime, due to be announced on 5 March, will pay some regard to the previous funding of science, technology, engineering and maths. As a chemist, I can see the attractions. However, this argument cannot be pushed too far. There is absolutely no evidence to support funding by critical mass; indeed, the evidence suggests that smaller research units are more productive, and the results of the RAE 2008 confirm this.

Also, large STEM facilities of strategic significance are already funded through the research councils and other initiatives. The ring-fence around STEM could have the obvious consequence of protecting those pre-1992 institutions that have been historically well funded. It also runs the risk of diverting funds away from excellent research in other areas of significance to the economy, including social science, public policy and the many disciplines critical to the creative industries. HEFCE must also consider the impact of the ending of research-capability funding, which has greatly benefitted post-1992 institutions.

But let’s not carp at this stage. After all, 150 institutions had at least 5 per cent of their research activity in one of their submissions classed as ‘world-leading’. No-one can seriously argue that the excellence found in a range of subjects all over the UK should be nurtured and allowed to flourish in only a handful of its universities; research of international excellence has the right to be funded wherever it has been found.

The real proof of the pudding will be in the final allocations, but the days of a closed shop on QR funding should be over. Universities that have demonstrated research excellence, in spite of very little research funding in the past, will want to see a real and significant increase in quality-related public research investment.

Crucially, this would benefit students, whatever course they choose to study and whichever university they choose to attend. They are, after all, one of our most precious assets—the research capital of our future. Let’s not forget them.
Les Ebdon, is vice-chancellor of the University of Bedfordshire, and chairman of Million+, a university think tank to which 28 post-1992 universities subscribe.

Investment must be targeted...

by Nancy Rothwell

This Research Assessment Exercise may well go out in a blaze of controversy, not because of its methods of assessment, but because of its impact on research funding for the leading UK research universities. Try as they might, the guidelines on funding allocations, released last week by the Higher Education Funding Council for England, still leave much to be desired (RF, 4/02/09, p1).

The RAE is now well established as a measure of the research quality at higher education institutions, and in departments and subjects, which informs existing and potential staff, students, funders and investors. It is also the means for distribution of the major government funding for research through the Higher Education Funding Council for England’s quality-related funding.

Many of our top research universities celebrated success in the results of RAE 2008, but now find that their success is not likely to be rewarded by more funds, or even by level funding, but potentially by a reduction in HEFCE QR funding. Funding models based on multipliers of research quality achieved in RAE 2008, including HEFCE’s newly released model (with its multipliers, 7/3/1/0 for 4*/3*/2*/1*), predict a mismatch between overall quality and funding.

This anomaly applies at both institutional level and for subject areas within institutions. In short, the very best universities, as assessed by HEFCE’s own methodology, may stand still or lose research funding. The problem arises because of HEFCE’s avowed intent to fund excellent research wherever it is found. Small clusters of excellence, in universities that previously had little or no QR income, will thus receive QR funding.

Small clusters that have achieved excellence without major infrastructure or a crowd of excellent colleagues deserve our acclaim. But research funding is about more than rewarding individuals or clusters; it is about optimising the UK’s research capacity and about the UK’s standing globally.

Few could oppose the principle of funding good researchers wherever they are located. But should this be achieved by diluting internationally significant concentrations of excellence that have been built up strategically and painstakingly over the past decade, which place the UK in a leading international position?

HEFCE has many complex models at its disposal to adjust the funding outcomes of the RAE, though most cannot make a major impact on the leading institutions. A higher proportion of funds for certain subjects, such as those in science, technology, engineering and mathematics, as HEFCE’s guidelines propose, will help to maintain funding for the top research universities. It is also justified by the government’s stated desire to invest in science, and will limit losses of our top universities. But what about the thriving subjects in humanities and business studies, and their leading institutions?

The funding model could yet provide some weighting for volume (‘critical mass’) by adjusting funds within each Unit of Assessment for the number of staff returned. This move would, in part, address concerns about the percentage of staff returned to the RAE. An adjustment for volume would provide a double reward for institutions that returned most of their staff, on the individual volume driver and on an additional weighting for volume.

Indeed, this was a key outcome of the major overhaul of the RAE by Gareth Roberts in 2003, but the recommendation was not taken up for the 2008 exercise. Such manipulations may seem contrived, but there is much evidence that concentrations of research excellence, and ‘establishment of critical mass’, have huge benefits for research and research training.

Critical mass is also a strong indicator of sustainability in a subject or institution, allows interdisciplinary research and establishment of significant infrastructure. Such concentrations are reflected in the success of large research-intensive universities in world rankings, in competitive funding at home and abroad, in publications and citations and in the training of young researchers.

We are on a global stage, and if the UK is to compete with the best in the world, investment must be strategically and selectively targeted. The RAE is a UK exercise, but it has a much wider audience. Will this, the last RAE as we know it, be the one that is remembered for depriving our best research institutions of funding and that moved towards a wider distribution of funding across the UK? How sensible is it to dilute the principle of selectivity?

In times of global economic hardship, there is an argument for focusing limited resources in areas of strategic importance. It will be interesting to see how HEFCE addresses these competing pressures and, indeed, how decisions, which might move the UK towards a more even funding model that takes funds from the very best universities, are received globally.
Nancy Rothwell, an MRC research professor, is deputy president and deputy vice-chancellor at the University of Manchester.

...not if you want competitiveness

by Nick Petford

The promise of the latest (and last) Research Assessment Exercise was clear—to fund research excellence wherever found. The mantra rested easy in the psyche of the elite universities that currently enjoy over 80 per cent of the Higher Education Funding Council for England’s total quality-related funding. Some minor readjustment perhaps, but 2008 would shape up to be a re-run of 2001. The sub-panels would do as required, and peace and harmony would be maintained in the QR universe. Oh dear!

It didn’t take long for the financial shock-waves to start reverberating when, after the dust had settled, it became clear that research submitted to RAE 2008 was ranked as either world-leading (4*) or internationally excellent (3*) in 150 out of 159 UK universities. Concerns were soon heard that, should HEFCE play a straight bat, QR funding would be spread more thinly.

Predictably, many of the ‘elite’, research-intensive universities are worried, claiming that without continued selectivity their ability to compete internationally as world-class universities will be put at risk. There’s faint praise for those lower-ranked institutions that managed to pull some 3* and 4* rabbits out of their hats, followed by a more extensive hyperbole on the inherent dangers in actually rewarding this acknowledged excellence.

Subtle asides hint at processes gone wrong, with over-generous sub-panels delivering inflated grades. What these folk fear is a process akin to osmosis—the diffusion of funds from an area of high concentration to an area of low concentration across the semi-permeable membrane that is HEFCE. The scare tactic is to claim loss of competitive edge as the inevitable longer-term smoothing of selectivity drives the system towards a state of thermodynamic mediocrity.

There is of course a certain inevitability to all this. Similar fears emerge whenever special interest groups feel threatened by change. Thankfully, HEFCE’s guidelines on funding allocations, released last week, endorse the council’s early promise (RF, 4/02/09, p1).

It is hardly surprising that the beneficiaries of the Matthew Effect—to those that have shall be given—are fighting to keep their cash. Yet would modest redistribution of funds really do so much harm? Some simple analysis may be instructive.

Take the five leading pre-1992 institutions, chosen on the basis of the proportion of their research activity that RAE 2008 rated as either 4* or 3*: Imperial (74 per cent), Cambridge (71), LSE (69), Oxford (69) and Manchester (64); and the five most-improved, post-1992 universities, chosen on the basis of the rise, since RAE 2001, in the proportion of their research activity that RAE 2008 rated as either 4* or 3*: Hertfordshire (up to 41 per cent), Bournemouth (37), Anglia Ruskin (33), Chichester (26) and Derby (23).

While the average proportion of 4* and 3* activity in the pre-1992 institutions (69 per cent) is a little over twice that in the post-1992 universities (32 per cent), the difference in combined QR funding (£2.36 billion, against £19.89 million) is a factor of 118. Put another way, the listed ‘ex-polys’ managed to drag themselves up by their bootstraps and achieve respectable levels of world class and internationally-rated research on a combined QR income of less than one per cent of that paid to the super elite. Just a little more resource given to the less well off would enhance disproportionately both their research environment and quality of output.

Such value added arguments, casually dismissed at high table and in the senior common room, may raise eyebrows with those untroubled by the politics of university research funding but who pay for most of it—UK taxpayers. Those outside the sector assume universities exist simply to teach. They may however be interested to know why their offspring—of whom the vast majority will never attend the QR elite their parents help fund—should find themselves in large numbers at universities invariably referred to over the past few weeks as "the bottom end" or "also-rans", and denied, through lack of funding, a student experience enriched by research.

No one would want to see a perverse outcome to RAE 2008 that diminished UK global leadership in research. The elite universities will continue to receive the lion’s share. But the argument is strong that some redistribution along quality lines will add to, not subtract from, UK competitiveness. In the UK’s new landscape of research excellence, who can say where step-change innovation, leading to new products, goods or services, will arise.

In this context, it is significant that the Universities of York, Warwick and Essex all feature in the research top 10. Not so long ago, they themselves were new universities. Imagine if the RAE had begun in the late 1960s or early 1970s, and vested interests had clubbed together to exclude them from any future research funding? Starved of resources, where would they be now and what opportunities would have been missed?
Nick Petford is pro vice-chancellor (Research and Enterprise) at Bournemouth University.

An RAE with something for everyone, but exactly what?

by Jonathan Adams

The RAE2008 results have produced more information about the spread of the UK’s research achievement than we could have hoped for back in the 1980s.  The profiling that Gareth Roberts proposed has overcome some of the worst deficiencies of the old grading system.  But it is pretty clear that the data avalanche has buried a lot and it will take a while to dig ourselves out.

Evidence planned to get a post on the blog by Thursday lunchtime last week, and we failed.  We failed because, in the words of Morris Zapp, ‘every decoding is another encoding’.  Each time we ran a query we came up with new questions, and very few answers.

The best bit about the data is the profiling, and that is also the trip-wire for analysts.  Profiling is great because it shows ‘what lies beneath’.  For research, most activity is skewed.  Some people earn a lot of money, train a lot of post-grads and publish a lot of papers.  Most people are less Stakhanovite.  Most papers get a couple of citations, or none.  Some papers get hundreds of citations.  So, wherever you look, you see a distribution that is anything but normal and that means that indices that use averages are not telling you much about the data spread and the centre of the distribution.  The Roberts profile takes us away from those averages, and it takes us away from single grades with their terrible funding cliffs.

But how do you analyse a profile?  One profile is OK.  We can look and see the spread, and reflect on the balance of national and international and whether that means ‘good’ in our dictionary.

A set of profiles across one subject is even better – more work to do the comparisons but we can look up and down the list and see how things shift about.  Interesting to see that one institution got a lot of 4* material but actually got less on [4* plus 3*] than another institution.  Interesting to speculate on whether those top end 4* outputs are the most critical element or whether it is an overall grade point average that establishes a ranking.   And how would you weight the elements in this subject, and would it be the same in that subject?

And that is what makes it very difficult when you start to try and create a combined picture across an institution, and then to put institutions into a single table.  We work on the data and start to pick out some strange differences.  Can it really be true that Anthropology, History of Art, Music and Drama all really have more than 20% of their output at 4* while Education, Psychology and Agriculture are under 10%?  In media subjects and the arts we get 4* values as high as 65% ‘international leading’.  I’m very happy for the institutions picked out but I do not really believe it.  That means two-thirds of the activity was at the international cutting edge and that is an almost infeasible standard to meet.

Biology, Chemistry, Physics and Mechanical Engineering are in the low teens.  Why so different?  Two things.  One is a much greater familiarity with the concept of categorising a portfolio of evidence about research activity.  It’s a pretty basic part of science culture but it doesn’t come so easily to humanities and arts (check the scars on the Warden of Goldsmiths if you don’t believe me).  So, for scientists, this is ‘what we do’.

The second thing is dialogue and consensus.  Running through 2007, at a lot of meetings and on visits to institutions, I heard people talking about what they thought the RAE2008 outcome would be like.  The view, which became a common one, was that selective submission meant few 1* items, a lot of 2*, not too difficult to hit 3* but bloody difficult to get a 4* except on the very best.   I thought that view was universal but I now think it was much more the view of scientists, and of scientists in pre-92 universities, than it was of academia as a whole.  But because the dialogue was going on, the scientists’ and the engineers’ consensus was reflected fairly consistently in the outcomes for the bigger subject areas.

More thoughts follow, but one thing is clear.  Any league table you create now may look pretty shaky by Twelfth Night.

Jonathan Adams

Director, Evidence Ltd

Keep the back-slapping in perspective

by John Rogers

So, once again we have the gift of RAE results to brighten our festive cheer. But, in the spirit of the season, should we not pause a moment to reflect on what the outcomes actually tell us? There will, inevitably, be much obsessive interrogation of the outcomes and their implications in the days and weeks ahead but, in the midst of that, let us not forget the limitations of the RAE.

Much might have been done to refine the assessment process for 2008, building on the experience of panels in previous exercises. For example, this exercise could have tried to improve the consideration of interdisciplinary research, applied work, and the lot of early career researchers.

I have no doubt that all panel members in 2008 will have worked hard at these issues, and yet the framework they were given to work within looks very familiar and increasingly anachronistic.

Research has moved on apace: it is ever more collaborative, trans-national, interdisciplinary; it is increasingly done in knowledge sharing and translational partnerships with its non-academic users and beneficiaries; it is disseminated through a multitude of formats and media.

Yet, the RAE remains stubbornly disciplinary in its structure and, as I am sure closer examination of submissions will show, concerned with a rather narrow range of ‘safe’, conventional, academic outputs.

Now is not really the time to talk about the prospective Research Excellence Framework, but it is hard to see how the REF can be other than a retrogressive step in these areas as it is currently proposed. Research is complex and sophisticated: it deserves the same qualities in the systems by which it is evaluated. The RAE is not perfect, but it is a subtle and mature assessment tool, pioneered in the UK and recognised internationally as a model of good practice that commands confidence among the research community. It could be further improved, drawing on the skills and experience of panel members and our own growing research-based understanding of peer review processes. Should we really abandon it so lightly in favour of the questionably cheaper and undoubtedly nastier blunt instrument of bibliometrics?

The only genuine major change introduced for the RAE 2008 is the one that is proving most problematic, in many ways, as we grapple with the results. The currency has changed: graded profiles replace numerical values. Moreover, the conversion rate from the old grading scale is far from clear, and significant questions remain over whether, and to what extent, the currency has been debased in the process.

Equally, the mix of base and precious metals does not appear, on first glance at the results at least, consistent across the different territories in the RAE landscape. There are variations in the grade profiles awarded by some panels that seem rather at odds with what we might otherwise expect of the disciplines they have assessed.

The results of previous RAEs have been broadly endorsed after the event by comparative bibliometric analyses and independent expert opinion. We will need the same tests to be applied to the 2008 results, I suspect, before we can have full confidence in their robustness and consistency.

Perhaps the most worrying aspect of the change in rating scale is that the familiar hallmarks of quality, the 5 and 5*s, which are understood and relied on by research partners, sponsors, prospective research students and staff, have disappeared. As a sector, we face major challenges ahead in communicating the meaning of the results to those outside our walls who are not steeped in the mysteries of the RAE and for whom the ‘ready reckoners’ of quality have gone.

The need to offer clear, unambiguous quality labelling of the UK research base to industry, the public, voluntary services, policy makers and potential international students and staff recruits, will only increase as we face the difficult financial times ahead.

The season calls for celebration, and there is undoubtedly much to celebrate in the outcomes of the 2008 RAE. Research in UK higher education institutions is bigger, better and more exciting than ever before, so let’s party. But as we do so, let’s remember that there is also much exciting, relevant, valuable and high quality research that is not included within these results. They are only one indicator of the quality of a limited selection of the work that is going on in the UK research base. Keep them in perspective!
John Rogers, Director of Research and Knowledge Transfer at the University of Stirling, was the Manager of RAE 2001.