"Further work"

The phrase "we will need to undertake further work..." crops up several times in HEFCE's consultation document on the new Research Excellence Framework.

The biggest area that is still up for debate is how to deal with the arts, humanities, social sciences, mathematics and statistics. For these, HEFCE will "develop a quality assessment regime involving a light touch form of peer review informed by quantitative indicators". But, so far, "we have not undertaken significant development work on this," says HEFCE. The consultation is seeking preliminary input on how this system should work. The time line for this is not as tight though, it only needs to be in place by 2013, to inform funding from 2014.

Other areas that are still under development include how researchers should be assigned to institutions and subject groups, the algorithm to produce the quality indicator based on citations, and indicators of research income and numbers of research students will be used. HEFCE is also looking for suggestions for quantitative indicators that can be used to capture user value and the quality of applied research.

HEFCE has commissioned further technical advice, to report in early 2008, on some of the data and methodological issues, and will run a "substantial" pilot exercise next year.

All this shows that the REF is still very much a work in progress, and although the general framework has been decided, there is still room for academics to tweak it here and there to try and deal with the problems that are bound to crop up.

But the timetable is tight, as HEFCE acknowledges:

November 2007 to February 2008 - Consultation on key elements of the framework and on bibliometric indicators. In parallel, further work on developing bibliometric techniques.

March to August 2008 - Substantial pilot of proposed approach

Autumn 2008 - Further consultation and decisions on the framework and indicators to be used for the science-based disciplines

Early 2009 - Launch of full bibliometrics exercise for science-based disciplines

November 2009 - Output of bibliometrics exercise available for use in funding; decisions on new funding approach to be phased in from 2010

From late 2009 - Consult on light touch peer review to run in 2013

The science-based disciplines

Here are HEFCE's proposed subject groups for the science-based disciplines for the new Research Excellence Framework. The six groups map to Main Panels A to E and G in the 2008 RAE, except for Unit of Assessment 23, Computer Science and Informatics, which has been taken from Panel F and combined with the engineering subjects covered by Panel G. The rest of Panel F, mathematics and statistics, will be assessed with the light-touch peer review system being developed for the arts and humanities.

Subject group	RAE 2008 units of assessment	HESA cost centres
Clinical Medicine (RAE Panel A)	1 Cardiovacular Medicine 2 Cancer Studies 3 Infection and Immunology 4 Other Hospital based Clinical Subjects 5 Other Lab based Clinical Subjects	1 Clinical Medicine (part)
Health Sciences (RAE Panel B)	6 Epidemiology and Public Health 7 Health Services Research 8 Primary Care and Other Community Based Clinical Subjects 9 Psychiatry, Neuroscience and Clinical Psychology	1 Clinical Medicine (part)
Subjects Allied to Health (RAE Panel C)	10 Dentistry 11 Nursing and Midwifery 12 Allied Health Professions and Studies 13 Pharmacy	2 Clinical Dentistry 5 Nursing and Paramedical Studies 6 Health and Community Studies 8 Pharmacy and Pharmacology (part)
Biological Sciences (RAE Panel D)	14 Biological Sciences 15 Pre-clinical and Human Biological Sciences 16 Agriculture, Veterinary and Food Sciences	1 Clinical Medicine (part) 3 Veterinary Science 4 Anatomy and Physiology 8 Pharmacy and Pharmacology (part) 10 Biosciences 13 Agriculture and Forestry
Physical Sciences (RAE Panel E)	17 Earth Systems and Environmental Science 18 Chemistry 19 Physics	11 Chemistry 12 Physics 14 Earth, Marine and Environmental Sciences
Engineering and Computer Science (RAE Panel G, with part of F)	23 Computer Science and Informatics 24 Electrical and Electronic Engineering 25 General Engineering and Mineral & Mining Engineering 26 Chemical Engineering 27 Civil Engineering 28 Mechanical, Aeronautical and Manufacturing Engineering 29 Metallurgy and Materials	25 Information Technology, Systems Sciences and Computer Software Engineering 16 General Engineering 17 Chemical Engineering 18 Mineral, Metallurgy and Materials Engineering 19 Civil Engineering 20 Electrical, Electronic and Computer Engineering 21 Mechanical, Aero and Production Engineering

HEFCE launches post-RAE consultation

The consultation on replacing the Research Assessment Exercise with a metrics-based approach, at least in the sciences and medicine, has just been launched by the HEFCE.

HEFCE's published a Consultation document and two reports that it has been looking at, a general one from the bibliometric team at the University of Leiden in the Netherlands and a smaller one on interdisciplinary research from Evidence in Leeds. The main web page for what's now termed the "Research Excellence Framework" is here.

There'll be more here later in the day, but my first impressions are that there are few surprises in the proposals. Outside of six subject groupings (three for medicine, three for the rest of science), "light touch" peer review will be retained. In the sciences, universities will choose the staff to submit and assessment will then be based on bibliometrics, research income and numbers of postgraduate students.

There's no mention of selecting papers, so although it's not stated, it seems HEFCE is leaning towards assessing all publications, which could have a big impact on the willingness of academics to publish work they expect to get a low citation count, often for users. This isn't tackled directly, but HEFCE's flagging behavioural consequences as a concern and asking people to suggest additional metrics that could allow it to measure the value of research to users. 

Rather alarmingly for those who hoped the end of the RAE would eliminate a huge swathe of work, HEFCE talks about increases in workload in the short term. The issue seems to be in verifying who has actually published what, despite the promise of new technology.

And HEFCE's already flagging that the timetable for doing all of this is short and that it may struggle to find the expert resources it needs to make it all a success.

The utter pointlessness of Nick Clegg

The latest developments in the ongoing saga of the National Institute of Medical Research:

PART 1 Like all other important decisions on science in the UK, the future of the National Institute for Medical Research now lies with the Treasury. It’s weighing up whether to allow the MRC and chums to buy the three acre site behind the British Library near St Pancras station so that they can construct the world’s biggest biomedical lab around a relocated NIMR. No one can move until the men in charge of Northern Rock give the nod. But of course, our economic overlords won’t be taking responsibility for the decision or answering any questions about it. So the unaccountability of the UK
government grows.

PART 2 The land is actually owned by the Department of Culture, Media and Sport, which bizarrely—but totally consistently in the case of the celebrated anti-science department that still has no chief scientist—has tried to put the land on the open market. It’s the Two Cultures in action—it’s like something straight out of CP Snow’s The Corridors of Power.

PART 3 The site is in the Holborn and St Pancras constituency of Labour’s Frank Dobson. He’d prefer a local use but says the backing of Cancer Research UK and the Wellcome Trust will make the development very difficult to stop. So rather than lead Somerstown on a hopeless charge, he’s pushing for planning gain on the previous, smaller Temperance Hospital site bought by the consortium nearby on Hampstead Road.

PART 4 The site is also within the London Borough of Camden, now controlled by the Lib Dems with the aid of the Conservatives. Its borough plan has the site zoned as a mixture of local housing and other local-friendly amenities. Both Ming Campbell and Nick Clegg have been along to tell locals that the site must be saved for local use.

PART 5 Now that he wants to be Prime Minister, we called Clegg to find out what he thought should happen to the NIMR and why it was right in this case to put local concerns ahead of the future of a large chunk of Britain’s medical research. We phoned and phoned. We emailed. We left messages. Nada. Oops, could that be the embarrassed silence of a tinker toy politician we hear, one who thought it would be nice to play to the local gallery and wasn’t interested in the bigger questions?

Medical research is of course only one of a multitude of issues any leader has to deal with. But that’s why this story is telling. It’s not a case of vision and spin. It’s a matter of making a decision. The most damaging accusation levelled at Lib Dems has always been that they’re irrelevant. But Clegg is in a different league. The man who wants the keys to Number 10 is making an art out of evasive pointlessness.