The Metric Tide
Independent Review of the Role of Metrics in Research Assessment and Management
- James Wilsdon - University of Sussex, UK
‘Represents the culmination of an 18-month-long project that aims to be the definitive review of this important topic. Accompanied by a scholarly literature review, some new analysis, and a wealth of evidence and insight... the report is a tour de force; a once-in-a-generation opportunity to take stock.’
– Dr Steven Hill, Head of Policy, HEFCE, LSE Impact of Social Sciences Blog
‘A must-read if you are interested in having a deeper understanding of research culture, management issues and the range of information we have on this field. It should be disseminated and discussed within institutions, disciplines and other sites of research collaboration.’
– Dr Meera Sabaratnam, Lecturer in International Relations at the School of Oriental and African Studies, University of London, LSE Impact of Social Sciences Blog
Metrics evoke a mixed reaction from the research community. A commitment to using data and evidence to inform decisions makes many of us sympathetic, even enthusiastic, about the prospect of granular, real-time analysis of our own activities. Yet we only have to look around us at the blunt use of metrics to be reminded of the pitfalls.
Metrics hold real power: they are constitutive of values, identities and livelihoods. How to exercise that power to positive ends is the focus of this book. Using extensive evidence-gathering, analysis and consultation, the authors take a thorough look at potential uses and limitations of research metrics and indicators. They explore the use of metrics across different disciplines, assess their potential contribution to the development of research excellence and impact and consider the changing ways in which universities are using quantitative indicators in their management systems. Finally, they consider the negative or unintended effects of metrics on various aspects of research culture.
Including an updated introduction from James Wilsdon, the book proposes a framework for responsible metrics and makes a series of targeted recommendations to show how responsible metrics can be applied in research management, by funders, and in the next cycle of the Research Excellence Framework.
The metric tide is certainly rising. Unlike King Canute, we have the agency and opportunity – and in this book, a serious body of evidence – to influence how it washes through higher education and research.
I’m genuinely at a loss to describe how good The Metric Tide is. Something that could so easily have been a clunky and breathless paean to the oversold benefits of big data is nuanced, thoughtful and packed with evidence. Read it. Seriously, take it to the beach this summer. It’s that good.”
The Metric Tide provides a balanced and sensible perspective on the value metrics can bring to merit systems...It has been our consistent position that quantitative data inform, but do not and should not ever replace, peer review judgments of research quality – whether in the REF, or for any other purpose. Metrics can support human judgment and contribute to a fully rounded view on a research question being asked.
The Metric Tide is a must-read if you are interested in having a deeper understanding of research culture, management issues and the range of information we have on this field. It should be disseminated and discussed within institutions, disciplines and other sites of research collaboration.
Scientists like to grumble about the peer-review system for judging research quality, but there is one sure way to make most of them defend it: suggest that peer review should be replaced with numerical measures of academic output. A major UK report, The Metric Tide, reinforces this defence of the status quo. Metrics, it concludes, are not yet ready to replace peer review as the preferred way to judge research papers, proposals and individuals.
The Metric Tide represents the culmination of an 18-month-long project that aims to be the definitive review of this important topic. Accompanied by a scholarly literature review, some new analysis, and a wealth of evidence and insight...the report is a tour de force; a once-in-a-generation opportunity to take stock.
We strongly endorse the conclusions of the The Metric Tide.
We’ve just put down our virtual copies of The Metric Tide, a report compiled by an expert panel of academics, scientometricians, and university administrators on the role of bibliometrics and altmetrics in research assessment (including the UK’s next REF). What an excellent read...We look forward to seeing how the recommendations made in The Metric Tide play out in the coming years, both here in the UK and in academia worldwide, and to working together towards a future where metrics are used intelligently as part of a much wider scholarly agenda.
The report takes a middle line, but not simply a lazy middle ground. Its title, The Metric Tide, is an unequivocal statement that more data, and most likely more measurement, is coming our way. The data is not yet good enough to rely on in general, and it may never be, but pretending the tide isn’t rising won’t make it go away. So the report calls for engagement, but not passive acceptance. It calls for responsible use of indicators, for a critical engagement with what they can and can’t do, what they are and aren’t good for, and for applying them in context.
A major report published this week describes a “metric tide” that has washed over universities in recent years...In essence, it concludes that metrics are not, in fact, the work of the devil – as is so often the case, it’s how they are used that is key, with contextual information being vitally important (in the research excellence framework, for instance, metrics were used as background for panels that wanted it, but never as the sole criterion for judgement). For James Wilsdon, who led the review, there is also an underlying problem in the polarised tone of the discussion about them. The message of his review panel is that universities need to employ common sense in the way they use metrics: to be responsible and proportionate and human.
It is time for a critical conversation to take place about the use and abuse of metrics. Despite The Metric Tide’s chilling preface, announcing a “new barbarity” in our universities, we continue to witness the misuse of metrics as a tool of management in UK higher education... Universities should proceed with caution, then, lest metrics should spread like a digital Himalayan Balsam and undermine the ethical architecture of universities.
