Irish Humanities Alliance Brings Together Leading Academics to map out future of metrics for Arts and Humanities research in Ireland
‘Research systems everywhere are grappling with the same questions.’
A workshop hosted by the Trinity Long Room Hub Arts and Humanities Research Institute in partnership with the Irish Humanities Alliance and leading Irish higher educational institutes has addressed the pressing issue of metrics and research impact indicators for the Arts and Humanities.
The workshop held on February 1st forms part of a new series of lunchtime workshops on ‘Impact in the Humanities’ developed by the Irish Humanities Alliance (IHA) in partnership with Trinity College Dublin, University College Dublin, Maynooth University and University College Cork.
With two leading experts on research policy and metrics from the UK, the workshop compared developments in Ireland with the experiences of the review system in the UK. Professor James Wilsdon, University of Sheffield and Professor Geoffrey Crossick, University of London, both highlighted the need for HEI leaders to decide as a community on clear accepted principles around research measurement and to establish clearly what it is about Arts and Humanities research that needs to be captured. Professor Nicholas Canny, of the National University of Ireland, Galway appealed to early-career researchers to engage with the tools that are available to them within an institutional capacity and on an international level to ensure their publications are prominently chronicled. He also said researchers should be clear on the accepted and ‘esteemed’ practices relevant to their discipline. Professor Micheál Ó Siochrú of Trinity College Dublin highlighted academic freedom as the driver of top class research which he said is being eroded through forced models of assessment and alignment of research with institutional research priorities.
Measuring Quality over Activity
Jane Ohlmeyer, Director of the Trinity Long Room Hub, opened the workshop by addressing the need to rethink how Arts and Humanities research is measured. Professor Ohlmeyer, who is also the Chair of the Irish Research Council (IRC), spoke of the misleading picture presented of Arts and Humanities research outputs on the current research databases in Ireland. She warned against a system which merely measures quantity and not the quality of the research being produced. She argued that the only way to measure quality is through peer review. An attempt to capture this to date on a national scale, has been done through the global rankings, which are driven in large part by reputation (as determined by a global community of scholars and employers), and where Arts and Humanities in Ireland are the most highly ranked area.
James Wilsdon, co-author of ‘The Metric Tide’, spoke about how the debate in relation to research and impact has played out in the last number of years in the UK context, drawing on his experience as chair of the independent review of the role of metrics and indicators in the research system for the UK Higher Education Funding Council for England (HEFCE).
During the course of the review for the Research Excellence Framework (REF), Professor Wilsdon showed how the process highlighted the need to ‘retain the primacy of peer review within the system’, without discounting, however the need for quantitative data. Professor Crossick, who has also worked with HEFCE on an enquiry into monographs in the Arts, Humanities and Social Sciences and their relationship to open access to research, said that HEIs should emphasise the need for the research methodology to be rigorous arguing that quantitative and qualitative research should be treated equally. ‘We need to move away from the obsession with quantification as having a neutrality that qualitative evidence does not have – they both can be objective, they both can be subjective.’ He also warned against an approach which seeks to claim that the Arts and Humanities are a special case and demand different treatment to other research disciplines. He suggested that the question was not ‘which metrics are right but what are we trying to capture’. Professor Crossick, who is also the Director of the UK’s Arts and Humanities Research Council’s Cultural Value Project, said that it is crucial to match the mode of evaluation to the objectives.
Capturing Outputs
Professor John Boland, Dean of Research at Trinity College Dublin, challenged the Arts and Humanities research community to contribute to and shape a conversation on how outputs are captured and to ‘do a much better job at articulating the excellence of your research to the outside world.’ While Ireland does not have a REF as is the case in the UK, we still need to find a way to measure the quality of our outputs, and explain to others the value of this research.
Ms Niamh Brennan, Programme Manager for Research Informatics in Trinity College Library Dublin, mapped out the landscape for the development of Irish research metrics, which have to date, she said, taken a ‘utilitarian’ approach which seeks to show why investment in research has been ‘worthwhile.’ ‘There is no mention of cultural value’, explained Ms Brennan who highlighted the focus on jobs and commerciality in the Irish system. ‘If Humanities are unrepresented in the metrics, they are represented less so in the metrics in Ireland.’
Professor Wilsdon spoke to the issue around databases, which he said ‘are only as good as the numbers we put in.’ Professor Crossick said that in some cases the benefits of research cannot be monetized. He highlighted the use of case studies in providing a narrative and ‘telling a story’ around the importance of research which in many cases in the Arts and Humanities data alone cannot capture.
Early Career Researchers
Professor Nicholas Canny, who has served a five year term as a Chair of the Working Group on Open Access and Key Performance Indicators with the Scientific Council of the European Research Council, addressed his comments to those looking to advance their careers and early stage researchers who need to get their research published. Professor Canny said that there is now an expectation that everybody accounts for what they do and metrification is going to be part of this accountability process. He advised researchers to ensure that effective peer review processes are in place when collaborating with outside organisations, be those cultural institutions or otherwise. He argued that in the absence of a third party to attest to the quality of a researcher’s work, there is no record of how important that activity was, or what its impact is. He also advised researchers to engage more intensively with record keeping, ensuring that their research is inputted into their institution’s open archive repository.
Academic Freedom
Professor of Modern History at Trinity College Dublin, Micheál Ó Siochrú, who chairs Trinity College’s Library and Information Policy Committee, addressed the challenges of institutional metrics. He spoke of the need to ‘articulate an alternative vision to the intellectually bankrupt assessment models being forced upon’ academic institutions. He also cited the ‘chronic’ underfunding of the Irish university system, which despite its lack of resources and poor staff-student ratios, continues to perform alongside its global competitors. He credited this to a commitment to academic freedom. While ‘academic freedom is the engine which drives top-class research’, Professor Ó Siochrú argued that institutional strategic research planning risks marginalising many academics whose research does not fit the strategic objectives. He advocated a bottom-up, discipline-specific approach, developed from within the academic community and benchmarked across each of Ireland’s universities. This would address issues of accountability, while at the same time guaranteeing research quality.
Professor Daniel Carey, Chairman of the IHA, echoed the key points raised around methodology bias and the danger of alignment of research outputs with institutional research priorities. Professor Carey highlighted the increasingly ‘evidence based’ decision making at policy level and a logic of ‘marketisation’ in our society. He acknowledged the level at which other disciplines and countries are struggling with the same issues, but concluded by asking if an Irish approach to questions of research measurement might emerge from these discussions.
Niamh Brennan suggested that the humanities should take a lead in Ireland among all disciplines to articulate a different vision for research measurement. Professor Canny appealed to Irish research leaders to engage in extensive conversation with each other and with the Vice Presidents for Research to come to a common understanding around expectations of research activity. Professor Wilsdon also said that HEI leaders should have this debate as a community rather than reaching for a solution ‘off the shelf’. He said that HEI leaders should attempt to agree on clear principles on the approach to research.
Speaking at the workshop were Professor James Wilsdon of the University of Sheffield; Professor Geoffrey Crossick, University of London, and Chair of the Trinity Long Room Hub Board; Professor Nicholas Canny of the National University of Ireland, Galway; and Niamh Brennan and Professor Micheál Ó Siochrú of Trinity College Dublin. Professor Daniel Carey, Director of the Moore Institute (NUIG) and Chairperson of the Irish Humanities Alliance delivered the closing comments to the workshop audience. The event was chaired by Professor Jane Ohlmeyer, Director of the Trinity Long Room Hub.
Contact: Aoife King, Communications Officer |Trinity Long Room Hub | aoife.king@tcd.ie | 01 896 3895