This post has been cross-posted on the SLSA Blog and on Amanda\u2019s blog Approaching Law.<\/p>\n
The Hidden REF<\/a> project is on a mission to get \u2018diverse\u2019 outputs to form five percent of all outputs submitted to REF. They note that REF rules have long allowed, and are now more actively encouraging, the submission of non-traditional research outputs (NTOs). Furthermore, \u2018data released by the REF suggests that non-traditional outputs perform just as well as traditional outputs in terms of the proportion awarded a 4* rating\u2019. Yet Higher Education Institutions remain reticent to submit them.<\/p>\n A recent episode of What the REF<\/a>, a podcast series produced by the Hidden REF project, prompted me to wonder: What might output diversity look like in Law?<\/p>\n REF2021<\/a> Guidance defined \u2018outputs\u2019 as the \u2018products\u2019 of research; and \u2018research\u2019 as a \u2018process of investigation leading to new insights, effectively shared\u2019. \u2018Effectively shared\u2019 was defined as meaning that the output is in \u2018assessable form\u2019, whether \u2018published\u2019, \u2018publicly available\u2019, or in a \u2018confidential report\u2019 (Annex C)<\/a>.<\/p>\n It emphasised that, \u2018In addition to printed academic work, research outputs may include, but are not limited to: new materials, devices, images, artefacts, products and buildings; confidential or technical reports; intellectual property, whether in patents or other forms; performances, exhibits or events; and work published in non-print media.\u2019 (Para 208<\/a>)<\/p>\n Nevertheless, across all disciplines, 98 percent of output submitted to REF2021 were\u00a0 \u2018publications\u2019\u2014that is books or articles (up from 97 precent in REF2014<\/a>). Things appear to have been marginally worse than average in Law. By my count, 99.99 percent of the 5,855 outputs submitted to Panel 18 (Law) in REF2021<\/a> were entirely traditional\u2014that is, journal articles (3752), book chapters (1121), books (872), or edited collections (26). Of the remaining .01 percent of submitted outputs, all but two were highly traditional in the sense that there were text-based (working papers, reports, or conference contributions). More on those exceptional two below.<\/p>\n The evolving REF2029 Guidance on the Contributions to Knowledge and Understanding<\/a> places a special emphasis on \u2018diversity\u2019 of research outputs (5.6).<\/p>\n The REF2029 guidance retains the REF2021 definition of research outputs. It also follows REF2021 in noting that research outputs \u2018go well beyond journal articles, conference papers, monographs and book chapters\u2019 (5.6.4); and \u2018may include innovative formats, accessible reporting formats, and modes of expression that may depart from historical conventions within specific fields and contexts\u2019 (5.6.3).<\/p>\n However, it goes further than REF2021 in the sense that it promises to assess Units on the extent to which the outputs that they submit are \u2018representative<\/a>\u2019\u2014that is, \u2018reflect the diversity of research practices and outputs produced within the disciplinary area\u2019, \u2018including the extent to which different and diverse types of research are included in the submission\u2019 (7.1.1).<\/p>\n What kinds of outputs are eligible for submission to REF2029? The Guidance on Disciplinary Level Evidence Statements is still in development. For now we have an indicative list provided in section 5.6<\/a> of the general Guidance on the Contributions to Knowledge and Understanding. This mentions textual outputs of the kind which legal researchers have long undertaken, but which have not historically been celebrated in the wider research system (e.g. reviews, translations, policy summaries, and reports). It also mentions \u2018non-textual (artefacts, audio, visual, multi-modal)\u2019 and \u2018creative, performance and practice-led outputs\u2019.<\/p>\n As legal researchers increasingly turn to visual<\/a>, material, and creative<\/a> methods, Units will presumably need to include some of the resulting outputs if their REF2029 submissions are to be deemed \u2018representative\u2019. This turn seems to be especially prevalent among postgraduate and early career researchers. One recent example that comes to mind is (In)visible<\/a>, a short choreopoetry film by PhD candidate Bhumika Billa<\/a>, made in creative response<\/a> to testimonies of injustice she had gathered through interviews with female legal professionals in India.<\/p>\n How might such diverse outputs be presented in order to showcase their quality (originality, significance, and rigour)? We can look to the two primarily non-textual submissions made to REF2021 for inspiration. Both were presented as, in REF terminology, \u2018multi-component\u2019 outputs\u2014that is, comprised of several inter-connected pieces of work. In both cases, the quality of the non-textual research outputs was articulated through textual analysis, as well as a coherent (textual and visual) web-based narrative.<\/p>\n \u2018It stays with you: Use of force by UN Peacekeepers in Haiti<\/a>\u2019 (led by Siobhan Wills<\/a>, submitted by University of Ulster) was \u2018a body of associated work demonstrating sustained research effort and a complex multi-layered process of creative investigation\u2019. It emerged from two projects: one focusing on law enforcement by the Brazilian-led UN Stabilisation Mission in Haiti (MINUSTAH), the other on \u2018militarised policing\u2019 by Brazilian troops and police in Rio de Janeiro. It included two films co-produced with marginalised communities in Haiti and Brazil nine short videos produced by community members, and an article focused on the methodology, aims and outcomes of the project; all of which are gathered on a website<\/a> with links to independent films reviews, festival screenings and prizes.<\/p>\n \u2018Display at your own risk<\/a>\u2019 (led by Andrea Wallace<\/a>, submitted by University of Lancaster), was \u2018a research-led exhibition experiment\u2019. It emerged from an investigation into the use by cultural heritage institutions of \u2018digital surrogates\u2019 (digital reproductions of material artefacts). It included gallery and open-source versions of an \u2018exhibition intended for public use\u2019, and a series of reflective essays; all gathered together in an engaging and well-presented website with accessible explanations of the project rationale and methodology.<\/p>\n The requirement that a submission be \u2018representative\u2019 might encourage the submission of diverse outputs. But it will not necessarily promote the production of diverse outputs.<\/p>\n One pathway to incentivising Units to proactively support the production of diverse practices and outputs is to highlight their potential to generate engagement and impact. Indeed, it is reasonable to expect \u2018It stays with you\u2019 and \u2018Display at your own risk\u2019 to feature in the Engagement and Impact<\/a> dimension of REF2029 submissions.<\/p>\n Another pathway is the evolving People Culture and Environment (PCE)<\/a> dimension of REF2029.<\/p>\n The PCE Pilot Exercise<\/a> held in 2025 was \u2018structured around five factors which enable positive research culture\u2019, none of which focused directly on supporting diversity in research practices and outputs. However, the fifth factor, \u2018Development\u2019, begins with the words \u2018Recognising and valuing the breadth of activities, practices and roles involved in research\u2026\u2019 (PCE pilot exercise guidance, para 17<\/a>).<\/p>\n We shall see.<\/p>\n","protected":false},"excerpt":{"rendered":"Allowing diversity in REF2021<\/h3>\n
Requiring \u2018representative\u2019 submissions in REF2029<\/h3>\n
Submitting diverse Law outputs to REF2029<\/h3>\n
\u2018It stays with you\u2019<\/h3>\n
\u2018Display at your own risk\u2019<\/h3>\n
Promoting the production of diverse outputs<\/h3>\n