At the time of writing this post, the count of organisations signed up to the Declaration on Research Assessment (DORA) is 1,347. Results of the LIS-Bibliometrics 2018 Responsible Metrics State-of-the-Art survey showed a rise of engagement with DORA over a four-year period of annual surveys. But what is being assessed, and does this marry with what early career researchers are doing in the realms of Open Research? In his interesting blog post this week, Stephen Curry captures the essence of the conversation being had in many parts of the academic community at the moment -
" The community as a whole is still wrestling with the difficulty of balancing quantitative and qualitative aspects of assessment. The rapidly increasing scale of the research ecosystem, the diversification of outputs, and the different norms that adhere to different disciplines give quantitative indicators a lasting appeal. They seem to simplify the task of assessment in many ways, particularly for those operating outside the comfort zone of their expertise and particularly during initial triage when large numbers of applications may need to be evaluated. These are knotty and enduring problems that will require widespread culture change, but that change will only come about if it is lubricated by feasible and credible improvements to process. "
Though DORA doesn't mention Open Research in relation to assessment, it does encourage publishers to use open licenses on reference lists and citation data providers to be open and transparent with data and calculation methods. However, there are calls from the community for more change. Three organisations representing early-career and senior researchers in Europe have voiced the need to modernise researcher evaluation in their joint response to Plan S.
Who is starting to reward and incentivise Open Research, and how are they making it known? A few examples are linked to below, taken from a write-up of a workshop run by DORA in December 2018.
- A project on the Open Science Framework (OSF) is collecting faculty job advertisements that mention open science. The list includes the University of Essex, University of Toronto and a number of universities in Germany.
- The Charité Hospital in Berlin asks researchers applying for professorship positions to answer a set of questions related to research contributions, open science, team science and interactions with stakeholders.
- A list of institutions accepting publicly available preprints in hiring and promotion is being collated on the ASAPbio website. The list includes University of California, New York University and UC Davis.
- SPARC Europe is gathering data on the rewards and incentives for Open Science amongst funders in Europe - we'll be watching out for the results of this to report in a future blog post.
PlanS has shown that Open Research is firmly on the agenda at a senior level amongst research funders, though inevitable, it may take some time before we see it recognised and reflected in career progression.