San Francisco Declaration on Research Assessment

Last updated
San Francisco Declaration on Research Assessment
Formation13 May 2013;10 years ago (2013-05-13)
HeadquartersInternational organization
LeaderStephen Curry, Imperial College London
Website sfdora.org

The San Francisco Declaration on Research Assessment (DORA) is a statement that denounces the practice of correlating the journal impact factor to the merits of a specific scientist's contributions. Also according to this statement, this practice creates biases and inaccuracies when appraising scientific research. It also states that the impact factor is not to be used as a substitute "measure of the quality of individual research articles, or in hiring, promotion, or funding decisions". [1]

Contents

The declaration originated from the December 2012 meeting of the American Society for Cell Biology, and was published on May 13, 2013, signed by more than 150 scientists and 75 scientific organizations. [1] [2] The American Society for Cell Biology states that, as of 30 May 2013, there were more than 6,000 individual signatories to the declaration and that the number of scientific organizations "signing on has gone from 78 to 231" within two weeks. [3] As of 14 December 2017, the number of individual signatories has risen to over 12,800 and the number of scientific organizations to 872. [4] Some organization signatories in 2017 include the British Library, Nature Research, BioMed Central, Springer Open and Cancer Research UK. [5]

DORA is also an organization whose mission is to "advance practical and robust approaches to research assessment globally and across all scholarly disciplines". [6]

Motivation

On 16 December 2012, a group of editors and publishers of academic journals gathered at the Annual Meeting of The American Society for Cell Biology in San Francisco to discuss current issues related to how the quality of research output is evaluated and how the primary scientific literature is cited. [7]

The motivation behind the meeting was the consensus that impact factors for many cell biology journals do not accurately reflect the value to the cell biology community of the work published in these journals. The group therefore wanted to discuss how to better align measures of journal and article impact with journal quality.

All the above considerations also extend to other fields and the organizers consider DORA "a worldwide initiative covering all scholarly disciplines". In fact, the declaration has been signed by scientific associations with general scope (such as the American Association for the Advancement of Science and the Academy of Sciences of the Czech Republic), by more specialized associations working in fields quite removed from biology (such as the European Mathematical Society, Geological Society of London, and Linguistic Society of America), as well as by some universities and other general institutions (such as the Higher Education Funding Council for England).

The outcome of the meeting and further discussions was a set of recommendations that is referred to as the San Francisco Declaration on Research Assessment, published in May 2013. [7]

Signatories

As of 3 May 2021, 2203 organisations and 17,354 individuals had signed the declaration, including universities, research institutes, learned societies and funding bodies from around the world. [8] On 20 May 2020 Springer Nature became the largest research publisher to sign the declaration. [9]

Implementation

In November 2019, Dutch universities and research funders announced the introduction of a new system of research recognition and reward, based on the DORA principles. [10]

In November 2019, in the context of its roadmap for open science, the French national research agency CNRS announced the introduction of a system of individual assessment of researchers based on the DORA principles. [11]

In March 2021, in the context of a reorganization, the university of Liverpool used the field-weighted citation-impact metric for determining which faculty jobs were at risk of being cut. This happened although that university was a signatory of DORA. This prompted the DORA organization, as well as the authors of the Leiden Manifesto, to contact the university for raising concerns. [12]

In June 2021, Utrecht University announced that it would no longer use the impact factor in hiring and promotion decisions. This decision was partly inspired by the DORA declaration, which the university had signed in 2019. [13]

Activities of the organization

Demo of the Reformscape database developed by DORA

DORA is working on Project TARA (Tools to Advance Research Assessment), which aims to "facilitate the development of new policies and practices for academic career assessment". [14]

As part of Project TARA, DORA has launched the database Reformscape, which collates policies on research assessment from research institutions. [15]

See also

Related Research Articles

The impact factor (IF) or journal impact factor (JIF) of an academic journal is a scientometric index calculated by Clarivate that reflects the yearly mean number of citations of articles published in the last two years in a given journal, as indexed by Clarivate's Web of Science.

Scientometrics is the field of study which concerns itself with measuring and analysing scholarly literature. Scientometrics is a sub-field of informetrics. Major research issues include the measurement of the impact of research papers and academic journals, the understanding of scientific citations, and the use of such measurements in policy and management contexts. In practice there is a significant overlap between scientometrics and other scientific fields such as information systems, information science, science of science policy, sociology of science, and metascience. Critics have argued that over-reliance on scientometrics has created a system of perverse incentives, producing a publish or perish environment that leads to low-quality research.

Citation analysis is the examination of the frequency, patterns, and graphs of citations in documents. It uses the directed graph of citations — links from one document to another document — to reveal properties of the documents. A typical aim would be to identify the most important documents in a collection. A classic example is that of the citations between academic articles and books. For another example, judges of law support their judgements by referring back to judgements made in earlier cases. An additional example is provided by patents which contain prior art, citation of earlier patents relevant to the current claim. The digitization of patent data and increasing computing power have led to a community of practice that uses these citation data to measure innovation attributes, trace knowledge flows, and map innovation networks.

Timothy John Mitchison is a cell biologist and systems biologist and Hasib Sabbagh Professor of Systems Biology at Harvard Medical School in the United States. He is known for his discovery, with Marc Kirschner, of dynamic instability in microtubules, for studies of the mechanism of cell division, and for contributions to chemical biology.

<span class="mw-page-title-main">Marc Kirschner</span> American biologist

Marc Wallace Kirschner is an American cell biologist and biochemist and the founding chair of the Department of Systems Biology at Harvard Medical School. He is known for major discoveries in cell and developmental biology related to the dynamics and function of the cytoskeleton, the regulation of the cell cycle, and the process of signaling in embryos, as well as the evolution of the vertebrate body plan. He is a leader in applying mathematical approaches to biology. He is the John Franklin Enders University Professor at Harvard University. In 2021 he was elected to the American Philosophical Society.

<span class="mw-page-title-main">Bruce Alberts</span> American biochemist (born 1938)

Bruce Michael Alberts is an American biochemist and the Chancellor’s Leadership Chair in Biochemistry and Biophysics for Science and Education, emeritus at the University of California, San Francisco. He has done important work studying the protein complexes which enable chromosome replication when living cells divide. He is known as an original author of the "canonical, influential, and best-selling scientific textbook" Molecular Biology of the Cell, and as Editor-in-Chief of Science magazine.

Citation impact or citation rate is a measure of how many times an academic journal article or book or author is cited by other articles, books or authors. Citation counts are interpreted as measures of the impact or influence of academic work and have given rise to the field of bibliometrics or scientometrics, specializing in the study of patterns of academic impact through citation analysis. The importance of journals can be measured by the average citation rate, the ratio of number of citations to number articles published within a given time period and in a given index, such as the journal impact factor or the citescore. It is used by academic institutions in decisions about academic tenure, promotion and hiring, and hence also used by authors in deciding which journal to publish in. Citation-like measures are also used in other fields that do ranking, such as Google's PageRank algorithm, software metrics, college and university rankings, and business performance indicators.

The h-index is an author-level metric that measures both the productivity and citation impact of the publications, initially used for an individual scientist or scholar. The h-index correlates with success indicators such as winning the Nobel Prize, being accepted for research fellowships and holding positions at top universities. The index is based on the set of the scientist's most cited papers and the number of citations that they have received in other publications. The index has more recently been applied to the productivity and impact of a scholarly journal as well as a group of scientists, such as a department or university or country. The index was suggested in 2005 by Jorge E. Hirsch, a physicist at UC San Diego, as a tool for determining theoretical physicists' relative quality and is sometimes called the Hirsch index or Hirsch number.

The American Society for Cell Biology (ASCB) is a professional society that was founded in 1960.

<span class="mw-page-title-main">Elaine Fuchs</span> American biologist

Elaine V. Fuchs is an American cell biologist famous for her work on the biology and molecular mechanisms of mammalian skin and skin diseases, who helped lead the modernization of dermatology. Fuchs pioneered reverse genetics approaches, which assess protein function first and then assess its role in development and disease. In particular, Fuchs researches skin stem cells and their production of hair and skin. She is an investigator at the Howard Hughes Medical Institute and the Rebecca C. Lancefield Professor of Mammalian Cell Biology and Development at The Rockefeller University.

Journal Citation Reports (JCR) is an annual publication by Clarivate. It has been integrated with the Web of Science and is accessed from the Web of Science Core Collection. It provides information about academic journals in the natural and social sciences, including impact factors. The JCR was originally published as a part of the Science Citation Index. Currently, the JCR, as a distinct service, is based on citations compiled from the Science Citation Index Expanded and the Social Sciences Citation Index. As of the 2023 edition, journals from the Arts and Humanities Citation Index and the Emerging Sources Citation Index will also be included.

Journal ranking is widely used in academic circles in the evaluation of an academic journal's impact and quality. Journal rankings are intended to reflect the place of a journal within its field, the relative difficulty of being published in that journal, and the prestige associated with it. They have been introduced as official research evaluation tools in several countries.

<span class="mw-page-title-main">Web of Science</span> Online subscription index of citations

The Web of Science is a paid-access platform that provides access to multiple databases that provide reference and citation data from academic journals, conference proceedings, and other documents in various academic disciplines. Until 1997, it was originally produced by the Institute for Scientific Information. It is currently owned by Clarivate.

<span class="mw-page-title-main">Anthony A. Hyman</span> British biologist

Anthony Arie Hyman is a British scientist and director at the Max Planck Institute of Molecular Cell Biology and Genetics.

<i>eLife</i> Open-access scientific journal

eLife is a not-for-profit, peer-reviewed, open access, science publisher for the biomedical and life sciences. It was established at the end of 2012 by the Howard Hughes Medical Institute, Max Planck Society, and Wellcome Trust, following a workshop held in 2010 at the Janelia Farm Research Campus. Together, these organizations provided the initial funding to support the business and publishing operations. In 2016, the organizations committed US$26 million to continue publication of the journal.

Zena Werb was a professor and the Vice Chair of Anatomy at the University of California, San Francisco. She was also the co-leader of the Cancer, Immunity, and Microenvironment Program at the Hellen Diller Family Comprehensive Cancer Center and a member of the Executive Committee of the Sabre-Sandler Asthma Basic Research Center at UCSF. Her research focused on features of the microenvironment surrounding cells, with particular interest in the extracellular matrix and the role of its protease enzymes in cell signaling.

Jessica Polka is a biochemist and the Executive Director of ASAPbio, a non-profit initiative promoting innovation and transparency via preprints and open peer review. She was one of the organizers of a recent meeting they held on scholarly communication.

<span class="mw-page-title-main">Rebecca Heald</span> American cell and developmental biologist

Rebecca W. Heald is an American professor of cell and developmental biology. She is currently a Professor in the Department of Molecular & Cell Biology at the University of California, Berkeley. In May 2019, she was elected to the National Academy of Sciences. She has published over 120 research articles in peer reviewed journals.

The Leiden Manifesto for research metrics (LM) is a list of "ten principles to guide research evaluation", published as a comment in Volume 520, Issue 7548 of Nature, on 22 April 2015. It was formulated by public policy professor Diana Hicks, scientometrics professor Paul Wouters, and their colleagues at the 19th International Conference on Science and Technology Indicators, held between 3–5 September 2014 in Leiden, The Netherlands.

<span class="mw-page-title-main">David G. Drubin</span> American biologist, academic, and researcher

David G. Drubin is an American biologist, academic, and researcher. He is a Distinguished Professor of Cell and Developmental Biology at the University of California, Berkeley where he holds the Ernette Comby Chair in Microbiology.

References

  1. 1 2 Alberts, Bruce (May 17, 2013). "Impact Factor Distortions". Science . 340 (6134): 787. Bibcode:2013Sci...340..787A. doi: 10.1126/science.1240319 . PMID   23687012.
  2. Van Noorden, Richard (May 16, 2013). "Scientists join journal editors to fight impact-factor abuse". Nature News Blog.
  3. Fleischman, John (May 30, 2013). "Impact Factor Insurrection Catches Fire with Over 6,000 Signatures and Counting". The American Society for Cell Biology.[ permanent dead link ]
  4. "Dora - ASCB". ASCB. Retrieved 14 December 2017.
  5. "Dora - ASCB". ASCB. Retrieved 2017-05-01.
  6. "About DORA". DORA. 2024-01-10. Retrieved 2024-01-31.
  7. 1 2 Cagan, R. (2013). "San Francisco Declaration on Research Assessment" (PDF). Disease Models & Mechanisms . 6 (4): 869–870. doi:10.1242/dmm.012955. PMC   3701204 . PMID   23690539.
  8. "Signers – DORA". sfdora.org. Retrieved 2021-05-03.
  9. "Springer Nature enticed by DORA aura". Research Information. Retrieved 2020-06-09.
  10. "Dutch universities and research funders move away from the impact factor". ScienceGuide. 18 November 2019. Retrieved 22 November 2019.
  11. "Feuille de route du CNRS pour la science ouverte" (PDF). Retrieved 27 November 2019.
  12. Else, Holly (2021-03-25). "Row erupts over university's use of research metrics in job-cut decisions". Nature. Springer Science and Business Media LLC. 592 (7852): 19. Bibcode:2021Natur.592...19E. doi: 10.1038/d41586-021-00793-7 . ISSN   0028-0836. PMID   33767466.
  13. Woolston, Chris (2021-06-25). "Impact factor abandoned by Dutch university in hiring and promotion decisions". Nature. Springer Science and Business Media LLC. 595 (7867): 462. Bibcode:2021Natur.595..462W. doi: 10.1038/d41586-021-01759-5 . ISSN   0028-0836. PMID   34172959. S2CID   235647170.
  14. "Project TARA". DORA. 2024-01-29. Retrieved 2024-01-31.
  15. Owens, Brian (2024-01-30). "How to make academic hiring fair: database lists innovative policies". Nature. Nature Publishing Group. doi:10.1038/d41586-024-00273-8. S2CID   267333016 . Retrieved 2024-01-31.

Further reading