Unjournal | Transparent Evaluation of Impactful Research

Home

Lifecycle Journal aims to put publishing and evaluation in the control of the scholarly community itself. As a 3-year research and development project initiated by the Center for Open Science (COS), the ultimate goal is to transform the vision and value of a journal as an effective facilitator of knowledge production and self-correction while aligning the rewards of publishing with transparency and rigor. It commits to transparency in innovation, evaluation, and continuous improvement through research in order to raise the quality and trustworthiness of the research, reduce bias and increase overall accountability.

A New Paradigm for Scientific Publishing, Peer Review, and Impact Assessment

Scientific publishing and peer review have evolved little in three centuries, while the demands placed on them have grown profoundly. The growing role of artificial intelligence has underscored deep, systemic shortcomings of an aging system that has largely evaded innovation, a system whose origins are appallingly closer to the invention of the printing press than to the internet. We can do better – much better. This article is intended as the beginning of a communal experiment: a living document that critically reviews the modern academic publishing and peer-review system and presents a concrete framework to address what bibliometrics experts¹ have characterized as "the pervasive misapplication of indicators to the evaluation of scientific performance". Building on the Leiden Manifesto, DORA, and a body of scholarship spanning many disciplines and decades, we present a community-governed, non-profit platform organized around three trust-weighted impact factors, for articles, authors, and reviewers, with full algorithmic transparency, an open development log, and structural decoupling of credibility scoring from content moderation and from monetization. We invite the community to discuss, critique, and help shape it.

Why we built the Journal of Research on Research and what it cost us - LSE Impact

Gemma Derrick, Bart Penders, Serge Horbach & Tony Ross-Hellauer reflect on the choices, challenges & compromises of launching the Journal of Research on Research

Open Evaluation: A Vision for Entirely Transparent Post-Publication Peer Review and Rating for Science

The two major functions of a scientific publishing system are to provide access to and evaluation of scientific papers. While open access (OA) is becoming a reality, open evaluation (OE), the other side of coin, has received less attention. Evaluation steers the attention of the scientific community and thus the very course of science. It also influences the use of scientific findings in public policy. The current system of scientific publishing provides only journal prestige as an indication of the quality of new papers and relies on a non-transparent and noisy pre-publication peer review process, which delays publication by many months on average. Here I propose an OE system, in which papers are evaluated post-publication in an ongoing fashion by means of open peer review and rating. Through signed ratings and reviews, scientists steer the attention of their field and build their reputation. Reviewers are motivated to be objective, because low-quality or self-serving signed evaluations will negatively impact their reputation. A core feature of this proposal is a division of powers between the accumulation of evaluative evidence and the analysis of this evidence by paper evaluation functions (PEFs). PEFs can be freely defined by individuals or groups (e.g. scientific societies) and provide a plurality of perspectives on the scientific literature. Simple PEFs will use averages of ratings, weighting reviewers (e.g. by H-factor) and rating scales (e.g. by relevance to a decision process) in different ways. Complex PEFs will use advanced statistical techniques to infer the quality of a paper. Papers with initially promising ratings will be more deeply evaluated. The continual refinement of PEFs in response to attempts by individuals to influence evaluations in their own favor will make the system ungameable. OA and OE together have the power to revolutionize scientific publishing and usher in a new culture of transparency, constructive criticism, and collaboration.

La comunicación cientí­fica en América Latina es abierta, colaborativa y no comercial. Desafí­os para las revistas

Latin America, where research and journals that publish research results, are both activities that are mainly publicly-funded, is the region of the world with the greatest adoption of open access, to scientific journals, highlighting scholarly-led regional initiatives such as Latindex, SciELO, Redalyc, which are complemented by national and institutional open access journal portals, all collaborative non-commercial initiatives. In this article, and based on CLACSO's experience in the region and its participation in international debates on the future of open access scholarly communications, views are shared on some of the challenges that this collaborative open access model poses to journals published in countries of Latin America. As is the case of open science requirements for journals, the need to have open access indicators so that evaluation processes can assess the contributions from research output published in journals within the region, and the urgency to strengthen collaborative non-commercial models to manage open access and its indicators.

Econ Journal Watch: Scholarly Comments on Academic Economics

Econ Journal Watch (EJW) is an online academic journal publishing Comments on economics research, articles appearing in economics journals, and the ec…

Landing — Open Journals Collective

We're a collective of libraries, university presses, and non-profit publishers reshaping academic research.

Publishing Beyond the Market: Open Access, Care, and the Commons

<I>Publishing Beyond the Market</I> argues that the move to open access should focus less on the free accessibility of research outputs and more on who controls the publications and infrastructures for scholarly communication. By deploying theoretical literature on science and technology studies, care ethics, and the commons, the book critically interrogates open access and reimagines a more ethical future for researcher-led publishing. A case study of Plan S—the multifunder European policy for open access publishing—explores its tendency to rehearse all the failures of commercialisation. Through critical engagement with the open access landscape, the book reveals the shortcomings of market-centric and policy-based approaches to open access book and journal publishing, particularly their tendency to reinforce conservatism, commercialism, and private control of publishing.<BR /><BR /> Going forward, <I>Publishing Beyond the Market</I> explores the importance of collectivity and democratic governance within the transition to open access publishing. It suggests that developing a commons-based, scholar-led publishing landscape through a series of presses that are each managed by working academics could offer a productive counterpoint to marketised systems of open access and subscription publishing. In weaving themselves together in order to "scale small" these publishing initiatives would act as a counter-hegemonic project based on mutual reliance and care. By illustrating how these projects build toward a commons-based publishing future, and how they may complement other approaches to publishing within university presses and libraries, the book culminates in an argument for the infrastructures, policies, and forms of governance needed to nurture such a collective vision.

The Dissemination of Scholarly Information: Journals, Open-Access and Distributed Filtering

Current methods of disseminating scholarly information focus on the use of journals who retain exclusive rights in the material they publish. Recently there has been increasing dissatisfaction with this model, with suggestions for alternative approaches such as "Open Access".

In an era where research evaluation methods are evolving, the Research Contribution Claim Network makes trustworthy tracking of non-traditional research output easy!

In this whitepaper, Patrick Hochstenbach (Ghent University Library), Thomas van Himbergen (SURF), Laurents Sesink (SURF) and Herbert Van de Sompel (DANS) introduce the...

Why the Stockholm Declaration will never work

Abstract The Stockholm Declaration calls for a moral reformation of scientific publishing—replacing commercial publishers with scholar-led journals, rewarding quality over quantity and instituting independent oversight. While noble in spirit, it fundamentally misdiagnoses the disease. The crisis of science stems not from paper mills or profit motives but from systemic incentives that reward volume, novelty and prestige over validity and reliability. Non-profit ownership will not neutralize the prestige economy, sleuth-led policing cannot scale, and ‘integrity registries’ would merely replace metrics with surveillance. True reform requires rebuilding the publishing infrastructure as a public utility: publicly funded, free to publish and read, and stripped of hierarchy and financial incentives.

Center for Open Science

At the Center for Open Science, our mission is to increase openness, integrity, and reproducibility of scholarly research. Promoting these practices within the research funding and publishing communities accelerates scientific progress

Reformation of science publishing: the Stockholm Declaration

Science relies on integrity and trustworthiness. But scientists under career pressure are lured to purchase fake publications from ‘paper mills’ that use AI-generated data, text and image fabrication. The number of low-quality or fraudulent publications is rising to hundreds of thousands per year, which—if unchecked—will damage the scientific and economic progress of our societies. The result is editor and reviewer fatigue, irreproducible experiments, misguided experiments, disinformation and escalating costs that devour funding from taxpayers intended for research. It is high time to reevaluate current publishing models and outline a global plan to stop this unhealthy development. A conference was therefore organized by the Royal Swedish Academy of Sciences to draft an action plan with specific recommendations, as follows. (i) Academia should resume control of publishing using non-profit publishing models (e.g. diamond open-access). (ii) Adjust incentive systems to merit quality, not quantity, in a reputation economy where the gaming of publication numbers and citation metrics distorts the perception of academic excellence. (iii) Implement mechanisms to prevent and detect fake publications and fraud which are independent of publishers. (iv) Draft and implement legislations, regulations and policies to increase publishing quality and integrity. This is a call to action for universities, academies, science organizations and funders to unite and join this effort.

A budding discipline around how science gets read and judged - Aris

Scholarly publishing spent thirty years arguing about access. It has barely begun arguing about what readers do with the paper once they have it. At Aris we call that second argument scholarly interface design. Mike Morrison's community calls it ScienceUX. Either way it is becoming a field, and here is why it is worth your attention.

The strain on scientific publishing

Scientists are increasingly overwhelmed by the volume of articles being published. Total articles indexed in Scopus and Web of Science have grown exponentially in recent years; in 2022 the article total was approximately ~47% higher than in 2016, which has outpaced the limited growth - if any - in the number of practising scientists. Thus, publication workload per scientist (writing, reviewing, editing) has increased dramatically. We define this problem as the strain on scientific publishing. To analyse this strain, we present five data-driven metrics showing publisher growth, processing times, and citation behaviours. We draw these data from web scrapes, requests for data from publishers, and material that is freely available through publisher websites. Our findings are based on millions of papers produced by leading academic publishers. We find specific groups have disproportionately grown in their articles published per year, contributing to this strain. Some publishers enabled this growth by adopting a strategy of hosting special issues, which publish articles with reduced turnaround times. Given pressures on researchers to publish or perish to be competitive for funding applications, this strain was likely amplified by these offers to publish more articles. We also observed widespread year-over-year inflation of journal impact factors coinciding with this strain, which risks confusing quality signals. Such exponential growth cannot be sustained. The metrics we define here should enable this evolving conversation to reach actionable solutions to address the strain on scientific publishing.

annotations: Les économistes orthodoxes sont incapables d’expliquer les inégalités domestiques et la compétition entre les nations

Blog de sciences économiques et sociales de Martin Anota. Traductions, revues de presse.