Publication Bias and the Institutional Response: The Case of Nullius in Verba — Journal of Negative, Null and Failed Results

1. Introduction

Science's capacity for self-correction depends on its fidelity to what the evidence says — not to what the researcher hoped to find. Adopted shortly after the Royal Society's founding in 1660, the motto nullius in verba ("take nobody's word for it") expresses this principle in its most concise form. Drawn from Horace's Epistulae — "Nullius addictus iurare in verba magistri" (not bound to swear by the words of any master) — the phrase declares that scientific claims must rest not on authority but on facts determined by experiment (Royal Society, 1660/2024).

Publishing practice, however, has evolved in precisely the opposite direction. The published scientific record systematically excludes results that fail to confirm researchers' expectations; findings that do not support a hypothesis remain in file drawers on the grounds that they are "uninteresting," "unoriginal," or "unpublishable." This phenomenon transforms the scientific literature from a faithful record of what science has learned into a biased subset of it.

This article pursues three aims. First, it synthesizes the historical, statistical, and epistemological dimensions of publication bias in light of the academic literature. Second, it presents null result examples from various disciplines within a fluent narrative, evaluating their contributions to knowledge and their potential for paradigm rupture. Third, it critically evaluates the design and original mechanisms of Nullius in Verba (NiV), an institutional solution attempt addressing this problem.

2. Historical and Conceptual Background

2.1 Origins of the File Drawer Problem

The systematic documentation of publication bias begins with the work of psychologist Robert Rosenthal (1979). Rosenthal demonstrated that when meta-analytic syntheses rely solely on published studies, effect sizes are inflated because unpublished studies are systematically excluded. The "file drawer problem" metaphor captures the reality that negative results accumulate in researchers' drawers and never enter the literature.

The problem's origins, however, reach further back. Sterling (1959) found that 97% of articles published in psychology journals reported statistically significant results — a proportion dramatically higher than would be expected under the true distribution of effects. Smart (1964) documented a similar pattern in physics and chemistry. Greenwald (1975) showed that "the publication decision functions as a hypothesis test," revealing that editors consciously or unconsciously favor significant results.

2.2 Statistical Foundations

The statistical consequences of publication bias operate through two fundamental mechanisms. The first mechanism is selective reporting: even when no true effect exists in a research area, random sampling will cause a certain proportion of studies — typically 5% at the α = 0.05 significance level — to produce false positives. If these false positives are published while true negatives are not, the literature gradually becomes an accumulation consisting solely of erroneous positives. Ioannidis (2005) modeled these dynamics, showing that a substantial proportion of published research findings — especially in studies with small samples, low prior probabilities, and flexible designs — may be false. The second mechanism is selective analysis: Rosenthal's (1979) "fail-safe N" approach calculates how many unpublished null-result studies would be required to invalidate a published meta-analysis. This metric has been widely used as a quantitative indicator of publication bias. Egger and colleagues (1997) enabled the systematic detection of publication bias by statistically testing funnel plot asymmetry.

2.3 Empirical Evidence

Over the past two decades, empirical evidence of publication bias has accumulated strikingly. Turner and colleagues (2008) showed that only 51% of 74 clinical trials registered with the FDA were published, and that positive results were significantly more likely to be published than negative ones. Franco and colleagues (2014) tracked 221 studies in the social sciences, finding that strong results were twice as likely to be published as weak results. The Open Science Collaboration (2015) attempted to replicate 100 psychology studies and found that only 36% of replications produced statistically significant results, with effect sizes roughly half the magnitude of the originals.

3. Consequences of Publication Bias at the Field Level

The consequences of publication bias become apparent at four levels. First, when the published literature contains only significant results, the reported average effect size systematically exceeds the true population effect; this deviation is particularly pronounced in small-sample studies, because in small samples the observed effect must be substantially larger than the true effect in order to cross the significance threshold (Button et al., 2013). Second, although meta-analyses are regarded as the gold standard of systematic reviews, when they rely solely on published literature they are misled by precisely this bias; Kulinskaya and colleagues (2018) showed that meta-analyses without publication bias correction systematically overestimate effect sizes. Third, invisibility is a direct cause of resource waste: researchers unaware that another group has already failed repeat the same unproductive approaches; Chalmers and Glasziou (2009) estimate that up to 85% of the evidence produced in biomedical research is unusable or unnecessary. Fourth, a biased literature increases the probability that any given "positive" finding is a false positive; formulated by Ioannidis (2005), this consequence is not merely statistical but also epistemological, because the credibility of scientific claims depends on the integrity of the evidence base.

4. Examples of Null Results in the Scientific Literature: Categories, Contributions to Science, and Rupture Potential

Although null results are widely documented in the scientific literature, they are systematically rendered invisible in publishing practice. Yet the historical record shows that a significant portion of the most influential scientific advances arose from null results. This section presents selected null result examples from various disciplines within a fluent narrative, evaluating each example's contribution to knowledge and its potential to rupture existing paradigms.

4.1 Physics and the Basic Sciences

The most famous null result in the history of physics is the interferometry experiment conducted by Michelson and Morley in 1887. In nineteenth-century physics, a hypothetical medium called the "luminiferous aether" was accepted as the medium for light propagation, and the Earth's motion through this aether was expected to produce a measurable "aether wind." Michelson and Morley failed to detect the expected drag; the measured value fell far below the theoretical expectation and was assessed as within the noise level. This null result contributed directly to the development of the special theory of relativity, and the abandonment of the aether concept became one of the cornerstones of the transition in physics from Newtonian mechanics to Einsteinian relativity. This example carries revolutionary rupture potential in that it demonstrates how null results can invalidate not merely a hypothesis but a fundamental assumption of physical reality — absolute space-time and the aether.

Similarly, the failure to detect gravitational waves during LIGO's first data-taking period beginning in 2002 with first-generation detectors is an instructive null result. This result did not refute Einstein's general theory of relativity; on the contrary, it demonstrated that detector sensitivity was insufficient and determined the direction of technological improvements. Advanced LIGO's detection of gravitational waves in 2015 rested directly on the improvements motivated by this first null result. This example shows that null results can assume a constructive function — rather than refuting a theory, they can guide technological development to enhance its testability. In high-energy physics, null results produce a gradual rupture effect: findings such as the ATLAS experiment's exclusion of theoretically predicted production cross-sections for excited quarks below 6 TeV "close off" particular regions of probability space, constraining theoretical models and informing the design of future experiments. Although no single null result creates a paradigm shift on its own, their cumulative effects can change a field's direction over the long term.

4.2 Biomedical and Clinical Research

In clinical research, the value of null results becomes particularly evident in rendering visible the structural problems of the drug development process. In the field of idiopathic pulmonary fibrosis (IPF), three agents that had shown promising results in Phase II trials — zinprentraxin alfa, ziritaxestat, and pamrevlumab — failed at the Phase III stage. The STARSCAPE trial with zinprentraxin alfa found no significant difference between the placebo and treatment groups in forced vital capacity change among 664 patients (p = 0.54) and was terminated early for futility; the ziritaxestat trials were stopped due to safety concerns, and pamrevlumab produced a null result in the ZEPHYRUS-1 study. These failures reinforced the concept of "regression to the truth," suggesting that the positive Phase II results may have stemmed from statistical anomalies or outlier values in the placebo group rather than from genuine efficacy. Indeed, a post-hoc analysis of the zinprentraxin Phase II data revealed that the positive result stemmed from two extreme outliers in the placebo group. These examples point to structural problems such as inadequate Phase II sample sizes and overreliance on surrogate endpoints; they therefore carry methodological rupture potential in clinical trial design and drug development processes.

The selective publication of antidepressant trials is a striking example demonstrating that the invisibility of null results can produce direct clinical consequences. Turner and colleagues (2008) found that only 51% of 74 antidepressant trials registered with the FDA were published, and that positive results were significantly more likely to be published than negative ones. A meta-analysis based on the published literature was shown to overestimate the true effect size by as much as 32%. These findings revealed the concrete impact of publication bias on clinical decisions and contributed to the mandatory establishment of clinical trial registries such as ClinicalTrials.gov. This example is significant in demonstrating that the invisibility of null results can produce a rupture at the policy level — leading to erroneous treatment decisions.

4.3 Psychology and the Cognitive Sciences

The most visible example of null results in psychology concerns the replication attempts of Daryl Bem's 2011 study, published in a prestigious journal, which claimed to provide experimental evidence for "precognition" (the unconscious perception of future events). Numerous independent replication attempts produced null results; the meta-analysis by Galak and colleagues (2012) showed that the average effect size did not differ statistically from zero. However, Bem and colleagues' (2016) more comprehensive meta-analysis of 90 experiments reported a significant effect, leaving the field contested. This case became the most visible example of the replication crisis in psychology; it reinforced the principle that "extraordinary claims require extraordinary evidence" and accelerated the spread of preregistration and open data practices. This example shows that null results can create a rupture at the methodological and cultural levels, triggering a profound transformation in a discipline's research practices.

A similar transformation was observed in replication studies of the "facial feedback hypothesis" of Strack, Martin, and Stepper (1988). Following the original study testing the effect of holding a pen in the lips (inhibiting smiling) versus the teeth (facilitating smiling) on humor perception, Wagenmakers and colleagues (2016) obtained a null result in a multi-laboratory replication study with thousands of participants. This result raised the problem that theories in psychology lack a "paradigmatic" background; how "micro-theories" resting on a single effect should be interpreted in the face of null results remained unclear. Thus this null result not only refuted an effect but demonstrated the need to rethink psychology's theoretical architecture, revealing conceptual rupture potential.

4.4 Neuroscience and Physiology

In neuroscience, null results carry the capacity to challenge the universality of specific brain region-behavior mappings. Akil and colleagues (2024) tested the hypothesis that the right dorsolateral prefrontal cortex (rDLPFC) is a central region of the inhibitory control system; anodal tDCS stimulation was expected to increase inhibitory control and reduce approach behavior. However, in an experiment with 65 participants, tDCS showed no significant effect on frontal alpha asymmetry, N2 and P3 event-related potentials, or stop-signal reaction time. The authors evaluated the null finding with a Bayesian approach, overcoming the limitations of conventional p-value-based analysis. This null result weakens existing evidence regarding the role of the rDLPFC in inhibitory control and creates a theoretical limitation.

4.5 Social Sciences and Policy Research

In the social sciences, null results carry the capacity to demonstrate that widespread policy assumptions may lack empirical foundation. Contrary to the widespread view that women's representation in government reduces corruption, a study published in the Journal of Spurious Correlations found no positive relationship between women's representation and corruption. This null result calls into question assumptions regarding the relationship between gender and corruption and invites a reassessment of the evidentiary basis of policy recommendations. This example shows that null results can generate questioning at the policy level.

4.6 Applied Sciences and Engineering

In the applied sciences, null results render visible the limits of technological optimism. Although genetically modified (GM) crops are widely claimed to increase productivity, some independent studies have shown that yield increases in certain GM crops are not statistically significant. These null results reveal that the promises of GM technologies are context-dependent and cannot be generalized; they therefore carry rupture potential at the level of policy and technology assessment. Similarly, in the field of educational technologies, although digital learning platforms are widely claimed to increase student achievement, large-scale randomized controlled trials have shown that many educational technology interventions — for example, certain MOOC platforms and tablet-based interventions — have no significant effect on learning outcomes. These null results counterbalance exaggerated claims regarding the effectiveness of educational technologies, provide a basis for evidence-based education policy, and necessitate a reassessment of resource allocation.

4.7 Categorical Synthesis: A Rupture Typology of Null Results

When the above examples are considered together, it becomes evident that the scientific value of null results is not one-dimensional and that distinct rupture types can be distinguished. Revolutionary ruptures such as the Michelson-Morley experiment invalidate a fundamental physical or metaphysical assumption. Constructive ruptures such as LIGO's first null result do not refute a theory; they guide technological and methodological development. Gradual ruptures such as the ATLAS excited quark search narrow probability space and produce cumulative effects. Methodological ruptures such as the IPF Phase III trials and the Bem replications trigger reform in research design and reporting practices. Conceptual ruptures such as the facial feedback hypothesis case require the rethinking of theoretical architecture. Policy-level ruptures such as the antidepressant publication bias case directly affect clinical, educational, or policy decisions. Epistemological ruptures such as the chimpanzee Theory of Mind case clarify principles for interpreting null results. This typology shows that null results sometimes produce an immediate and dramatic effect, while in other cases they accumulate gradually and change a field's direction.

5. An Institutional Response: The Nullius in Verba Journal

5.1 Positioning of the Journal

Nullius in Verba — Journal of Negative, Null and Failed Results (NiV) is an international, peer-reviewed, Diamond Open Access journal published under AnKa Publisher. The journal takes its name from the Royal Society's nullius in verba motto and constructs a three-layered meaning through it: the principle of relying on evidence rather than authority; the fact that negative results are also "facts determined by experiment"; and the requirement that science treat no claim — including its own findings — as beyond questioning. The journal's positioning aims to fill the gap left after the closure of existing negative result journals — New Negatives in Plant Science, Journal of Negative Results in BioMedicine, All Results Journals. The closure of these journals reflects the structural difficulties faced by negative result publishing — low submission volume, financial sustainability problems, narrow scope. NiV aims to respond to these difficulties with an institutional backbone (AnKa Publisher) and broad interdisciplinary scope.

5.2 Aims and Scope

NiV's aims can be grouped under five headings: to reduce publication bias by providing a dedicated, indexed, and permanently archived venue for methodologically sound negative, null, and failed results; to prevent research waste by making unsuccessful approaches, abandoned hypotheses, and failed replications visible, discoverable, and citable; to improve the accuracy of meta-analyses by providing synthesis with the full evidence base rather than only the positive subset; to promote transparency and reproducibility through open peer review, open data, open reporting standards, and preregistration; and to recognize the scientific value of failure by developing editorial practices and metrics that assess rigor, contribution, and resource-saving potential rather than novelty or statistical significance.

The journal's scope encompasses a broad range from the natural sciences to the social sciences, from engineering to data science and artificial intelligence. Publication categories include negative results, null results, failed replications, abandoned hypotheses, failed or discontinued clinical trials, methodological failures, data notes, publication bias meta-research, and inconclusive exploratory findings. As a critical design decision, NiV does not accept "novelty," "impact," or statistical significance as acceptance criteria; evaluation is conducted on the basis of methodological soundness, ethical conduct, clarity of reporting, and contribution to the scientific record.

5.3 Editorial Model: Three-Tier Review

NiV's review model aims to balance rigor with efficiency. At the first tier, every submission undergoes an automated pre-check within 48 hours for plagiarism, statistical consistency, data integrity, ethical compliance, and reporting-standard compliance; this tier is not a decision point but a quality-assurance step, and its report is shared with authors before peer review begins. At the second tier, NiV practices open peer review: reviewer identities and full reports are published alongside the article, and author-reviewer dialogue is public. Reviewers evaluate against six criteria: methodological soundness (30%), scientific significance of the negative/null result (25%), transparency (15%), reproducibility (15%), clarity and presentation (10%), ethical compliance (5%). The mandatory question reviewers must answer is: "Why is this negative or null result scientifically meaningful?" Rejection on the grounds of "lack of interest" alone is not permitted; rejection must be justified on methodological or ethical grounds. This rule is a structural intervention in the review process: it prevents the editor and reviewer from acting on the assumption that "this result is not interesting." At the third tier, articles remain open for community comment for 90 days after publication; the most valuable contributions receive a DOI and a "Community Endorsement" label, turning publication into the beginning rather than the end of evaluation.

5.4 Registered Reports

NiV offers the Registered Reports format. In this format, the introduction, hypotheses, and methods (Stage 1) undergo peer review before results exist and can receive in-principle acceptance. This removes outcome-based bias from the editorial decision entirely — because there is no outcome yet to bias it. This mechanism is regarded as one of the most structural interventions against publication bias (Chambers, 2013).

5.5 Diamond Open Access and Equity

NiV is published under a Diamond Open Access model: neither authors nor readers pay fees; there are no submission charges, article processing charges, page or color charges, subscriptions, or paywalls. This choice is directly connected to the journal's mission, because publication bias is not only statistical but also a matter of equity. Fee-based models disproportionately affect researchers at underfunded institutions, early-career researchers, and scholars in the Global South. Diamond Open Access removes the ability to publish negative results from dependence on the ability to pay. Diamond Open Access is not a marginal experiment: according to DOAJ data, approximately 65% of open access journals worldwide are Diamond model; in South America this proportion reaches 95%, and in Europe 63%. Turkey actively supports Diamond compliant models through platforms such as DergiPark. NiV joins this global movement as a broad-scope negative result venue.

5.6 Sustainability and Financial Transparency

NiV's financial model is two-tiered, comprising primary and supplementary support. Primary support comes from the publisher AnKa Publisher: hosting, manuscript management system, DOI registration, ISSN maintenance, technical support, editorial coordination, and indexing support. Thanks to this backbone, the journal is not dependent on any external grant or donation. Supplementary support — from universities, libraries, public funding agencies, learned societies, foundations, and individual donors — is accepted only when consistent with the journal's mission, editorial independence, and Diamond Open Access principles. The safeguards are clear: no supporter receives preferential treatment in editorial decisions; no supporter influences review outcomes; no supporter gains privileged access to content; all support is disclosed transparently in the annual report. The journal also charges no author fees, charges no reader subscription fees, accepts no advertising that compromises editorial integrity, and accepts no support conditioned on editorial influence.

5.7 Original Metrics and Features

NiV offers a range of features distinguishing it from conventional journals. The Negative Impact Factor (NIF) is designed as a metric estimating the research resources saved by a published negative finding. The "Wasted Effort Saved" label provides an estimate of the number of redundant experiments the publication may prevent. The Failure Taxonomy tags each article by failure type (methodological, statistical, conceptual, resource-related, ethical). Living Negative Reviews provide continuously updated syntheses of negative evidence in a given area. The Cross-disciplinary Failure Graph offers a map linking failures in one field to advances in another. Preprint and Repository Integration enables integration with arXiv, bioRxiv, medRxiv, SSRN, OSF Preprints, Zenodo, Figshare, and Dryad. The most notable of these features is the NIF, because it attempts to measure the value of negative results along a dimension that traditional citation metrics cannot measure — resource saving. However, the operational definition and validity of this metric remain questions to be tested in the journal's future academic evaluation.

6. Critical Evaluation

6.1 Strengths

NiV's design strikingly overlaps with the intervention points identified in the publication bias literature. Dedicating a separate journal to negative results structurally prevents their exclusion on grounds of "lack of interest." Open peer review, by publishing reviewer reports, encourages reviewers to be more rigorous and transparent while enabling readers to judge the evaluation independently. Registered Reports remove outcome-based bias from the editorial decision entirely. Diamond Open Access responds to the equity problem by removing economic barriers. Taking methodological soundness and scientific contribution rather than "novelty" and "impact" as criteria provides conceptual consistency in evaluating negative results.

6.2 Potential Challenges and Open Questions

Nevertheless, NiV faces structural challenges. The historical experience of negative result journals shows that submission volume remains low; unless researchers are incentivized to publish their negative results — an incentive largely dependent on career evaluation systems — NiV's volume may remain limited. The operational definitions and validity of metrics such as the NIF and "Wasted Effort Saved" have not yet been academically tested; these metrics should be expected to complement rather than replace traditional metrics. It takes time for a new journal to enter indexes such as DOAJ, Scopus, and Web of Science, and during this period the journal's visibility and citation potential may remain limited. The cultural assumption that negative results are "worthless" can be overcome not only through editorial policies but also through the redesign of academic incentive and promotion systems. Finally, although the Diamond Open Access model rests on a publisher backbone, long-term sustainability will require supplementary support in areas such as indexing fees, language editing, and community outreach.

6.3 Position in the Literature

NiV is part of a growing meta-research ecosystem. This ecosystem includes journals offering the Registered Reports format (such as Cortex, Royal Society Open Science), journals dedicated to negative results (Journal of Negative Results in BioMedicine — closed; New Negatives in Plant Science — closed), meta-research journals examining publication bias (such as Research Integrity and Peer Review, Meta-Psychology), and open data and preregistration platforms (OSF, AsPredicted, ClinicalTrials.gov). NiV's distinctive position lies in its combination of broad interdisciplinary scope, Diamond Open Access model, and original metrics.

7. Conclusion and Recommendations

Publication bias is a structural and multidimensional problem documented for over half a century. The fact that the scientific literature contains only positive results produces cascading consequences: inflated effect sizes, distorted meta-analyses, research waste, and the erosion of scientific trust. This problem is related not to the good intentions of individual researchers but to the incentive structures of the publishing system. Historical examples of null results show that the scientific value of these results has been systematically underestimated. Across a broad spectrum — from the Michelson-Morley experiment to LIGO's first null result, from the IPF Phase III trials to the replication crisis in psychology — null results have stood at the center of science's self-correction mechanism. These examples demonstrate that null results cannot be reduced to a single rupture type — revolutionary, constructive, gradual, methodological, conceptual, policy-level, and epistemological ruptures are all possible.

Nullius in Verba — Journal of Negative, Null and Failed Results is designed as an institutional response to this structural problem. The journal's dedicated scope, open peer review model, Registered Reports format, Diamond Open Access principle, and original metrics constitute a multi-layered intervention against publication bias. The journal's name — the Royal Society's nullius in verba motto — declares that science must rely on evidence rather than authority, while reminding us that negative results are also "facts determined by experiment."

This article's recommendations are as follows. Career incentive systems should reward the publication of negative and null results; articles published in journals such as NiV should be regarded as equivalent in promotion and appointment processes. Funding agencies should support infrastructure and journal initiatives aimed at reducing publication bias. Universities and libraries should provide institutional support to Diamond Open Access journals. Meta-researchers should empirically evaluate the impact of journals such as NiV — for example, testing the validity of the NIF metric, the effect of open peer review on quality, and the role of the Registered Reports format in reducing publication bias. Journal editors, in turn, should learn from NiV's experience and create special issues, sections, or formats for negative results in their own journals. Science learns not only from what works but also from what does not. The principle nullius in verba — take nobody's word for it — requires that science treat no claim, including its own findings, as beyond questioning. Negative results are an inseparable part of this principle; rendering them invisible weakens science's capacity for self-correction.