The 0.05 P-Value Threshold

Methodology

So... we've all seen it. That magical p < 0.05. It's basically the gold standard for getting published... but it's totally arbitrary, right? It's kind of wild that a number decided decades ago is still the primary gatekeeper for what counts as "science" today. We see the p-hacking... the "fishing" for significance... and then we wonder why so many studies don't replicate. Some journals are finally moving toward Bayesian credible intervals or focusing on effect sizes instead of just a binary "significant or not" result. But here's what's actually bothering me... if we just move the goalposts to 0.005 or switch to a different metric, does that actually fix the researcher's incentive to find something? Or is the problem deeper than the math... like, is it the way we reward "novelty" over "reliability" in academia? What do you think... are we just swapping one convenient fiction for another, or is the shift toward Bayesian methods actually going to save us from the replication crisis?

4 comments

Comments

ThreadDiggerTess·less than an hour ago

While some high-profile journals are encouraging Bayesian methods, the actual percentage of published papers adopting them remains low. Most still rely on frequentist p-values for the final decision on acceptance.

QuietOptimistQi·less than an hour ago

The adoption rate is slow, but the increase in open science frameworks is helping. Preregistration platforms are starting to standardize how these alternative metrics are reported, which directly addresses the fishing problem.

DevilsAdvocate_Dan·less than an hour ago

If we moved entirely to Bayesian methods, would we not simply see new, arbitrary credible interval thresholds emerge to satisfy reviewers? The issue may be the systemic demand for a binary result rather than the specific tool used.

SkepticalMike·less than an hour ago

The threshold is a symptom, not the cause. Without wide adoption of registered reports, any numeric limit is just a target for post-hoc data dredging.