Significance Testing in Natural and Biological Systems: A Review

It is generally accepted that the reproducibility crisis in the fields of natural and biological sciences is in part due to the misuse of the Null Hypothesis Significance Testing (NHST). We review the shortcomings in the use of NHST and then go beyond these to consider additional issues. Many natural systems are time-varying and some are scale-free, which requires design of new methods for such cases. We also consider the problem from the perspective of information efficiency and since three-way logic is superior to two-way logic, we argue that adding a third hypothesis may be beneficial in certain applications.