Ton J. Cleophas
Clinical trials and p-values, beware of the extremes
Background: In randomized controlled trials, prior to statistical analysis, the data are checked for outliers and erroneous data. Statistical tests are, traditionally, not very good at distinguishing between errors and outliers, but they should be able to point out main endpoint results closer to expectation than compatible with random sampling.
Objectives: To explain from hypothesized and published examples why extreme p-values like p>0.95 and p<0.0001 may indicate that sampling was not completely random.
Results: Extreme p-values can be readily observed in recent issues ofhigh-impact journals. A p-value >0.95 literally means that we have a >95% chance off inding a result less close to expectation and, consequently, a <5% chance off inding a result this close or closer. Often in studies a statistical power of 80% is agreed upon, corresponding with a p-value of approximately 0.01. The ultimate p-value may then be a bit larger or smaller. However, a p-value much smaller than 0.01 will be rarely observed, because it would indicate that the study is overpowered. If the p-values can be assumed to follow a normal distribution around 0.01, then we will have a less than 5% chance ofobserving a p-value of <0.0001.
Conclusions: In randomized controlled trials, main endpoint p-values larger than p=0.95 will be rare, because they would indicate similarities closer than compatible with a normal distribution ofrandom data samples. Also very low p-values like p<0.0001 will be rarely encountered, because it would mean that the trial was overpowered and should have had a smaller sample size. It would seem appropriate, therefore, to require investigators to explain such results and to consider rejecting the research involved. So far, in randomized controlled trials the null-hypothesis is generally rejected at p<0.05. Perhaps we should consider rejecting the entire study if the main endpoint p-values are >0.95 or <0.0001.
Clinical Chemical Laboratory Medicine, Walter de Gruyter
Print ISSN: 1434-6621
Volume: 42, 03/2004
Pages: 300 - 304
Show full article (external site)
Show all available items of this journal
Search
