Summary of changes made to p-curve app
Last update: 2024 08 28
Questions? Email
Uri
App 4.10    (2024.08.28)
Fixed bug identified by Richard Morey where the half p-curve test for null of 33% power was too conservative, conditioning twice on p<.05.
The line of code changed is
- old: prop25 = 3*prop33(.025)
- new: prop25 = prop33(.025)
App 4.06    (2017.11.30) (R Code for 4.06)
Added half p-curve result for flat test to figure legend and table beneath it.
App 4.052    (2017.02.28) (R Code for 4.052)
More minor changes to accommodate APS strict copy-editing requirements (e.g., spaces before/after "=")
App 4.051    (2017.02.24)
Minor changes to legend of figures to accommodate APS strict copy-editing requirements.
App 4.05    (2016.06.17)
Modified main figure to accommodate journal styles. Test results now reported within figure legend and in separate table
rather than in embedded table.
App 4.04    (2016.06.10)
Fixed bug that dropped green line for 33% power when including n.s. results
App 4.03    (2016.04.08)
Expanded y-axis of diagnostic plot for power so that extreme results were still viewable
App 4.02    (2016.03.30)
Fixed bug that p-values below precision level of R (p<2.2e-16) led algorithm that estimates power to crash
Fixed typo in number of studies total vs significant.
App 4.01    (2016.03.06)
Fixed bug in computation of binomial test for 33% power
App 4.00    (2016.01.10)
Most of the changes follow from our third paper on
p-curve: "Better P-Curves" (
SSRN)
Started reporting (90%) Confidence interval for power estimate (computed by finding the level of power that
leads to expected p-curves that lead the Stouffer test comparing observed with expected have p=.05 and p=.95)
Started reporting results for 'half p-curve' (p<.025 results)
New test of evidential value: is half p-curve p<.05 or both full and half p-curve p<.1?
New test of inadequate evidential value: is flatter than 33% power test p<.05 for full p-curve or p<.1 for both the half and binomial flatter than 33% power?
Interpretation of reported results, in terms of overall conclusions, printed below summary of results
Statistical results embedded within the summary figure.
Cumulative p-curve adjusts for exclusions, testing whether pp-values are uniform between the highest dropped pp-value and 1, rather than between 0 and 1.
Left-skew results no longer reported (the overall p-value for left skew is 1 minus the p-value for right-skew)
Power is estimated by finding the level of power that leads to an overall p-value for Stouffer's test of p=.5 rather
than by minimizing Kolmogorov-Smirnov D stat against uniform
Modified computation of 33% power for t-tests and Z-tests. Up to App3 power was computed for directional predictions (tested with two-sided tests),
starting with App4 non-directional ones. This ensures that t() & F(), and Z & chi(), obtain equivalent non-centrality parameters (since F() and chi() are not directional tests).
So up to App3, the noncentrality parameter would be set so that there is a 1/3 probability of obtaining a positive effect that is p<.05 (two sided).
Starting with App4, so that there is a 1/3 probability of obtaining a positive or negative effect that is p<.05 (two-sided).
App 3.01    (2015.04.13)
Fixed bug with p-values smaller than precision level of R (p<2.2e-16)
App 3.0
Estimates publication bias corrected statistical power
Cumulative p-curve analyses, reports results without most extreme p-values
Detailed explanations for calculations using user entered data as example
Uses Stouffer instead of Fisher's method to aggregate pp-values
App 2.0
New user interface, instead of entering the attributes of each test one at a time,
they are all entered together into a large text-area following a pre-specified syntax.