Summary of changes made to p-curve app



Last update: 2017 11 30


Questions? Email Uri


App 4.06    (2017.11.30)
  • Added half p-curve result for flat test to figure legend and table beneath it.

    • App 4.052    (2017.02.28) (R Code for 4.052)
  • More minor changes to accommodate APS strict copy-editing requirements (e.g., spaces before/after "=")

    • App 4.051    (2017.02.24)
  • Minor changes to legend of figures to accommodate APS strict copy-editing requirements.

    • App 4.05    (2016.06.17)
  • Modified main figure to accommodate journal styles. Test results now reported within figure legend and in separate table rather than in embedded table.

    • App 4.04    (2016.06.10)
  • Fixed bug that dropped green line for 33% power when including n.s. results

    • App 4.03    (2016.04.08)
  • Expanded y-axis of diagnostic plot for power so that extreme results were still viewable

    • App 4.02    (2016.03.30)
  • Fixed bug that p-values below precision level of R (p<2.2e-16) led algorithm that estimates power to crash
  • Fixed typo in number of studies total vs significant.

    • App 4.01    (2016.03.06)
  • Fixed bug in computation of binomial test for 33% power

    • App 4.00    (2016.01.10)
    Most of the changes follow from our third paper on p-curve: "Better P-Curves" (SSRN)
  • Started reporting (90%) Confidence interval for power estimate (computed by finding the level of power that leads to expected p-curves that lead the Stouffer test comparing observed with expected have p=.05 and p=.95)
  • Started reporting results for 'half p-curve' (p<.025 results)
  • New test of evidential value: is half p-curve p<.05 or both full and half p-curve p<.1?
  • New test of inadequate evidential value: is flatter than 33% power test p<.05 for full p-curve or p<.1 for both the half and binomial flatter than 33% power?
  • Interpretation of reported results, in terms of overall conclusions, printed below summary of results
  • Statistical results embedded within the summary figure.
  • Cumulative p-curve adjusts for exclusions, testing whether pp-values are uniform between the highest dropped pp-value and 1, rather than between 0 and 1.
  • Left-skew results no longer reported (the overall p-value for left skew is 1 minus the p-value for right-skew)
  • Power is estimated by finding the level of power that leads to an overall p-value for Stouffer's test of p=.5 rather than by minimizing Kolmogorov-Smirnov D stat against uniform
  • Modified computation of 33% power for t-tests and Z-tests. Up to App3 power was computed for directional predictions (tested with two-sided tests), starting with App4 non-directional ones. This ensures that t() & F(), and Z & chi(), obtain equivalent non-centrality parameters (since F() and chi() are not directional tests). So up to App3, the noncentrality parameter would be set so that there is a 1/3 probability of obtaining a positive effect that is p<.05 (two sided). Starting with App4, so that there is a 1/3 probability of obtaining a positive or negative effect that is p<.05 (two-sided).

    • App 3.01    (2015.04.13)
  • Fixed bug with p-values smaller than precision level of R (p<2.2e-16)

    • App 3.0
  • Estimates publication bias corrected statistical power
  • Cumulative p-curve analyses, reports results without most extreme p-values
  • Detailed explanations for calculations using user entered data as example
  • Uses Stouffer instead of Fisher's method to aggregate pp-values


    • App 2.0
  • New user interface, instead of entering the attributes of each test one at a time, they are all entered together into a large text-area following a pre-specified syntax.