How the Calculator Is Built and Tested
Every number on this site comes from one statistics engine. Here is how it computes, and the published results it must reproduce before any page is built.
At a glance
- Engine test cases
- 25 of 25 pass
- SciPy cross-check
- 8,100 values
- Engine version
- 1.0.0
- Last checked
- November 21, 2026
On this page
Who checks the numbers
There is no statistician on staff, and no outside statistician has reviewed the site. The check on every figure is mechanical instead: the build compares the engine with the published R, NIST, SciPy and textbook results listed below, and refuses to publish if any of them disagree. James Graham, the editor, reviews the pages and the test list; the about page says more about him and the publisher.
Numerical methods
Chi-square p-values are the regularized upper incomplete gamma function Q(k/2, χ²/2) (NIST DLMF chapter 8). The engine evaluates it with the power series when χ²/2 is below k/2 + 1 and with Legendre's continued fraction (modified Lentz method) otherwise, computing the tail that is needed directly so small p-values keep full relative accuracy. For large degrees of freedom the prefactor is computed in a form that avoids cancellation, following the approach described by DiDonato and Morris (DiDonato and Morris 1986).
Critical values start from the Wilson-Hilferty approximation and are refined by safeguarded Newton steps on the exact tail. Fisher's exact test adds hypergeometric probabilities computed on the log scale, with R's relative tolerance of one part in ten million when deciding which tables are "no more likely" than the observed one. R's conditional odds ratio and its exact confidence interval are found by bisection to full double precision.
The same engine file runs when the pages are built and in your browser, and one rounding function formats every number in both places, so the default results in the page and the results after you edit always agree.
Accuracy targets and results
Targets: relative error below one part in ten billion for p-values and critical values against SciPy, which uses the Boost math library, and agreement with every digit that R or NIST prints. R and SciPy differ from each other only in the last few digits of conditional odds ratio intervals, because R stops its root finder early; there we match R to four significant figures.
On the last build all 25 test cases passed, and all 8,100 stored SciPy 1.18.1 values (chi-square tails, quantiles and Fisher p-values for random inputs) agreed, with a worst relative error of 1.9 × 10−12. The build stops if any case fails.
Every test case
Kinds are labeled plainly: published reference output was printed by R or SciPy themselves; an official table is NIST's; a textbook example has a printed result we match; historical data means the counts are published but the result is our calculation; constructed cases are our own tables built to test a rule.
| No. | Kind | Source | Our value | Result |
|---|---|---|---|---|
| 1 | Published reference output (R or SciPy) | R chisq.test example: party identification by gender (Agresti) | χ² 30.0701, df 2, p 2.954 × 10−7 | Pass |
| 2 | Published reference output (R or SciPy) | R chisq.test example: 2x2 table, Yates correction | Yates χ² 0.6411, p 0.4233 | Pass |
| 3 | Published reference output (R or SciPy) | R chisq.test example: three counts against equal shares | χ² 2.5000, df 2, p 0.2865 | Pass |
| 4 | Published reference output (R or SciPy) | R chisq.test example: five counts against stated proportions | χ² 9.9901, df 4, p 0.0406 | Pass |
| 5 | Published reference output (R or SciPy) | R chisq.test example: small expected count warning | χ² 5.7947, df 4, p 0.2150 | Pass |
| 6 | Historical data; result computed by us | Mendel's pea counts (1866) against 9:3:3:1 | χ² 0.4700, df 3, p 0.9254 | Pass |
| 7 | Textbook or course example | OpenStax Introductory Statistics 2e, Example 11.6: volunteer hours | χ² 12.9909, df 4, p 0.0113 | Pass |
| 8 | Textbook or course example | OpenStax Introductory Statistics 2e, Example 11.2: absences by weekday | χ² 3.0000, df 4, p 0.5578 | Pass |
| 9 | Textbook or course example | University of Wisconsin Statistics 571 (2003): 3x3 table | χ² 1.8571, df 4, p 0.7620 | Pass |
| 10 | Published reference output (R or SciPy) | SciPy chi2_contingency documentation example (with G-test) | χ² 2.7778, df 2, p 0.2494 | Pass |
| 11 | Published reference output (R or SciPy) | SciPy chisquare documentation examples | χ² 2.0000, df 5, p 0.8491 | Pass |
| 12 | Published reference output (R or SciPy) | R fisher.test example: Fisher's tea tasting | two-sided p 0.4857 | Pass |
| 13 | Published reference output (R or SciPy) | R fisher.test example: convictions of twins (Fisher 1962) | two-sided p 5.367 × 10−4 | Pass |
| 14 | Published reference output (R or SciPy) | SciPy fisher_exact documentation example | two-sided p 0.0350 | Pass |
| 15 | Official table (NIST) | NIST/SEMATECH e-Handbook chi-square critical value table | df 1, 0.05: 3.841; df 100, 0.001: 149.449 | Pass |
| 16 | Reference value from SciPy | SciPy 1.17.1 reference values for tails and quantiles | 14 tail and quantile values | Pass |
| 17 | Our constructed case | Our 2x2 table testing the Yates cap and phi | Yates χ² 2.6667, Fisher p 0.1000 | Pass |
| 18 | Our constructed case | Our 2x2 table testing the N-1 chi-square (Campbell 2007) | N−1 χ² 12.1600, p 4.883 × 10−4 | Pass |
| 19 | Our constructed case | APA 7 formatting of a chi-square result | χ²(2, N = 2,757) = 30.07, p < .001 | Pass |
| 20 | Our constructed case | APA 7 formatting of a chi-square result | χ²(4, N = 839) = 12.99, p = .011 | Pass |
| 21 | Our constructed case | APA 7 formatting check for the shared number formatter | formatting matches | Pass |
| 22 | Our constructed case | APA 7 formatting check for the shared number formatter | formatting matches | Pass |
| 23 | Our constructed case | APA 7 formatting check for the shared number formatter | formatting matches | Pass |
| 24 | Published reference output (R or SciPy) | R mcnemar.test example: approval in two surveys (Agresti) | χ² 16.818, p 4.115 × 10−5 | Pass |
| 25 | Published reference output (R or SciPy) | R binom.test example: dwarf and giant plants (Conover) | p 0.3825 | Pass |
R is not installed on the build machine, so R values come from R's own reference output file, stats-Ex.Rout.save (R reference output), and NIST values from the e-Handbook (NIST table).
What the engine does not do yet
- Exact tests for tables larger than 2x2 (Freeman-Halton) and Monte Carlo p-values.
- A bias-corrected Cramér's V and confidence intervals for V.
- An exact multinomial test for goodness of fit.
Sources and recheck schedule
Software behavior is rechecked every 180 days, style guidance and published sources every 365 days. The build refuses to publish when a source is badly overdue.
Corrections
If you find a result that disagrees with R, SciPy or a textbook, please tell us through the contact page with the table you used. We check every report and record fixes in the changelog.
Sources
- R reference example output, stats-Ex.Rout.save checked November 21, 2026
- NIST/SEMATECH e-Handbook of Statistical Methods, 1.3.6.7.4: critical values of the chi-square distribution checked November 21, 2026
- NIST Digital Library of Mathematical Functions, chapter 8: incomplete gamma and related functions checked November 21, 2026
- DiDonato A.R. and Morris A.H. (1986). Computation of the incomplete gamma function ratios and their inverse. ACM Transactions on Mathematical Software 12(4):377-393 checked November 21, 2026
- SciPy documentation: scipy.special.gammainc checked November 21, 2026