Skip to content

VALIDATION.md

Latest commit

 

History

History
231 lines (198 loc) · 14.7 KB

VALIDATION.md

File metadata and controls

231 lines (198 loc) · 14.7 KB
Raw

Validation report

Generated by python docs/validate.py. Every number is recomputed on each run; CI fails if any check fails.

151 / 151 checks pass.

References: NIST Statistical Reference Datasets (certified values), published statistical tables, textbook examples, independent implementations (SciPy, statsmodels), exact analytic results, and large Monte Carlo simulations for coverage and unbiasedness.

Note on D4: tables compute D4 = 1 + 3 d3/d2 from 3-decimal d2 and d3; engstat integrates exactly, so D4(3) = 2.5746 vs the tabulated 2.574 (the tolerance for D4 is one unit in the last place).

Linear regression - NIST StRD Longley (higher difficulty)

Check Reference engstat Reference value Status
beta0 estimate NIST certified -3482258.635 -3482258.635 pass
beta0 std. deviation NIST certified 890420.3836 890420.3836 pass
beta1 estimate NIST certified 15.06187227 15.06187227 pass
beta1 std. deviation NIST certified 84.91492577 84.91492577 pass
beta2 estimate NIST certified -0.03581917929 -0.03581917929 pass
beta2 std. deviation NIST certified 0.03349100777 0.03349100777 pass
beta3 estimate NIST certified -2.020229804 -2.020229804 pass
beta3 std. deviation NIST certified 0.4883996817 0.4883996817 pass
beta4 estimate NIST certified -1.033226867 -1.033226867 pass
beta4 std. deviation NIST certified 0.2142741632 0.2142741632 pass
beta5 estimate NIST certified -0.05110410565 -0.05110410565 pass
beta5 std. deviation NIST certified 0.2260732001 0.2260732001 pass
beta6 estimate NIST certified 1829.151465 1829.151465 pass
beta6 std. deviation NIST certified 455.4784991 455.4784991 pass
residual standard deviation NIST certified 304.8540736 304.8540736 pass
R-squared NIST certified 0.9954790046 0.9954790046 pass
regression sum of squares NIST certified 184172401.9 184172401.9 pass
residual sum of squares NIST certified 836424.0555 836424.0555 pass
F statistic NIST certified 330.2853392 330.2853392 pass

Nonlinear regression - NIST StRD Misra1a (lower difficulty)

Check Reference engstat Reference value Status
b1 (start [500, 0.0001]) NIST certified 238.9421305 238.9421292 pass
b2 (start [500, 0.0001]) NIST certified 0.0005501564282 0.0005501564318 pass
b1 (start [250, 0.0005]) NIST certified 238.9421298 238.9421292 pass
b2 (start [250, 0.0005]) NIST certified 0.0005501564302 0.0005501564318 pass
std. deviation of b1 NIST certified 2.707007884 2.707007524 pass
std. deviation of b2 NIST certified 7.266869774e-06 7.266868844e-06 pass
residual standard deviation NIST certified 0.1018787633 0.1018787633 pass

Control-chart constants (computed by integration) vs published tables

Check Reference engstat Reference value Status
d2(n=2) Montgomery, SQC, App. VI 1.128379167 1.128 pass
d2(n=3) Montgomery, SQC, App. VI 1.692568751 1.693 pass
d2(n=4) Montgomery, SQC, App. VI 2.058750746 2.059 pass
d2(n=5) Montgomery, SQC, App. VI 2.325928947 2.326 pass
d2(n=6) Montgomery, SQC, App. VI 2.534412721 2.534 pass
d2(n=7) Montgomery, SQC, App. VI 2.704356751 2.704 pass
d2(n=8) Montgomery, SQC, App. VI 2.847200612 2.847 pass
d2(n=9) Montgomery, SQC, App. VI 2.970026324 2.97 pass
d2(n=10) Montgomery, SQC, App. VI 3.077505462 3.078 pass
d3(n=2) Montgomery, SQC, App. VI 0.8525024664 0.853 pass
d3(n=3) Montgomery, SQC, App. VI 0.888368004 0.888 pass
d3(n=4) Montgomery, SQC, App. VI 0.8798082027 0.88 pass
d3(n=5) Montgomery, SQC, App. VI 0.8640819408 0.864 pass
d3(n=6) Montgomery, SQC, App. VI 0.8480396861 0.848 pass
d3(n=7) Montgomery, SQC, App. VI 0.8332053356 0.833 pass
d3(n=8) Montgomery, SQC, App. VI 0.8198314897 0.82 pass
d3(n=9) Montgomery, SQC, App. VI 0.8078342745 0.808 pass
d3(n=10) Montgomery, SQC, App. VI 0.7970506734 0.797 pass
c4(n=2) Montgomery, SQC, App. VI 0.7978845608 0.7979 pass
c4(n=3) Montgomery, SQC, App. VI 0.8862269255 0.8862 pass
c4(n=4) Montgomery, SQC, App. VI 0.9213177319 0.9213 pass
c4(n=5) Montgomery, SQC, App. VI 0.939985603 0.94 pass
c4(n=6) Montgomery, SQC, App. VI 0.9515328619 0.9515 pass
c4(n=10) Montgomery, SQC, App. VI 0.9726592741 0.9727 pass
c4(n=25) Montgomery, SQC, App. VI 0.9896403756 0.9896 pass
A2(n=2) Montgomery, SQC, App. VI 1.879971206 1.88 pass
A2(n=3) Montgomery, SQC, App. VI 1.023326708 1.023 pass
A2(n=4) Montgomery, SQC, App. VI 0.7285971859 0.729 pass
A2(n=5) Montgomery, SQC, App. VI 0.5768193341 0.577 pass
A2(n=6) Montgomery, SQC, App. VI 0.4832460243 0.483 pass
A2(n=10) Montgomery, SQC, App. VI 0.3082637252 0.308 pass
D4(n=2) Montgomery, SQC, App. VI 3.266531919 3.267 pass
D4(n=3) Montgomery, SQC, App. VI 2.57459129 2.574 pass
D4(n=4) Montgomery, SQC, App. VI 2.282051561 2.282 pass
D4(n=5) Montgomery, SQC, App. VI 2.114499145 2.114 pass
D4(n=7) Montgomery, SQC, App. VI 1.924292257 1.924 pass
D4(n=10) Montgomery, SQC, App. VI 1.776977344 1.777 pass
D3(n=5) Montgomery, SQC, App. VI 0 0 pass
D3(n=7) Montgomery, SQC, App. VI 0.07570774251 0.076 pass
D3(n=8) Montgomery, SQC, App. VI 0.1361709959 0.136 pass
D3(n=10) Montgomery, SQC, App. VI 0.2230226559 0.223 pass
in-control ARL, 3-sigma chart 1 / (2 Phi(-3)) 370.3983473 370.3983473 pass
ARL for 1-sigma shift, n = 1 Montgomery Table 5.x: 43.9 43.89468172 43.9 pass

Design of experiments - Montgomery 2^4 filtration-rate example

Check Reference engstat Reference value Status
effect A Montgomery DAE, Ex. 6.2 21.625 21.625 pass
effect B Montgomery DAE, Ex. 6.2 3.125 3.125 pass
effect C Montgomery DAE, Ex. 6.2 9.875 9.875 pass
effect D Montgomery DAE, Ex. 6.2 14.625 14.625 pass
effect AB Montgomery DAE, Ex. 6.2 0.125 0.125 pass
effect AC Montgomery DAE, Ex. 6.2 -18.125 -18.125 pass
effect AD Montgomery DAE, Ex. 6.2 16.625 16.625 pass
effect BC Montgomery DAE, Ex. 6.2 2.375 2.375 pass
effect BD Montgomery DAE, Ex. 6.2 -0.375 -0.375 pass
effect CD Montgomery DAE, Ex. 6.2 -1.125 -1.125 pass
effect ABC Montgomery DAE, Ex. 6.2 1.875 1.875 pass
effect ABD Montgomery DAE, Ex. 6.2 4.125 4.125 pass
effect ACD Montgomery DAE, Ex. 6.2 -1.625 -1.625 pass
effect BCD Montgomery DAE, Ex. 6.2 -2.625 -2.625 pass
effect ABCD Montgomery DAE, Ex. 6.2 1.375 1.375 pass
active effects by Lenth ME = {A, C, D, AC, AD} Montgomery DAE, Ex. 6.2 1 1 pass

Confidence and tolerance intervals

Check Reference engstat Reference value Status
one-sided normal tolerance k (n=10, 90 %, 95 %) published tables 2.354640132 2.355 pass
one-sided normal tolerance k (n=20, 99 %, 95 %) published tables 3.295156936 3.295 pass
two-sided tolerance: achieved confidence (n=10, 90%, k=2.8563) Monte Carlo, 200 000 trials 0.950065 0.95 pass
two-sided tolerance: achieved confidence (n=10, 99%, k=4.4369) Monte Carlo, 200 000 trials 0.94954 0.95 pass
two-sided tolerance: achieved confidence (n=30, 95%, k=2.8509) Monte Carlo, 200 000 trials 0.98994 0.99 pass

Power and sample size

Check Reference engstat Reference value Status
n per group, two-sample, d = 1.0, power 0.8 R power.t.test (n = 16.71, 63.77, 393.4) 17 17 pass
n per group, two-sample, d = 0.5, power 0.8 R power.t.test (n = 16.71, 63.77, 393.4) 64 64 pass
n per group, two-sample, d = 0.2, power 0.8 R power.t.test (n = 16.71, 63.77, 393.4) 394 394 pass

ANOVA

Check Reference engstat Reference value Status
one-way ANOVA F scipy.stats.f_oneway 26.16981132 26.16981132 pass

Reliability

Check Reference engstat Reference value Status
Weibull MLE shape (complete data) scipy weibull_min.fit(floc=0) 2.257777326 2.257777474 pass
Weibull MLE scale (complete data) scipy weibull_min.fit(floc=0) 951.2386371 951.2386944 pass
censored Weibull MLE shape scipy logpdf/logsf likelihood 2.027933273 2.027933261 pass
censored Weibull MLE scale scipy logpdf/logsf likelihood 1006.65333 1006.653324 pass
Kaplan-Meier S(t=1) hand calculation 0.8333333333 0.8333333333 pass
Kaplan-Meier S(t=2) hand calculation 0.6666666667 0.6666666667 pass
Kaplan-Meier S(t=3) hand calculation 0.4444444444 0.4444444444 pass
Kaplan-Meier S(t=4) hand calculation 0.2222222222 0.2222222222 pass
median rank, i=1 of 10 (Benard) (1 - 0.3)/(10 + 0.4) 0.06730769231 0.06730769231 pass

Measurement uncertainty

Check Reference engstat Reference value Status
GUM: u(L W), first order analytic sqrt((W uL)^2 + (L uW)^2) 0.05 0.05 pass
Welch-Satterthwaite nu_eff hand calculation 12.32876712 12.32876712 pass
GUM with correlation r = 0.5 sqrt(ua^2 + uc^2 + 2 r ua uc) 0.2645751311 0.2645751311 pass
Monte Carlo u(L W) (2e6 trials) exact: sqrt(W^2uL^2 + L^2uW^2 + uL^2uW^2) 0.04998302927 0.0500004 pass
rectangular +/-1: standard uncertainty 1/sqrt(3) 0.5774204295 0.5773502692 pass
rectangular +/-1: 95 % symmetric interval, upper 0.95 0.9500873588 0.95 pass

Gauge R&R and capability (unbiasedness / coverage by simulation)

Check Reference engstat Reference value Status
mean repeatability variance over 400 studies (true 0.0625) simulation truth; tolerance 3 SE 0.06227857873 0.0625 pass
mean operator variance over 400 studies (true 0.09) simulation truth; tolerance 3 SE 0.09227439736 0.09 pass
mean part variance over 400 studies (true 4.0) simulation truth; tolerance 3 SE 4.008564616 4 pass
Cp 95 % CI coverage (true Cp = 1, n = 50, 2000 sims) nominal 0.95; tolerance 3 SE 0.9515 0.95 pass

Real measurement data (NIST StRD files bundled with engstat; certified values parsed from the headers)

Check Reference engstat Reference value Status
SiRstv.dat (silicon resistivity, 5 probes): between-group SS NIST certified 0.0511462616 0.0511462616 pass
SiRstv.dat: within-group SS NIST certified 0.21663656 0.21663656 pass
SiRstv.dat: F statistic NIST certified 1.180462374 1.180462374 pass
SiRstv.dat: R-squared NIST certified 0.1909990391 0.1909990391 pass
SiRstv.dat: residual standard deviation NIST certified 0.1040760683 0.1040760683 pass
AtmWtAg.dat (atomic weight of silver, 2 instruments): between-group SS NIST certified 3.638341875e-09 3.638341875e-09 pass
AtmWtAg.dat: within-group SS NIST certified 1.049517292e-08 1.049517292e-08 pass
AtmWtAg.dat: F statistic NIST certified 15.94673357 15.94673357 pass
AtmWtAg.dat: R-squared NIST certified 0.2574265445 0.2574265445 pass
AtmWtAg.dat: residual standard deviation NIST certified 1.510483144e-05 1.510483144e-05 pass
Norris (ozone-monitor calibration): intercept NIST certified -0.2623230738 -0.2623230738 pass
Norris: slope NIST certified 1.002116818 1.002116818 pass
Norris: residual standard deviation NIST certified 0.8847963961 0.8847963961 pass
Norris: R-squared NIST certified 0.9999937459 0.9999937459 pass
Chwirut2 (ultrasonic calibration): b1 NIST certified 0.1665766671 0.1665766654 pass
Chwirut2 (ultrasonic calibration): b2 NIST certified 0.005165329182 0.005165329129 pass
Chwirut2 (ultrasonic calibration): b3 NIST certified 0.012150007 0.0121500071 pass

Acceptance sampling

Check Reference engstat Reference value Status
OC curve n=50, c=2 at p=4 %: vs explicit binomial sum exact 0.6767140041 0.6767140041 pass
hypergeometric OC (lot of 100 000) approaches binomial limit N -> infinity 2.768652057e-06 0 pass
smallest plan for AQL 1.0%, LTPD 5%: n (c = 3) brute-force search 132 132 pass
smallest plan for AQL 0.5%, LTPD 3%: n (c = 3) brute-force search 221 221 pass
smallest plan for AQL 2.0%, LTPD 8%: n (c = 4) brute-force search 98 98 pass
AOQL, finite lot (n=80, c=2, N=2000, hypergeometric) brute force over D = 0..N defectives 0.01639559751 0.01639559751 pass
AOQL, binomial model (N = 2000 for scaling) fine grid search 0.01642675309 0.01642675309 pass
ATI at p = 0 (every lot accepted) = n exact 80 80 pass
ATI at p = 1 (every lot rejected) = N exact 2000 2000 pass
variables plan, known sigma: Pa at AQL target 1 - alpha = 0.95 0.9525081009 0.95 pass
variables plan, known sigma: Pa at LTPD target beta = 0.10 0.09664803116 0.1 pass
variables plan, unknown sigma (Wallis): exact Pa at AQL target 0.95 0.9527863933 0.95 pass
variables plan, unknown sigma (Wallis): exact Pa at LTPD target 0.10 0.1056522543 0.1 pass
unknown-sigma OC (noncentral t) at p = 3 % vs simulation 200 000 simulated lots 0.4124339261 0.41477 pass

Non-normal capability

Check Reference engstat Reference value Status
percentile method with a normal fit: Pp = tolerance / (6 sigma_MLE) exact identity 1.441377674 1.441377674 pass
... Ppk = min distance to a limit / (3 sigma_MLE) exact identity 1.279249264 1.279249264 pass
lognormal data, USL only: expected ppm (true 643) known distribution 674.114107 643.4710129 pass
Box-Cox lambda scipy.stats.boxcox_normmax (MLE) -0.01098663429 -0.01098663429 pass
Box-Cox on lognormal data: expected ppm known distribution 735.9338419 643.4710129 pass
normal Ppk on the same skewed data would claim (ppm) demonstrates the need 0.0001327446213 643.4710129 pass

Attribute agreement

Check Reference engstat Reference value Status
Fleiss' kappa, 10 subjects x 14 raters x 5 categories published example: 0.210 0.2099307044 0.21 pass