Validation Framework¶
SCPN-Fusion-Core is validated against published device datasets from real tokamaks. The validation framework provides regression-grade checks that detect numerical regressions and ensure physically reasonable behaviour across releases.
Full-Fidelity Campaign Gate¶
The production-parity gate is intentionally fail-closed:
python validation/full_fidelity_end_to_end_campaign.py
The current gate distinguishes local solver contracts from accepted external
reference parity. Local nonlinear GK, electromagnetic, runaway, impurity,
FreeGS, and decomposition contracts can be ready while the overall campaign is
still not_full_fidelity. Accepted parity requires same-case reference
outputs, redistributable artefacts, checksums, quantitative thresholds, and
native solver-output comparisons.
The primary public reports are:
validation/reports/full_fidelity_end_to_end_campaign.mdvalidation/reports/full_fidelity_acceptance_benchmark.mdvalidation/reports/full_fidelity_reference_artifact_conversion.mdvalidation/reports/production_decomposition_contract.mdvalidation/reports/gk_electromagnetic_fidelity.md
Validation Datasets¶
Dataset |
Source |
Contents |
|---|---|---|
SPARC GEQDSK |
CFS SPARCPublic (MIT) |
8 equilibrium files (\(B = 12.2\,\text{T}\), \(I_p\) up to \(8.7\,\text{MA}\)) |
ITPA H-mode |
Verdoolaege et al., NF 61 (2021) |
20-row confinement dataset from 10 tokamaks (JET, DIII-D, C-Mod, ASDEX-U, …) |
ITER baseline |
ITER Physics Basis |
15 MA \(Q = 10\) scenario parameters |
SPARC V2C |
Creely et al., JPP 2020 |
Compact high-field scenario parameters |
DIII-D |
Luxon, NF 42 (2002) |
Medium-size tokamak reference parameters |
JET |
Pamela et al. (2007) |
Largest tokamak, DT fusion reference |
IPB98(y,2) Confinement Validation¶
The regression suite validates the confinement time prediction against the IPB98(y,2) scaling law (ITER Physics Basis, Nuclear Fusion 39, 1999):
Validated accuracy against the ITPA H-mode database (20-shot subset):
Metric |
Value |
Source |
|---|---|---|
\(\tau_E\) RMSE |
|
|
\(\tau_E\) relative RMSE |
|
|
\(\tau_E\) mean abs. relative error |
|
|
Aux MW->keV/s source max rel. error |
|
|
2-sigma coverage |
|
|
Note
The 13.5% value reported for the neural transport MLP is a
surrogate-fit metric and is not the same as full physics-transport
validation on the 20-shot ITPA lane.
SPARC Equilibrium Topology Validation¶
The 8 SPARC GEQDSK files are validated for equilibrium topology:
Magnetic axis position: \(R_\text{axis}\) and \(Z_\text{axis}\) within expected ranges for the SPARC geometry
Safety factor monotonicity: \(q(\rho)\) is monotonically increasing from axis to edge (no reversed shear in these scenarios)
GS operator sign: the discrete Grad-Shafranov operator applied to the stored \(\psi\) field has the correct sign pattern (current density is positive inside the plasma)
Point-Wise Psi RMSE¶
The psi_pointwise_rmse module performs point-wise \(\psi(R,Z)\)
reconstruction error analysis on the bundled EFIT/GEQDSK equilibria:
Finite-difference GS operator \(\Delta^*\psi\)
Relative \(L^2\) and max-abs GS residuals
Manufactured-solution Red-Black SOR verification
Normalised \(\psi\) RMSE on the plasma region
Strict 18-file aggregate gate with a 10-file minimum and 5% threshold
The current bundled gate is intentionally reported as failing rather than hidden: all 18 rows are classified as profile-source mismatches, while the operator-source manufactured solve remains near machine precision. This separates solver consistency from unresolved EFIT profile/source convention alignment.
ITER Reference Scenarios¶
Three reference scenarios are used for regression testing:
ITER 15 MA baseline (\(Q = 10\) target):
\(I_p = 15.0\,\text{MA}\), \(B_T = 5.3\,\text{T}\)
\(R = 6.2\,\text{m}\), \(a = 2.0\,\text{m}\), \(\kappa = 1.7\)
\(\tau_E = 3.7\,\text{s}\), \(P_\text{fus} = 500\,\text{MW}\)
SPARC V2C (compact high-field):
\(I_p = 8.7\,\text{MA}\), \(B_T = 12.2\,\text{T}\)
\(R = 1.85\,\text{m}\), \(a = 0.57\,\text{m}\), \(\kappa = 1.97\)
\(\tau_E = 0.77\,\text{s}\), \(P_\text{fus} = 140\,\text{MW}\)
DIII-D (L-mode sanity check):
\(I_p = 1.0\,\text{MA}\), \(B_T = 2.1\,\text{T}\)
\(R = 1.67\,\text{m}\), \(a = 0.67\,\text{m}\)
Running the Validation Suite¶
Generate an RMSE dashboard report:
python validation/rmse_dashboard.py
This writes JSON and Markdown reports to validation/reports/
containing confinement time accuracy, beta normalised surrogate RMSE,
SPARC axis position errors, and point-wise \(\psi\) RMSE.
Benchmark transport-source power-balance reconstruction:
python validation/benchmark_transport_power_balance.py
Validate against SPARC GEQDSK files:
python validation/validate_against_sparc.py
Run the full regression test suite:
pytest tests/test_validation_regression.py -v
ITER configuration validation:
python validation/validate_iter.py
Property-Based Testing¶
The test suite includes property-based tests using Hypothesis (Python) and proptest (Rust), covering:
Numerical invariants – symmetry of solution operators, positive definiteness of energy functionals
Topology preservation – equilibrium topology is preserved under perturbations
Solver convergence – residuals monotonically decrease for well-conditioned problems
Physical bounds – temperatures, densities, and pressures remain non-negative
Code Health¶
The codebase has undergone 8 systematic hardening waves (248 tasks)
that replaced silent clamping, unwrap() calls, and implicit
coercion with explicit FusionResult<T> error propagation throughout
the Rust workspace.
Wave |
Scope |
Tasks |
Highlights |
|---|---|---|---|
S2 |
Scaffold integrity |
8 |
Module wiring, import consistency |
S3 |
CI pipeline |
6 |
|
S4 |
Baseline coverage |
4 |
Property-based tests (Hypothesis + proptest) |
H5 |
SCPN compiler/controller |
37 |
Deterministic replay, fault injection, contracts |
H6 |
Digital twin + RL |
9 |
Chaos monkey, bit-flip resilience |
H7 |
Control + diagnostics |
90 |
Scoped RNG isolation, sensor guards, MPC validation |
H8 |
All 10 Rust crates |
94 |
Every |