Monitor — Validation status

The machine-readable record of how much external evidence each monitor family carries. SPO ships dozens of monitors, but only one detection niche — grid modal damping — has been checked against an independent ground truth. This registry keeps a broad monitor gallery from reading as a broad set of field-ready detectors.

Why it exists

The three tiers restate the repository's own README §Evidence status in structured form, so downstream code and the Studio can filter on validation posture without parsing prose:

• EXTERNALLY_VALIDATED ("external") — clears a matched-false-alarm operating point and a permutation significance test on independent real data. Today: the grid modal envelope-growth detector and its streaming form.
• SYNTHETIC_ONLY ("synthetic-only") — recovers an analytic or planted ground truth on synthetic data, yet is at chance on real data under the same honest test. The generic early-warning suite and the matrix-pencil estimator.
• RESEARCH ("research") — an exploratory diagnostic with no external- or synthetic-reference validation record. The conservative default: a monitor is never promoted above this tier without a citable study section.

Usage

from scpn_phase_orchestrator.monitor import (
    MonitorValidationStatus,
    monitors_by_status,
    validation_record,
    validation_summary,
)

validation_record("grid_modal_growth").status
# <MonitorValidationStatus.EXTERNALLY_VALIDATED: 'external'>

[r.monitor for r in monitors_by_status(MonitorValidationStatus.EXTERNALLY_VALIDATED)]
# ['grid_modal_growth', 'grid_modal_stream']

{s.value: n for s, n in validation_summary().items()}
# {'external': 2, 'synthetic-only': 6, 'research': 28}

A test drift-guard fails closed if a newly added public monitor module is left neither classified nor explicitly excluded, so the honest posture cannot silently rot as the monitor suite grows.

validation_status

Declare the honest external-validation posture of every monitor family.

SPO ships dozens of dynamical monitors, but only one detection niche has been checked against an independent ground truth: grid modal damping. To keep a broad monitor gallery from reading as a broad set of field-ready detectors, this module records, for each public monitor module, a machine-readable :class:MonitorValidationStatus and a citable basis for that status.

The three tiers restate the repository's own README §Evidence status verbatim in structured form, so the registry cannot quietly overclaim:

EXTERNALLY_VALIDATED The detector clears a matched-false-alarm operating point plus a permutation significance test on an independent, real-data corpus — the only tier that carries field evidence. Today that is the grid modal envelope-growth detector and its causal streaming form (study §3.5–3.6: 36/90 real PSML transitions led at permutation p = 0.0001, held-out 24/45 at p = 0.0002).

SYNTHETIC_ONLY The detector recovers an analytic or planted ground truth on synthetic data (the eigenvalue-regime map), yet is demonstrated at chance on real data under the same honest test. The generic early-warning suite and the matrix-pencil modal estimator sit here.

RESEARCH An exploratory diagnostic with no external- or synthetic-reference validation record. This is the conservative default: a monitor is never promoted above RESEARCH without a citable study section.

The registry is the single source of truth. It is exported from :mod:scpn_phase_orchestrator.monitor, surfaced in the API reference and the Monitor validation status guide, and guarded by a test that fails closed if a newly added monitor module is left unclassified (see :data:NON_MONITOR_MODULES).

Attributes

MONITOR_VALIDATION module-attribute

MONITOR_VALIDATION: Mapping[
    str, MonitorValidationRecord
] = _build_registry(_RECORDS)

Read-only registry of every classified monitor's validation posture.

Classes

MonitorValidationStatus

Bases: Enum

The external-validation tier a monitor family is honestly entitled to.

The three members are ordered from strongest to weakest evidence. Their string values ("external", "synthetic-only", "research") are the stable, machine-readable tokens used across the API, the documentation, and any Studio surface, so a downstream consumer can filter on them without parsing prose.

MonitorValidationRecord dataclass

MonitorValidationRecord(
    monitor: str,
    display_name: str,
    status: MonitorValidationStatus,
    basis: str,
    evidence: str,
)

The validation posture of a single monitor family.

Parameters

monitor: The monitor family name — the stem of a top-level .py module or the name of a subpackage under scpn_phase_orchestrator/monitor/ — used as the registry key. display_name: A short human-readable label for documentation and Studio surfaces. status: The :class:MonitorValidationStatus the monitor is entitled to. basis: A one-line, citable justification for status — a study section, a reference, or an explicit statement that no validation record exists. evidence: A pointer to the evidence backing basis (a study document, the README evidence-status section, or an empty string when the basis is simply the absence of a validation record).

Raises

ValueError If monitor, display_name or basis is empty or blank, so a record can never be silently underspecified. TypeError If status is not a :class:MonitorValidationStatus.

Methods:

__post_init__

__post_init__() -> None

Validate the record fields, failing closed on an empty specification.

Source code in src/scpn_phase_orchestrator/monitor/validation_status.py

def __post_init__(self) -> None:
    """Validate the record fields, failing closed on an empty specification."""
    if not isinstance(self.status, MonitorValidationStatus):
        msg = (
            "status must be a MonitorValidationStatus, "
            f"got {type(self.status).__name__}"
        )
        raise TypeError(msg)
    for field_name in ("monitor", "display_name", "basis"):
        value = getattr(self, field_name)
        if not value or not value.strip():
            msg = f"{field_name} must be a non-empty string"
            raise ValueError(msg)

Functions:

validation_record

validation_record(monitor: str) -> MonitorValidationRecord

Return the validation record for a monitor, failing closed if unknown.

Parameters

monitor: The monitor module name (for example "grid_modal_growth").

Returns

MonitorValidationRecord The record declaring the monitor's validation posture.

Raises

KeyError If monitor is not a classified monitor family, listing the known monitors so the caller cannot fall through to a silent default.

Source code in src/scpn_phase_orchestrator/monitor/validation_status.py

def validation_record(monitor: str) -> MonitorValidationRecord:
    """Return the validation record for a monitor, failing closed if unknown.

    Parameters
    ----------
    monitor:
        The monitor module name (for example ``"grid_modal_growth"``).

    Returns
    -------
    MonitorValidationRecord
        The record declaring the monitor's validation posture.

    Raises
    ------
    KeyError
        If ``monitor`` is not a classified monitor family, listing the known
        monitors so the caller cannot fall through to a silent default.
    """
    try:
        return MONITOR_VALIDATION[monitor]
    except KeyError:
        known = ", ".join(sorted(MONITOR_VALIDATION))
        msg = f"unknown monitor {monitor!r}; known monitors: {known}"
        raise KeyError(msg) from None

monitors_by_status

monitors_by_status(
    status: MonitorValidationStatus,
) -> tuple[MonitorValidationRecord, ...]

Return every monitor record at a validation tier, ordered by name.

Parameters

status: The :class:MonitorValidationStatus to filter on.

Returns

tuple[MonitorValidationRecord, ...] The matching records, sorted by monitor name for determinism (possibly empty).

Raises

TypeError If status is not a :class:MonitorValidationStatus.

Source code in src/scpn_phase_orchestrator/monitor/validation_status.py

def monitors_by_status(
    status: MonitorValidationStatus,
) -> tuple[MonitorValidationRecord, ...]:
    """Return every monitor record at a validation tier, ordered by name.

    Parameters
    ----------
    status:
        The :class:`MonitorValidationStatus` to filter on.

    Returns
    -------
    tuple[MonitorValidationRecord, ...]
        The matching records, sorted by monitor name for determinism (possibly
        empty).

    Raises
    ------
    TypeError
        If ``status`` is not a :class:`MonitorValidationStatus`.
    """
    if not isinstance(status, MonitorValidationStatus):
        msg = f"status must be a MonitorValidationStatus, got {type(status).__name__}"
        raise TypeError(msg)
    return tuple(
        sorted(
            (
                record
                for record in MONITOR_VALIDATION.values()
                if record.status is status
            ),
            key=lambda record: record.monitor,
        )
    )

validation_summary

validation_summary() -> Mapping[
    MonitorValidationStatus, int
]

Return the count of classified monitors at each validation tier.

Returns

Mapping[MonitorValidationStatus, int] A read-only mapping from every :class:MonitorValidationStatus member to the number of monitors at that tier, including tiers with a zero count so the shape is stable.

Source code in src/scpn_phase_orchestrator/monitor/validation_status.py

def validation_summary() -> Mapping[MonitorValidationStatus, int]:
    """Return the count of classified monitors at each validation tier.

    Returns
    -------
    Mapping[MonitorValidationStatus, int]
        A read-only mapping from every :class:`MonitorValidationStatus` member
        to the number of monitors at that tier, including tiers with a zero
        count so the shape is stable.
    """
    counts: dict[MonitorValidationStatus, int] = dict.fromkeys(
        MonitorValidationStatus, 0
    )
    for record in MONITOR_VALIDATION.values():
        counts[record.status] += 1
    return MappingProxyType(counts)