Replay-Only Autotune Learners

Why this surface is replay-only

Learner proposal contracts are intentionally placed behind non-actuating gates. The practical safety objective is to let users compare control candidates with traceable evidence before any actuator can ever be touched.

That separation is why these interfaces are suitable for experimentation and policy research without introducing a production autonomy path by default.

The learner surface provides PPO-like, SAC-like, and hybrid-physics proposal generators behind the existing non-actuating replay gates. These are learner-shaped proposal interfaces, not claims of trained production policies.

All learner outputs are LearnerPolicyProposal records:

• actuation_permitted is always false,
• replay search evidence is serialised for audit,
• unsafe proposals are rejected by the same reward/proposal gates used by the deterministic replay search,
• optional safe-RL gates can require Lyapunov exponent, STL robustness, and bounded safety-cost evidence before a replay proposal is accepted,
• future PPO/SAC implementations must plug into the same contract before they can be reviewed.

Example:

from scpn_phase_orchestrator.autotune import (
    PolicyProposalConfig,
    SafetyConstraintConfig,
)
from scpn_phase_orchestrator.autotune.learners import generate_ppo_like_proposal

proposal = generate_ppo_like_proposal(
    seed,
    evaluator,
    seed_value=7,
    proposal_config=PolicyProposalConfig(
        safety_constraints=SafetyConstraintConfig(
            max_lyapunov_exponent=0.0,
            min_stl_robustness=0.0,
            max_safety_cost=0.05,
            require_lyapunov=True,
            require_stl=True,
            require_safety_cost=True,
        ),
    ),
)
audit = proposal.to_audit_record()
assert audit["actuation_permitted"] is False

Use this surface to compare replay candidates and export review records. A candidate whose reward is high but whose Lyapunov/STL evidence is missing or violating remains rejected. Do not feed learner proposals directly to hardware or live actuation.

Operational posture

In production pipelines, the learner interface is most useful during policy benchmarking and review simulation runs where reward, stability, and cost can be compared under controlled replay datasets before any runtime policy override.

When this posture is respected, teams gain faster proposal discovery while keeping live control logic bound to explicit safety gates.

Practical operating pattern

The intended path for this surface is an explicit sequence:

1. replay-based scoring of learner candidates,
2. proposal evidence serialization,
3. safety gate re-evaluation,
4. manual/qualified operator approval.

This ordering prevents reward-chasing proposals from bypassing explicit safety and audit expectations.

Proposal governance

All learner outputs are treated as review input, not control input. The same actuation_permitted=false boundary used by other replay-only lanes applies here, so every high-performing proposal still needs explicit promotion through the production control gate before any live actuation.

When teams compare PPO-like and SAC-like proposals, record:

• reward trend,
• Lyapunov/stability evidence,
• STL robustness margin,
• and audit-replay hash lineage.

That evidence set is the minimum for an auditable tuning review.

Governance boundary and rollout flow

This learner surface is kept separate from live control by design. The boundary is an operational control measure to prevent reward-optimised behavior from crossing into actuation without explicit checks.

Rollout sequence:

1. Capture proposal payload and gate signals with fixed seeds and deterministic replay settings.
2. Store safety evidence and replay artefacts in the same evidence bundle.
3. Route candidates through explicit operator or policy owner approval.
4. Promote only through the existing production actuation gate.