Defence Update Pipeline

DefenseUpdatePipeline is the reviewed promotion gate between three experimental hardening surfaces:

• SelfImprovingGuardLoop proposes calibration or training changes from reviewed feedback
• ContinualEngine mines recent failures into an adversarial suite and trained adversary scorer
• DefenseRegistry hot-swaps the active defence only after the review and adversarial gates pass

The pipeline does not train, mine, or approve anything by itself. It checks that an already-created proposal is approved, checks that adversarial mining produced enough cases, checks the held-out score delta, then performs one atomic registry promotion with tenant-safe metadata.

from director_ai.core.defense_genome import DefenseRegistry, DefenseUpdatePipeline

registry = DefenseRegistry()
pipeline = DefenseUpdatePipeline(
    registry=registry,
    min_adversarial_cases=8,
    min_holdout_improvement=0.02,
)

report = pipeline.review_and_promote(
    proposal=approved_guard_loop_proposal,
    evolve_report=continual_engine_report,
    defense=candidate_defence,
    version=2,
    label="defence-v2",
    baseline_score=0.72,
    candidate_score=0.84,
)

Promotion metadata contains identifiers and metrics only:

• proposal id, proposal type, approval id, manifest id, rollback id
• continual suite version, mined pattern count, adversarial case count
• baseline score, candidate score, and held-out delta

Raw prompts, responses, credentials, private evidence text, and tenant payloads remain in their owning stores and are not copied into registry metadata.

Auto-Redteam Defence Loop

AutoRedteamDefenceLoop wraps the same reviewed promotion gate in a repeatable cycle. Each cycle loads fresh failure events into a bounded FailureStore, mines an adversarial suite with ContinualEngine, scores the active and candidate defences against the newly mined cases, and promotes the candidate only when detection uplift clears the configured gate.

from director_ai.core.defense_genome import (
    AutoRedteamCycleInput,
    AutoRedteamDefenceLoop,
    DefenseRegistry,
)

registry = DefenseRegistry()
registry.promote(defense=current_defence, version=1, label="defence-v1")

loop = AutoRedteamDefenceLoop(
    registry=registry,
    min_failures=16,
    min_detection_uplift=0.02,
)

report = loop.run_cycle(
    AutoRedteamCycleInput(
        failures=recent_failure_events,
        safe_corpus=reviewed_safe_prompts,
        proposal=approved_guard_loop_proposal,
        candidate_defence=candidate_defence,
        version=2,
        label="defence-v2",
        baseline_score=0.72,
        candidate_score=0.84,
    )
)

report.to_dict() is safe to archive with release evidence. It includes suite version, promoted version, detection rates, uplift, holdout delta, and a digest of mined patterns. It does not include raw prompts, raw feedback, credentials, or the defence object.

Generate the local evidence packet with:

PYTHONPATH=src python -m benchmarks.auto_redteam_defence_evidence

Full API

director_ai.core.defense_genome.update_pipeline.DefenseUpdatePipeline

DefenseUpdatePipeline(*, registry: DefenseRegistry, min_adversarial_cases: int = 1, min_holdout_improvement: float = 0.0)

Promote a candidate defence only after review and adversarial gates.

The pipeline is deliberately narrow: it never trains a model, mines failures, or mutates proposals. It joins the already-reviewed GuardLoopProposal with a ContinualEngine report and then performs one atomic registry promotion if every gate passes.

review_and_promote

review_and_promote(*, proposal: GuardLoopProposal, evolve_report: EvolveReport, defense: Defense, version: int, label: str, baseline_score: float, candidate_score: float) -> DefenseUpdateReport

Validate review, adversarial evidence, and holdout score, then promote.

director_ai.core.defense_genome.update_pipeline.DefenseUpdateReport dataclass

DefenseUpdateReport(snapshot: DefenseSnapshot, proposal_id: str, suite_version: int, adversarial_case_count: int, promoted: bool, metadata: dict[str, str] = dict())

Tenant-safe report for one reviewed defence promotion.

director_ai.core.defense_genome.redteam_loop.AutoRedteamDefenceLoop

AutoRedteamDefenceLoop(*, registry: DefenseRegistry, pipeline: DefenseUpdatePipeline | None = None, min_failures: int = 16, window_last_n: int = 512, block_threshold: float = 0.5, min_detection_uplift: float = 0.01, min_adversarial_cases: int = 1, min_holdout_improvement: float = 0.0)

Run repeated adversarial-mining cycles before defence promotion.

run

run(cycles: Sequence[AutoRedteamCycleInput]) -> tuple[AutoRedteamCycleReport, ...]

Run cycles sequentially so each promotion becomes the next baseline.

run_cycle

run_cycle(cycle: AutoRedteamCycleInput) -> AutoRedteamCycleReport

Run one reviewed mining + detection-uplift + promotion cycle.

director_ai.core.defense_genome.redteam_loop.AutoRedteamCycleInput dataclass

AutoRedteamCycleInput(failures: Sequence[FailureEvent], safe_corpus: Sequence[str], proposal: GuardLoopProposal, candidate_defence: Defense, version: int, label: str, baseline_score: float, candidate_score: float)

Inputs for one reviewed auto-redteam promotion cycle.

__post_init__

__post_init__() -> None

Reject empty corpora, a bad version/label, or out-of-range scores.

director_ai.core.defense_genome.redteam_loop.AutoRedteamCycleReport dataclass

AutoRedteamCycleReport(suite_version: int, promoted_version: int, label: str, proposal_id: str, adversarial_case_count: int, mined_pattern_count: int, baseline_detection_rate: float, candidate_detection_rate: float, detection_uplift: float, holdout_delta: float, pattern_digest: str, promoted: bool, metadata: Mapping[str, str] = dict())

Tenant-safe result for one auto-redteam cycle.

to_dict

to_dict() -> dict[str, Any]

Serialise without raw prompts, raw feedback, or defence objects.