Skip to content

Production Checklist

Before deploying Director-AI to production, verify each item:

Scaffold

  • Generate the authenticated scaffold with director-ai quickstart --profile production
  • Validate the generated scaffold contract: director-ai production-check --path director_guard
  • Fill .env with DIRECTOR_API_KEY_TENANT_MAP, DIRECTOR_PROXY_API_KEYS, DIRECTOR_LLM_API_URL, DIRECTOR_UPSTREAM_URL, DIRECTOR_KB_HMAC_KEYS, and exact DIRECTOR_CORS_ORIGINS
  • Validate secret completeness before startup: director-ai production-check --path director_guard --require-secrets
  • Write the Prometheus scrape key to secrets/director-api-key before enabling the monitoring profile

Scoring

  • Set use_nli=True with a production NLI model (FactCG-DeBERTa-v3-Large recommended)
  • Configure ground_truth_store with your domain knowledge base
  • Set appropriate threshold per domain — see Domain Recommendations
  • Enable cache_size > 0 for repeated queries (reduces NLI inference cost)

Observability

  • Set up OpenTelemetry with setup_otel() for trace collection
  • Enable metrics_enabled=True for Prometheus-compatible metrics
  • Keep metrics_require_auth=True and scrape with Authorization: Bearer <api-key>
  • Configure log_json=True for structured log aggregation

Server

  • Use director-ai serve --workers N for multi-worker deployment (N = CPU cores)
  • Configure rate limiting: rate_limit_rpm > 0 for public endpoints
  • Set api_keys for authentication on exposed endpoints
  • Prefer api_key_tenant_map over unbound API keys for multi-tenant deployments
  • Set non-root user in Docker (already done in v2.0.0 Dockerfile)

Benchmarking

  • Run director-ai bench to establish baseline metrics before deployment
  • Run director-ai bench --dataset e2e to verify catch rate on your domain data
  • Run PYTHONPATH=src python -m benchmarks.auto_redteam_defence_evidence and archive the JSON packet with the release evidence
  • Run PYTHONPATH=src python -m benchmarks.formal_symbolic_evidence and archive the JSON packet with the release evidence
  • Attach formal-symbolic release-gate evidence with local packet, external Lean proof, actual Z3 release packet, domain contracts, code-contract packet, verification flags, operator sign-off, and evidence hash
  • Run PYTHONPATH=src python -m benchmarks.edge_mobile_evidence and archive the JSON packet with the release evidence
  • Run PYTHONPATH=src python tools/check_wasm_release_package.py after wasm-pack build when publishing the edge runtime
  • Run PYTHONPATH=src python tools/run_wasm_browser_worker_smoke.py for browser/Web Worker deployments
  • For browser, Worker, mobile, or embedded deployments, require ready_for_release=true; local-trial evidence alone is not enough for a customer release.
  • Run PYTHONPATH=src python -m benchmarks.sustained_load_evidence and archive the JSON packet with the release evidence
  • Run PYTHONPATH=src python tools/check_local_release_evidence.py --root . --mode local after local packets are generated
  • Run PYTHONPATH=src python tools/check_local_release_evidence.py --root . --mode release before customer release and resolve every release blocker
  • Use PYTHONPATH=src python tools/check_local_release_evidence.py --root . --mode release --format json when CI or release dashboards need machine-readable gate status
  • Review director-ai config --profile <domain> settings match your requirements

Security

  • Audit dependencies: pip-audit --strict
  • Run SAST: bandit -r src/director_ai/
  • Review CORS origins — default is empty; public browser deployments need exact https:// origins
  • Ensure caller credentials are not logged or exposed in responses

Compliance Evidence

  • Generate Article 15 documentation from the deployment audit database when the deployment is regulated or high-risk
  • Generate the SOC 2 / ISO 27001 readiness report and attach dated evidence references for every warning or failing control
  • For healthcare or ePHI-adjacent deployments, generate the HIPAA documentation packet and attach operator-owned evidence for risk analysis, business associate review, access controls, audit review, incident response, and contingency planning
  • Confirm no compliance packet contains raw PHI, raw prompt/response text, credentials, or raw security evidence

Recovery

  • Create a git bundle backup before release or incident-response changes.
  • Verify the backup checksum file.
  • Restore-test the bundle with tools/verify_repository_backup.py.
  • Store the JSON verification report with the release or incident evidence.