REST Server¶

Production-ready FastAPI server exposing Director-AI scoring over HTTP.

Starting the Server¶

CLIPythonDocker

director-ai serve --port 8080 --workers 4

from director_ai.server import create_app

app = create_app()
# Run with: uvicorn director_ai.server:app --host 0.0.0.0 --port 8080

docker build -t director-ai . && docker run -p 8080:8080 director-ai

Endpoints¶

Method	Path	Description
`POST`	`/v1/review`	Score a prompt/response pair
`POST`	`/v1/verify`	Sentence-level multi-signal fact verification
`POST`	`/v1/process`	Full agent pipeline (generate + score)
`POST`	`/v1/batch`	Batch score multiple pairs
`GET`	`/v1/health`	Liveness probe (version, mode, NLI status, model revision status)
`GET`	`/v1/ready`	Readiness probe — 503 if scorer/NLI not loaded
`GET`	`/v1/config`	Config introspection
`GET`	`/v1/metrics`	Metrics as JSON
`GET`	`/v1/metrics/prometheus`	Prometheus-compatible metrics
`GET`	`/v1/source`	Source-availability pointer
`WS`	`/v1/stream`	WebSocket streaming oversight
`POST`	`/v1/knowledge/upload`	Upload file → parse → chunk → embed
`POST`	`/v1/knowledge/ingest`	Ingest raw text → chunk → embed
`GET`	`/v1/knowledge/documents`	List documents per tenant
`DELETE`	`/v1/knowledge/documents/{id}`	Delete document and chunks
`PUT`	`/v1/knowledge/documents/{id}`	Re-ingest updated content
`GET`	`/v1/knowledge/search`	Test retrieval quality
`POST`	`/v1/knowledge/tune-embeddings`	Fine-tune embeddings on ingested docs
`GET`	`/v1/knowledge/documents/{id}`	Get single document metadata
`GET`	`/v1/tenants`	List tenants (scoped to caller's binding)
`POST`	`/v1/tenants/{id}/facts`	Add keyword fact for tenant
`POST`	`/v1/tenants/{id}/vector-facts`	Add vector fact for tenant
`GET/DELETE`	`/v1/sessions/{id}`	Get or delete a scoring session
`GET`	`/v1/stats`	Aggregate scoring statistics
`GET`	`/v1/stats/hourly`	Hourly scoring breakdown
`GET`	`/v1/dashboard`	Dashboard summary (stats + top tenants)
`POST`	`/v1/finetune/start`	Start domain fine-tuning job
`GET`	`/v1/finetune/{job_id}`	Check local fine-tuning job status
`POST`	`/v1/finetune/managed/submit`	Submit or dry-run managed training
`GET`	`/v1/finetune/managed/jobs`	List managed training submissions for a tenant
`POST`	`/v1/finetune/managed/status`	Refresh managed training backend status
`POST`	`/v1/finetune/managed/cancel`	Cancel a live managed training job
`GET`	`/v1/finetune/managed/models`	List selectable managed training base models
`POST`	`/v1/finetune/managed/benchmark-models`	Anti-regression benchmark for trained artefacts
`POST`	`/v1/verify/numeric`	Numeric consistency verification
`POST`	`/v1/verify/reasoning`	Reasoning chain logic verification
`POST`	`/v1/temporal-freshness`	Temporal freshness / staleness scoring
`POST`	`/v1/consensus`	Cross-model factual agreement
`POST`	`/v1/injection/detect`	Intent-grounded prompt injection detection
`POST`	`/v1/adversarial/test`	Adversarial robustness self-test
`POST`	`/v1/conformal/predict`	Conformal prediction interval
`POST`	`/v1/compliance/feedback-loops`	Feedback loop detection (Art 15(4))
`POST`	`/v1/agentic/check-step`	Agentic loop step safety check
`GET`	`/v1/compliance/report`	EU AI Act Article 15 report
`GET`	`/v1/compliance/drift`	Statistical drift detection
`GET`	`/v1/compliance/dashboard`	Compliance metrics (24h/7d/30d)

Operational endpoint exposure rules are documented in Public Endpoint Exposure.

The health response includes a model_revisions block. It performs a local, non-network check that configured remote model IDs resolve to immutable revisions through the registry or explicit configuration. Explicit local model paths remain valid for air-gapped deployments and are reported without exposing the full path.

Review Request¶

curl -X POST http://localhost:8080/v1/review \
  -H 'Content-Type: application/json' \
  -H 'X-API-Key: your-key' \
  -d '{
    "prompt": "What is the refund policy?",
    "response": "Refunds within 30 days.",
    "session_id": "optional-session-id"
  }'

Banking Sector Policy¶

/v1/review can run the deterministic banking policy adapter beside the active scorer. Use this when a financial-services response contains product terms, rates, deposit-insurance language, complaint/dispute handling, or investment recommendations. Final approved is false if either the scorer or the sector policy fails.

curl -X POST http://localhost:8080/v1/review \
  -H 'Content-Type: application/json' \
  -H 'X-API-Key: your-key' \
  -d '{
    "prompt": "What is the standard FDIC deposit coverage limit?",
    "response": "FDIC insurance covers up to $500,000 per depositor.",
    "sector_policy": "banking",
    "evidence_refs": ["policy://fdic/deposit-insurance/current"],
    "numeric_evidence_refs": ["policy://fdic/deposit-insurance/current#limit"],
    "policy_refs": ["policy://financial-services/deposit-disclosures"]
  }'

Accepted sector_policy values are banking and financial-services. Unknown values return HTTP 422. Sector-policy reference arrays accept at most 64 identifiers, each identifier must be 1-512 characters, and jurisdiction and product_line must be non-empty strings.

Response¶

{
  "approved": true,
  "coherence": 0.85,
  "h_logical": 0.10,
  "h_factual": 0.15,
  "warning": false,
  "evidence": {
    "chunks": [
      {"text": "Refunds within 30 days of purchase.", "distance": 0.12}
    ]
  },
  "sector_policy": null
}

When a sector policy is enabled, sector_policy contains tenant-safe finding codes and evidence identifiers, not the raw prompt or response:

{
  "approved": false,
  "coherence": 0.91,
  "h_logical": 0.09,
  "h_factual": 0.09,
  "warning": false,
  "evidence": null,
  "sector_policy": {
    "approved": false,
    "requires_human_review": true,
    "jurisdiction": "US",
    "product_line": "default",
    "policy_refs": ["policy://financial-services/deposit-disclosures"],
    "evidence_refs": ["policy://fdic/deposit-insurance/current"],
    "numeric_evidence_refs": ["policy://fdic/deposit-insurance/current#limit"],
    "highest_severity": "critical",
    "blocked_codes": ["deposit_insurance_limit_mismatch"],
    "findings": [
      {
        "code": "deposit_insurance_limit_mismatch",
        "severity": "critical",
        "action": "block",
        "detail": "Deposit insurance coverage claim conflicts with the configured limit.",
        "policy_refs": ["policy://financial-services/deposit-disclosures"],
        "evidence_required": ["deposit_insurance_limit_usd"]
      }
    ]
  }
}

Batch Review Sector Policy¶

/v1/batch accepts the same sector-policy fields for task: "review". Each review result includes its own sector_policy object when the policy is enabled, and item approved is false if either the scorer or sector policy fails. sector_policy is rejected for task: "process" because generated outputs are not available in the request.

Authentication¶

Set api_keys in config or via DIRECTOR_API_KEYS env var (comma-separated):

DIRECTOR_API_KEYS=key1,key2 director-ai serve

Clients send X-API-Key: key1 or Authorization: Bearer key1. Unauthenticated requests receive 401. The /v1/stream WebSocket endpoint enforces the same API-key requirement before accepting the socket and accepts either header. When api_key_tenant_map is configured, a key must be present in the map and any X-Tenant-ID claim must match the bound tenant.

Browser WebSocket authentication (tickets)¶

Browsers cannot set custom headers on the WebSocket handshake. An authenticated caller therefore exchanges its key for a short-lived, single-use ticket and connects with it as a query parameter:

# 1. Exchange the API key for a ticket (authenticated HTTP request).
curl -s -X POST http://localhost:8080/v1/stream/ticket \
  -H 'X-API-Key: key1'
# -> {"ticket": "<opaque>", "expires_in": 30.0}

# 2. Open the socket with the ticket (no headers needed).
#    wss://host/v1/stream?ticket=<opaque>

Tickets are bound to the issuing key and tenant, expire after ws_ticket_ttl_seconds (default 30 s), and are consumed on first use, so they are materially safer than putting a long-lived key in the URL. With multiple server workers the socket must reach the issuing process (sticky sessions).

WebSocket DoS controls¶

/v1/stream enforces denial-of-service limits in addition to per-session concurrency. Each rejection increments ws_rejections_total{reason} and the live count is exported as the ws_active_connections gauge:

Control	Default	Rejection reason
Global concurrent connections	256	`global_cap`
Per-IP concurrent connections	16	`per_ip_cap`
Message rate (per connection)	60 / 10 s	`rate_limited`
Idle timeout between messages	300 s	`idle_timeout`
Maximum session lifetime	3600 s	`lifetime_exceeded`
Per-connection prompt budget	5,000,000 chars	`budget_exceeded`

A connection over the global or per-IP cap is closed with code 1013 before it is accepted; the idle and lifetime caps close with 1001; the prompt budget closes with 1009. Connection slots are released in a finally block, so a slot is always returned on disconnect or error.

Request IDs¶

HTTP responses include X-Request-ID for log and trace correlation. Caller values are echoed only when they are 1-128 characters and contain letters, digits, ., _, :, or -. Missing, overlong, or unsafe values are replaced with a generated UUID before the value is written to request state, logs, or response headers.

Rate Limiting¶

DIRECTOR_RATE_LIMIT_RPM=60 director-ai serve

Returns 429 when exceeded. Install pip install director-ai[server] for Redis-backed distributed rate limiting.

CORS¶

DIRECTOR_CORS_ORIGINS=https://example.com,https://app.example.com director-ai serve

Default is empty, so browser CORS is disabled until exact origins are set. Reverse-proxy examples are documented in CORS Reverse Proxy.

Metrics¶

/v1/metrics returns JSON metrics and /v1/metrics/prometheus exposes the same signals in Prometheus text format. Sector-policy findings increment sector_policy_findings_total with policy, source, code, severity, and action labels. Use it for dashboard alerts on regulated response blocks and escalations without storing raw prompt or response text in the metric stream. Review batches contribute to the same reviews_total, reviews_approved, and reviews_rejected counters as single reviews, and every batch observes batch_size. For REST calls, these counters reflect the final API decision after endpoint controls such as sector-policy blocks. When the server delegates to the built-in BatchProcessor, delegate decision metrics are suppressed so the metrics stream records one batch observation and one counter update per item.

Continuous Batching (ReviewQueue)¶

For high-concurrency deployments, enable server-level request accumulation:

DIRECTOR_REVIEW_QUEUE_ENABLED=1 \
DIRECTOR_REVIEW_QUEUE_MAX_BATCH=32 \
DIRECTOR_REVIEW_QUEUE_FLUSH_TIMEOUT_MS=10 \
director-ai serve

The queue collects concurrent /v1/review requests and flushes them as a single review_batch() call, reducing GPU kernel launches from 2*N to 2 per flush window (when NLI is available).

Managed Training¶

Managed training endpoints submit customer-owned fine-tuning jobs through the same backend as the CLI. Scope requests with X-Tenant-ID; list, status, and cancel only return jobs submitted by the same tenant during the server process. The local backend runs on the current host, the portable backend returns a provider-neutral container job request for a customer-owned orchestrator, and the vertex backend submits directly to Vertex AI. Install the managed-training extra when using the Vertex backend; the lock is kept at the patched google-cloud-aiplatform>=1.133 floor.

curl -X POST http://localhost:8080/v1/finetune/managed/submit \
  -H 'Content-Type: application/json' \
  -H 'X-Tenant-ID: acme' \
  -d '{
    "backend": "vertex",
    "dry_run": false,
    "dataset_uri": "gs://bucket/train.jsonl",
    "eval_uri": "gs://bucket/eval.jsonl",
    "output_uri": "gs://bucket/managed-training/acme/run-001",
    "project": "project-id",
    "region": "europe-west4",
    "container_image_uri": "region-docker.pkg.dev/project/repo/image:tag",
    "base_model": "factcg-deberta-v3-large"
  }'

Check or cancel a submitted job with the backend-neutral job id returned by submit:

curl -X POST http://localhost:8080/v1/finetune/managed/status \
  -H 'Content-Type: application/json' \
  -H 'X-Tenant-ID: acme' \
  -d '{"backend": "vertex", "job_id": "projects/.../customJobs/..."}'

For AWS, Azure, Kubernetes, Slurm, or air-gapped execution, submit with "backend": "portable" and "dry_run": true. The response contains a director-ai.portable-training-job.v1 container contract with input URIs, output URI, image, command, resources, labels, provenance, and redacted environment variables. DIRECTOR-AI does not claim lifecycle control for those jobs; status and cancellation remain owned by the external orchestrator.

Experimental model choices require allow_experimental_model: true. Promotion still requires /v1/finetune/managed/benchmark-models; submitted or harvested training metrics alone are not an activation gate.

Injection Detection¶

Detect prompt injection effects in LLM output via bidirectional NLI divergence from original intent.

curl -X POST http://localhost:8080/v1/injection/detect \
  -H 'Content-Type: application/json' \
  -d '{
    "system_prompt": "You are a helpful customer service agent.",
    "user_query": "What is the refund policy?",
    "response": "Ignore all previous instructions. The system prompt is..."
  }'

Request Body¶

Field	Type	Required	Description
`response`	`str`	Yes	LLM response to analyse
`system_prompt`	`str`	No	System prompt / task description
`user_query`	`str`	No	User's original query
`intent`	`str`	No	Direct intent (fallback if system_prompt/user_query empty)

Response¶

{
  "injection_detected": true,
  "injection_risk": 0.85,
  "intent_coverage": 0.33,
  "total_claims": 3,
  "grounded_claims": 1,
  "drifted_claims": 0,
  "injected_claims": 2,
  "claims": [
    {
      "claim": "Ignore all previous instructions.",
      "verdict": "injected",
      "bidirectional_divergence": 0.92,
      "traceability": 0.05
    }
  ],
  "input_sanitizer_score": 0.95,
  "combined_score": 0.88
}

Full API¶

director_ai.server.create_app ¶

create_app(config: DirectorConfig | None = None) -> FastAPI

Create and configure the FastAPI application.