# AEGIS — AI Safety Layer

- **Document ID:** ARCH-16
- **Phase:** B.3 — AI, Plugins, Recovery
- **Status:** Draft for review (post three-reviewer discipline)
- **Version:** 0.1
- **Date:** 2026-07-10
- **Owner:** Chief Security Architect
- **Depends on:** ARCH-08, ARCH-09, ARCH-10, ARCH-11, ARCH-12, ARCH-15
- **Consumed by:** ARCH-17, ARCH-18

---

## 1. Purpose

Specify the AI Safety Layer: a separately-identified sub-engine that validates every AI output before it re-enters the platform. It sits between the AI Engine's DraftRecommendation and the Multi-Stage Decision Pipeline's downstream stages. Implements the eight constitutional AI Safety validators (constraint #9), works alongside Human Authority (mandate #1) and Explainability by Construction (mandate #6), and applies the Simplicity mandate (#24) by keeping each validator small, deterministic, and independently replaceable.

## 2. Position and Identity

- **Position in Pipeline (ARCH-10 F-4).** After AI Engine's *DraftRecommendation* (stage 6), before Multi-Engine Adjudication (stage 8). Safety Layer's output is the *ValidatedRecommendation*.
- **Identity.** The Safety Layer runs as a distinct process (or process group) with its own SVID. This is the operational difference from "sub-module of AI Engine" — the Safety Layer is architecturally its own gate, and only it holds `Publish-Validated-Recommendation`.
- **Consequence.** Compromising the AI Engine does not compromise the Safety Layer, and vice versa. Two independent identities must be defeated to forge a validated recommendation.

## 3. Simplicity Principle Applied

Each validator is a small, deterministic function with:
- A signed input contract (DraftRecommendation, its citations, provenance).
- A signed output (Verdict + reasons + reference IDs).
- No shared state with other validators except via signed inputs.
- No LLM.

Validators are **composable** but not **entangled**. A failing validator produces a specific failure code and does not silently affect other validators. This is the operational form of Security through Clarity.

## 4. The Eight Validators

### 4.1 V-1 Policy Validation
- **Input.** DraftRecommendation + agent identity + Policy corpus reference.
- **Function.** Re-evaluates whether the recommendation's proposed action, action-parameters, and evidence usage are permitted at the *classification* of the case, for the *actor* who would consume it, under the current Policy corpus.
- **Note.** The Policy Engine independently re-answers to prevent Safety Layer local caching from silently drifting. Divergence between Safety's cached expectation and Policy's live answer = fail.
- **Output.** `PolicyVerdict {allow|deny, rule_refs, reason}`.

### 4.2 V-2 Evidence Validation
- **Input.** DraftRecommendation citations.
- **Function.** For each `EvidenceRef`, resolve to Evidence Engine (E-15). Verify: (a) evidence exists, (b) content hash matches, (c) accessible under caller's classification ceiling, (d) not tombstoned, (e) transformation history intact.
- **Output.** `EvidenceVerdict {resolved | unresolved | invalid, per-ref detail}`. Any unresolved citation = fail.

### 4.3 V-3 Confidence Evaluation
- **Input.** DraftRecommendation confidence structure.
- **Function.** Recompute confidence using the signed formula from the Risk Engine's catalog (ARCH-11 §8) — Safety does not trust the AI's self-report of the aggregate. Adjusts by evidence confidence, source trust, model-provider trust, and citation consistency.
- **Output.** `ConfidenceVerdict {aggregate, components, safety_adjusted, reason}`.

### 4.4 V-4 Data Classification
- **Input.** DraftRecommendation + evidence classifications.
- **Function.** Compute the effective classification of the recommendation (max of contributing evidences + any policy uplift). Refuse egress paths that would violate classification (e.g., a recommendation implying export of R-classified data to an unauthorised recipient).
- **Output.** `ClassificationVerdict {effective_class, allow|deny, reason}`.

### 4.5 V-5 Output Validation
- **Input.** DraftRecommendation structured fields.
- **Function.** Schema conformance, size limits, referential integrity (every referenced ID is well-formed, resolvable), presence of required fields (per ARCH-15 §8), and prose-length bounds. Reject if any required structured field is missing or malformed.
- **Output.** `OutputVerdict {allow|deny, per-field detail}`.

### 4.6 V-6 Risk Classification
- **Input.** DraftRecommendation + Risk Engine (E-14) score for the proposed action context.
- **Function.** Compare the score to policy-defined ceilings for the classification and the tenant. If score above ceiling → require approval uplift (4-eyes or hold) or deny.
- **Output.** `RiskVerdict {score, ceiling, allow|require_approval|deny, reason}`.

### 4.7 V-7 Hallucination Detection
- **Input.** DraftRecommendation, citations, structured claims.
- **Function.** For each factual claim (e.g., "IP 10.1.2.3 attempted 40 logins"), resolve the claim against the cited evidence. If the claim cannot be constructed from the cited evidence, flag as hallucination. Includes:
  - Numeric-claim verification (counts, thresholds, ratios).
  - Entity-consistency (subjects, resources referenced also appear in cited evidence).
  - Timestamp reasonableness (claims align with `observed_at` monotonic ordering).
- **Output.** `HallucinationVerdict {clean|flagged, unresolved_claims: [...]}`. Flagged = fail unless approver overrides.

### 4.8 V-8 Prompt Injection Detection
- **Input.** DraftRecommendation + composed prompt (signed at AI Engine handoff) + response.
- **Function.** Multi-signal detector:
  - **Structural:** the response's structured fields don't imply cross-boundary intent that content should not induce (e.g., recommendation to publish to an unrelated tenant, action-parameters that exceed agent capabilities).
  - **Content-level:** classifier signal indicating instruction-like patterns in data fields (e.g., "ignore your prior instructions").
  - **Behavioral:** tool-call attempts or resource references that don't correspond to legitimate task context.
  - **Distributional:** deviation from baseline response distribution for the tenant + case shape.
- **Output.** `InjectionVerdict {clean|suspected|confirmed, signals, reason}`. Suspected → hold; Confirmed → deny + high-severity alert.

## 5. Composition Rule

- **All V-1..V-8 MUST return "allow" (or "allow with adjustment") for the ValidatedRecommendation to be emitted.**
- **Any single denial blocks emission.**
- **"Require approval" verdicts uplift the approval requirement in Multi-Engine Adjudication (ARCH-11 stage 7).**
- **Verdicts are signed** by the Safety Layer's SVID and included in the ValidatedRecommendation for downstream verification.

Determinism: given the same DraftRecommendation + Policy corpus + Evidence state + Risk catalog, the Safety Layer produces the same verdicts. This is the reproducibility contract.

## 6. Content Quarantine

The Safety Layer receives content that was already quarantined at prompt-composition time (ARCH-15 §7). It also treats *its own* free-text fields (recommendation prose) as untrusted for downstream tooling:
- Prose fields are not parseable for tool calls, capabilities, or entity references — only the structured fields are.
- Rendering guidelines (Phase D UI) forbid rendering prose as executable UI.

## 7. Multi-Model Consensus (R/HR)

For classification ≥ R:
- The Safety Layer requires **two independent AI outputs** for the same request context.
- Consensus rule: (a) both allow with matching structured conclusions → allow; (b) disagreement → hold for human approval with both DraftRecommendations presented and the disagreement structured (which fields differ).
- Independence: different model families (e.g., L2-Qwen + L2-Llama) or different tiers (L2 + L3 with fresh session), never the same model twice.

For classification HR:
- Consensus of two independent models is mandatory.
- Additionally, one model MUST be L2 (never L3 alone).
- Additionally, Multi-Engine Adjudication (ARCH-11) treats Risk ceiling as automatically requiring 4-eyes approval.

## 8. Adaptive Trust Integration

The Safety Layer emits **trust signals** consumed by the Adaptive Trust vector:
- Per provider: rejection rate, hallucination rate, injection-suspected rate.
- Per tenant source: injection-confirmed rate.
- Per model version: verdict-distribution drift.

Downstream: trust deltas feed the router (§ARCH-15 §3.2) and policy — a provider whose hallucination rate rises falls out of the routing set until reviewed.

## 9. Safe Failure Modes (Mandate #7)

| Mode | Trigger | Behavior |
|---|---|---|
| **Normal** | All validators available | Validate all; emit ValidatedRecommendation on allow |
| **Degraded** | One validator unhealthy (excluding V-2 Evidence and V-8 Injection) | Others still run; failed validator returns fail-secure deny; alert |
| **Recovery** | V-2 or V-8 unhealthy | Refuse *all* emission; queue; alert; L1 deterministic recommendations continue via AI Engine bypass path |
| **Maintenance** | Weight/catalog reload in progress | Queue; drain on completion; last-known-good pin |
| **Emergency** | Injection-confirmed alarm burst, Kernel integrity alarm, integrity failure on Safety Layer itself | Suspend all emission; drain; escalate; requires operator ack |

## 10. Recovery-First Answers

1. *How does it fail?* Slow validator, wrong verdict, injection undetected, Policy Engine unreachable, Evidence Engine unreachable, integrity failure.
2. *Detection.* Per-validator latency SLO, verdict-distribution drift, cross-checks against Multi-Engine Adjudication, integrity beacons for the Safety Layer itself.
3. *Initiation.* Automatic fail-secure per validator; runbook for Emergency.
4. *Automated?* Automatic for transient; Emergency requires operator ack.
5. *Rollback?* Validator versions pinned; last-known-good rollback via catalog.

## 11. Independent Architecture Review (Reviewer 1)

### 11.1 Hidden Assumptions

| Assumption | Handling |
|---|---|
| Hallucination Detection's numeric-claim verifier is precise enough. | Deterministic where feasible; probabilistic with flagged fallback; borderline → hold rather than allow. |
| Injection Detection's classifier is durable to novel patterns. | Multi-signal + distributional drift monitor; live red-team injection tests during Continuous Monitoring; residual risk accepted with compensating controls (mandatory citations + human review). |
| Policy corpus is coherent enough that Safety and Policy converge. | Policy Engine tests + property checks; divergence detected + alerted; ceremony to reconcile. |

### 11.2 SPOFs

| Finding | Response |
|---|---|
| **F-1.** *V-2 Evidence Engine unreachable → all validation stalls.* | Fail-secure deny; L1 deterministic pipeline continues (Deterministic Core); documented degraded mode. |
| **F-2.** *Safety Layer replica set outage.* | Kernel refuses to accept non-validated recommendations; L1 continues; ingest continues; case work continues without AI recs. |

### 11.3 Privilege Escalation

| Finding | Response |
|---|---|
| **F-3.** *Compromised Safety Layer emits ValidatedRecommendation without validators.* | Verdicts are signed by *each* validator sub-signer? To reduce complexity: single Safety Layer SVID with structured verdict records; watchdog (ARCH-13) samples verdict integrity; forgeries detected. |
| **F-4.** *AI Engine bypasses Safety by publishing directly to `ai.validated`.* | Broker ACL binds `Publish-Validated-Recommendation` exclusively to Safety Layer SVID; AI Engine cannot publish. |

### 11.4 Trust-Boundary Violations

| Finding | Response |
|---|---|
| **F-5.** *Safety Layer downgrades classification silently.* | V-4 is deterministic; downgrade requires explicit Policy rule; no silent path. |
| **F-6.** *V-8 Injection signals leak into telemetry.* | Detection signals are C-classified; egress gated; no raw injection content in telemetry. |

### 11.5 Bottlenecks

| Finding | Response |
|---|---|
| **F-7.** *Sequential validators add latency.* | Independent validators run in parallel; V-2 Evidence is the slowest, budgeted; end-to-end target ≤ 1s p95 excluding provider inference time. |
| **F-8.** *Multi-model consensus doubles inference cost for R/HR.* | Cost budget per tenant; policy governs when R/HR requires consensus; documented tradeoff. |

### 11.6 Supply Chain

| Finding | Response |
|---|---|
| **F-9.** *Injection classifier model compromised.* | Multi-signal (structural + content + behavioral + distributional); no single model failure allows silent bypass. |
| **F-10.** *Hallucination detector library vulnerability.* | Hash-pinned; SBOM; small surface; reviewed as security-critical path. |

### 11.7 AI-Specific Risks

| Finding | Response |
|---|---|
| **F-11.** *Injection that mimics permitted structural output shape.* | V-8 structural check plus V-2 evidence check plus Multi-Engine Adjudication — three independent gates. |
| **F-12.** *Model self-report of confidence dominates V-3.* | V-3 recomputes; discounts by trust; model self-report is one component among many. |

### 11.8 Operational Risks

| Finding | Response |
|---|---|
| **F-13.** *Alert fatigue on Injection-suspected verdicts.* | Suspected leads to hold + Multi-Engine Adjudication + human review only when confirmed patterns accumulate; distinct suspected/confirmed thresholds. |
| **F-14.** *Verdict-schema evolution breaks downstream tooling.* | Verdicts versioned; downstream tolerates forward-compatible additions; deprecation cycle. |

## 12. Adversarial Architect Review (Reviewer 2)

| Attack path | Design response |
|---|---|
| **A-1.** Prompt-inject to produce a DraftRecommendation whose structured fields exactly resemble a permitted class of recommendations. | V-2 Evidence resolves every citation to real evidence; V-7 Hallucination verifies numeric/entity claims against cited evidence; V-6 Risk score requires Approval uplift for high-consequence categories. Attacker must produce a recommendation that (a) has real citations, (b) is low-risk, (c) is *actually useful to them* — very hard combination. |
| **A-2.** Compromise Safety Layer SVID + secretly bypass validators. | Watchdog (Audit Engine, ARCH-13) samples ValidatedRecommendation → re-runs verdicts → alarms on divergence. Compromise is not silent. |
| **A-3.** Race condition where Safety accepts a DraftRecommendation whose evidence is being tombstoned. | V-2 fails if evidence tombstoned at read time; verdict is time-stamped; race window bounded by Evidence Engine's write ordering. |
| **A-4.** Multi-model consensus defeated by compromising both models similarly (supply-chain both). | Different model *families* required; ARCH-14 §8 canary evals; provider trust decay; documented as residual risk requiring vigilance. |
| **A-5.** Injection-classifier evasion via a novel encoding. | Multi-signal + drift monitor + explicit fail-secure on distributional anomalies + human red-team injections rotate through the injection corpus. |
| **A-6.** Insider modifies Injection thresholds to reduce sensitivity. | Thresholds live in ceremony-controlled catalog (ARCH-11 §8); changes ceremony-audited. |

## 13. Operational Reliability Review (Reviewer 3)

| Concern | Response |
|---|---|
| **O-1. Availability under provider outages.** | Safety Layer independent of providers; deterministic; scales horizontally. |
| **O-2. Verdict schema evolution.** | Versioned; forward-compat additions; documented deprecation. |
| **O-3. Runbook maturity for Emergency mode.** | Runbook with drilled steps; automatic escalation channels; documented recovery. |
| **O-4. Observability.** | Metrics: per-validator latency + rejection rate + verdict distribution; traces for each ValidatedRecommendation lifecycle; per-tenant summaries surfaced to the tenant admin UI. |
| **O-5. Upgrade safety.** | Canary tenant subset; catalog pin per Safety Layer version; automatic rollback on rejection-rate spike. |
| **O-6. Long-term maintenance.** | Small validators, each independently testable, replaceable. Simple structural rules over ML fashion. |
| **O-7. Debug ergonomics.** | Structured verdicts with per-field reasons — a rejection is grep-able. |
| **O-8. Sizing under bursty load.** | Queue metrics per validator; per-tenant fairness; documented sizing. |

## 14. Attacker's First-Target Analysis and Redesign

**"If I were an experienced attacker, what part of this design would I target first?"**

**V-7 Hallucination Detection.** Reason: this validator is the hardest to make *sharp*. Numeric and entity checks work well when the cited evidence *contains* the referenced numbers/entities. But recommendations often talk about *derived* properties — "abnormal login pattern for user X" — that aren't literal fields in evidence. A validator that's too strict rejects everything real; too loose lets hallucination through. Attackers who understand this fault line can craft injections that produce "plausibly derived" claims.

**Redesign response.**

1. **Two-mode V-7.**
   - **Strict mode (default for R/HR).** Numeric/entity checks must resolve literally in cited evidence. No derivation. Fail = deny.
   - **Derived mode (for P/I/C).** Numeric/entity claims may be *aggregate* of cited evidence, verified by a small deterministic aggregator (counts, sums, rate ratios) that AEGIS runs against the citation set. If the aggregator confirms the claim, allow. If not, fail.
2. **Deterministic aggregators over LLM verification.** V-7 never uses an LLM to check an LLM. Aggregators are simple structured operations (SQL-like over the evidence set).
3. **Claim schema.** DraftRecommendation's structured claims declare their type (literal / aggregate / narrative). V-7 routes verification by type. Narrative claims lack aggregate verification and are allowed only in the prose field where they cannot influence machine-verifiable actions.
4. **Attribution of unresolved claims.** Any unresolved claim is preserved in the verdict with a specific `unresolved_claim` code. Downstream (Multi-Engine Adjudication) can decide to hold, and the analyst sees exactly which claim failed to resolve.
5. **Continuous canary claims.** Recovery Engine periodically injects benign test cases whose ground truth is known; V-7 must correctly classify them; regressions alarm.

**Second target after redesign.** V-8 injection classifier. Response: multi-signal + distributional monitor + explicit red-team rotation (weekly test corpus refresh) + suspected-vs-confirmed thresholds tuned per tenant sensitivity. If the classifier is fully bypassed, structural and behavioral signals (unusual capability requests, unusual recipients) still catch typical injection patterns.

**Third target.** V-1 Policy validation caching. Response: Safety Layer's Policy check re-consults Policy Engine on cache miss and on periodic revalidation window; divergence between cached and live decision is an alarm. Continuous Validation applied to caching itself.

No further high-impact silent-compromise vector identified at this abstraction.

## 15. Security Economics (Mandate #10)

- **Attacker cost raised.** To produce a validated recommendation, an attacker must simultaneously: satisfy V-2 (real citations), V-7 (claims resolve to cited evidence), V-8 (no injection signals), and often multi-model consensus. Each is a separate mechanism.
- **Attacker cost raised.** Compromise of the Safety Layer is *loud* because watchdog re-runs a random sample.
- **Defender cost reduced.** Deterministic validators are grep-able; failures are structured; runbooks are short.

## 16. Decisions

### D-16-1. Safety Layer as a distinct sub-engine with its own SVID

- **Advantages.** Two-identity requirement to forge validated output; clean structural separation from AI reasoning.
- **Disadvantages.** One more deployment surface.
- **Security Impact.** *Strongly positive.*
- **Performance Impact.** *Neutral.*
- **Operational Complexity.** *Moderate.*
- **Maintainability.** *Positive.*
- **Scalability.** *Positive.*
- **Alternatives.** *Safety inside AI Engine.* Concentrates trust; rejected.
- **Reason.** Zero SPOT applied to AI output publication.

### D-16-2. Eight deterministic validators, not an LLM safety judge

- **Advantages.** Deterministic; grep-able; reproducible; independently maintainable.
- **Disadvantages.** Requires careful design of Hallucination Detection.
- **Security Impact.** *Strongly positive.*
- **Performance Impact.** *Positive* — no LLM-in-safety-loop latency.
- **Operational Complexity.** *Moderate.*
- **Maintainability.** *Positive.*
- **Scalability.** *Positive.*
- **Alternatives.** *LLM-as-judge.* Adds a novel attack surface; rejected. *Fewer validators.* Insufficient coverage; rejected.
- **Reason.** Mandate #24 (Simplicity), Mandate #1 (Human Authority).

### D-16-3. Two-mode V-7 with deterministic aggregators

- **Advantages.** Addresses the primary redesign target (§14); realistic sharpness.
- **Disadvantages.** Aggregator coverage is finite.
- **Security Impact.** *Strongly positive.*
- **Performance Impact.** *Small.*
- **Operational Complexity.** *Moderate.*
- **Maintainability.** *Positive.*
- **Scalability.** *Positive.*
- **Alternatives.** *Strict-only.* Too many false rejects; rejected. *Loose-only.* Injections pass; rejected.
- **Reason.** Balanced sharpness with deterministic verification.

### D-16-4. Multi-model consensus mandatory for R/HR

- **Advantages.** Single-model compromise doesn't silently produce bad output.
- **Disadvantages.** Cost + latency for R/HR.
- **Security Impact.** *Strongly positive.*
- **Performance Impact.** *Moderate* — accepted tradeoff.
- **Operational Complexity.** *Moderate.*
- **Maintainability.** *Positive.*
- **Scalability.** *Positive.*
- **Alternatives.** *Single-model for all.* Rejected. *Consensus for all.* Cost-prohibitive; rejected for low-classification.
- **Reason.** Risk-proportional design.

## 17. Open Questions

- Q-16-1. Injection-classifier model choice + refresh cadence. Resolved in ARCH-18.
- Q-16-2. Aggregator DSL (SQL vs. bespoke). Resolved in Phase C.
- Q-16-3. Consensus timeout policy when the second model is slow. Resolved in ARCH-18.

## 18. Change Log

- **0.1 (2026-07-10)** — Initial draft after three-reviewer discipline.