# AEGIS — Evidence Chain Architecture

- **Document ID:** ARCH-12
- **Phase:** B.2 — Data, Evidence, Supply Chain
- **Status:** Draft for review (post two-reviewer discipline)
- **Version:** 0.1
- **Date:** 2026-07-10
- **Owner:** Chief Security Architect
- **Depends on:** ARCH-04, ARCH-07, ARCH-10, ARCH-11
- **Consumed by:** ARCH-13 through ARCH-16

---

## 1. Purpose

Specify the Evidence Chain: how AEGIS preserves evidence, tracks provenance, scores confidence, records transformations, and produces reproducible investigations. This document specifies the **Evidence Engine (E-15)** and the **explainability contract** for AI recommendations. It implements elevation mandates #5 (Evidence Confidence), #6 (Explainable Security), and #11 (Digital Evidence Chain), building on the immutable-event-sourcing principle established in ARCH-04 and ARCH-10.

## 2. Model

Two artifact classes:

- **Facts** — the immutable, verbatim record of an observation (a raw event as received, an analyst action as performed, an audit record as written). Facts are never mutated.
- **Derivations** — additive artifacts computed from Facts and other Derivations (normalized events, findings, cases, timelines, recommendations, exports). Every Derivation references its inputs by ID and integrity hash.

Two graph structures over these:

- **Provenance graph.** Directed acyclic graph from Facts to Derivations. Every node signed by its producer.
- **Confidence overlay.** Per-edge and per-node confidence metadata contributed by the Evidence Engine.

## 3. Evidence Metadata (mandate #5)

Every evidence element carries the following, structurally required:

| Field | Meaning | Producer |
|---|---|---|
| `evidence_id` | Stable identifier | Producer engine |
| `type` | Fact class (raw event, analyst action, audit record, external signal…) | Producer |
| `source` | Structured origin: source identity, source type, tenant, connector version, collection endpoint | Producer + Connector |
| `integrity` | Content-addressed hash (SHA-256 v0; algorithm-agile) + producer signature | Producer + Kernel-audited |
| `confidence` | Aggregate confidence score (0..1) with component breakdown | Evidence Engine |
| `timestamp` | Both `event_time` (best-known) and `observed_at` (receipt) with monotonic anchor | Producer |
| `collection_method` | How obtained: pushed-syslog, pulled-audit-log, forwarder, IdP callback, manual-upload… | Connector |
| `transformation_history` | Ordered list of `(transformer_id, transformer_version, input_hashes → output_hash, signature)` | Every transformer |
| `trust_score` | Trust of the source of this evidence, from the Adaptive Trust vector at the time of use | Evidence Engine |
| `classification` | Data classification | Producer + Policy |
| `references` | Upstream evidence IDs this element depends on | Producer |
| `signature` | Signature over the tuple by the producer's SVID | Producer |

Rule: any evidence used in a decision that fails to carry all required fields is treated as *unusable*. Fail-secure.

## 4. Evidence Engine (E-15) Specification

### 4.1 Purpose
Own evidence provenance, confidence scoring, transformation history, trust scoring, and the explainability contract. Serve the Multi-Stage Decision Pipeline stages S-1 and S-2. Supply the Case Engine with the material for signed evidence bundles.

### 4.2 Responsibilities

1. **Provenance registry.** Ledger of Facts and Derivations with signatures. Independent of Storage Engine's storage substrate; verifiable independently.
2. **Confidence scoring.** Compute confidence per element and aggregate across a bundle; publish decomposition so downstream can reason about it.
3. **Transformation ledger.** For every transformer (Normalization, redaction, enrichment, AI summarization), record signed input→output tuples so any Derivation is reproducible.
4. **Trust scoring inputs.** Publish per-source, per-provider trust signals consumed by the Adaptive Trust vector (ARCH-11 §6).
5. **Explainability broker.** Materialize the answer to the 8 mandatory explainability questions for any AI recommendation (§7).
6. **Reproduction harness.** Given a `case_id` or `recommendation_id`, replay the exact input set and produce a deterministic verification.
7. **Chain-of-Custody records.** For every access to R/HR evidence, record who/when/why with signed receipts.

### 4.3 Interfaces

- `SubmitFact(FactRecord)` — from producers.
- `SubmitDerivation(DerivationRecord)` — from transformers.
- `ScoreEvidence(evidence_ids, context)` → `ConfidenceReport` — for pipeline S-2.
- `ExplainRecommendation(recommendation_id)` → `ExplainabilityBundle` — for UI + audit.
- `ReproduceRecommendation(recommendation_id)` → `ReproductionResult` — for verification.
- `ChainOfCustody(evidence_id)` → `CustodyTrail` — for compliance / DFIR.

Every response is signed by the Evidence Engine's SVID.

### 4.4 Recovery-First answers

1. *How does it fail?* Missing signatures; broken hash chains; slow scoring; wrong confidence weights.
2. *Detection.* Signature-verification failure rate; broken-link scanner; latency SLOs; drift monitor on confidence distributions.
3. *Initiation.* Automatic re-scoring on catalog change; runbook for broken-link incidents.
4. *Automated?* Yes for transient; ceremony-controlled for catalog changes.
5. *Rollback?* Confidence weights versioned; historical scores preserved with the version they were computed under.

## 5. Immutable Event Sourcing — formalized

### 5.1 Storage invariants

- Facts are written to append-only tables (PostgreSQL) or append-only-blocks (ClickHouse). No `UPDATE` or `DELETE` grant on Fact tables to any engine identity except the specific tombstone role (§5.3).
- Derivations carry `input_refs` with integrity hashes; Storage Engine refuses a Derivation whose `input_refs` cannot be resolved.
- Version bumps on transformers create new Derivations; old Derivations are preserved.

### 5.2 Historical replay

- Given any `case_id`, the platform can replay: pick the original Facts by `input_refs`, run the exact transformer versions (from Provenance registry), and produce the same Derivations.
- Non-determinism in AI outputs is bounded: prompts and provider/model/temperature/seed are recorded; L1 (deterministic) is bit-reproducible; L2 with `temperature=0` and `seed` is reproducible on the same runtime; L3 reproducibility depends on the provider (documented per provider).

### 5.3 Legal-hold and deletion

- Right-to-erasure and legal-hold are in tension. AEGIS resolves this via:
  - **Tombstone-with-preserve.** Erasure marks the evidence as tombstoned; content is replaced with a hash reference + reason; the *proof-of-existence* remains for the audit chain.
  - **Legal-hold veto.** An active hold prevents tombstoning; hold changes are Layer-A + Layer-B audited.
  - **Cryptographic erasure.** For asymmetric use cases, tenant DEK destruction cryptoshreds ciphertext (data becomes unreadable while hash-proof persists).
- Tombstoning is a ceremony operation (`Tombstone-Evidence` capability, one-shot, 4-eyes, break-glass required for R/HR).

## 6. Chain of Custody

Every access to R/HR evidence, and every access made under Break-Glass, is a signed custody event:

```
CustodyEvent {
  evidence_id
  actor_id           // human or agent
  action             // read | export | tombstone | reproduce
  reason             // required for R/HR
  case_id            // optional link to the investigating case
  approvals          // if any
  signature          // by Kernel + actor's SVID
  timestamp
}
```

Custody trails are stored by the Evidence Engine (control plane) and mirrored to Layer-A audit. On evidence export, the entire custody trail up to that point is included in the signed bundle.

## 7. Explainability Contract (mandate #6)

For every AI recommendation, the Explainability Bundle (§4.3) contains signed answers to eight questions:

1. **Why?** — the top-N reasoning steps in natural-language plus structured factors used.
2. **Which evidence?** — resolved list of `evidence_id + integrity_hash` referenced.
3. **Which policy?** — Policy Engine rules invoked by ID + version.
4. **What confidence?** — aggregate + component confidence + risk-adjusted confidence (see ARCH-11 §4.2 point 6).
5. **Alternative explanations?** — enumerated alternative hypotheses considered and why not chosen.
6. **Potential false positives?** — how this could be wrong (adverse conditions, coincidences).
7. **Potential false negatives?** — what would be missed if this recommendation were suppressed.
8. **Recovery recommendation?** — if acted upon, how to reverse; if not, how to detect harm.

If any of these cannot be produced, the recommendation is *held* — the AI Safety Layer refuses to publish it. This is a structural gate.

## 8. Digital Evidence Chain (mandate #11)

For every investigation, the exported evidence bundle contains:

- **Evidence Origin** — for each Fact, the source identity + collection method + collection endpoint.
- **Evidence Integrity** — SHA-256 hash + producer signature; verifiable from public keys.
- **Evidence Hash** — root Merkle hash of the bundle; individual leaf hashes.
- **Collection Timestamp** — receipt + best-known event time + monotonic anchor.
- **Collection Method** — precise mechanism (per ARCH-04 asset ID references).
- **Chain of Custody** — the custody trail per §6.
- **Transformation History** — full ledger from each Fact to the bundle content.
- **Analyst Actions** — every case action from Case Engine, signed.
- **AI Reasoning References** — the Explainability Bundle for every AI-derived element.

Every investigation is **reproducible** via §5.2.

## 9. Bundle Format and Signing

- **Format.** Structured, versioned, CBOR (compact) or JSON (readable). Both options published; downstream tools accept both.
- **Signing.** Case Engine signs with its export-signing key (AS-K-03). Merkle-tree structure enables partial disclosure without disclosing the whole bundle.
- **Verification tools.** AEGIS publishes an open-source verification CLI so any auditor can verify a bundle offline using only public keys. This is the primary reason for the algorithm-agile signature format (D-09-3).

## 10. Independent Architecture Review

### 10.1 Hidden Assumptions

| Assumption | Handling |
|---|---|
| Every producer will implement metadata correctly. | Producer libraries provide a single API that populates required fields; SAST checks in ARCH-17; runtime schema validation refuses malformed evidence. |
| L2/L3 model outputs are deterministic enough to replay. | Provider adapter records seed/temperature/model version; documented reproducibility per provider; where non-deterministic, the recommendation is still verifiable by *content*, not by *reproduction*. |
| Transformation ledger will not become unbounded. | Ledger is time-tiered; hot in Postgres, cold in object store; per-tenant retention. |

### 10.2 SPOFs

| Finding | Response |
|---|---|
| **F-1.** *Evidence Engine down → S-1/S-2 halt.* | Fail-secure default (per ARCH-11 §9 F-2). Horizontally scalable. |
| **F-2.** *Provenance registry corruption.* | Registry is dual-substrate: PG + immutable object store; hash-chain verification; watchdog compares. |
| **F-3.** *Signing key compromise for the Evidence Engine.* | Short-lived leaf keys; root anchored under Z0; compromise runbook. |

### 10.3 Privilege Escalation

| Finding | Response |
|---|---|
| **F-4.** *An engine forges a fact with a plausible source.* | Fact producer identity is the SVID; forging requires SVID theft; Connector Engine is the only publisher of source-facts; tenant-scope binds. |
| **F-5.** *Analyst tombstones evidence to hide activity.* | Tombstone-Evidence is one-shot + 4-eyes + break-glass; auditable at high severity. |
| **F-6.** *AI Engine calls ExplainRecommendation on a peer's rec.* | Explainability is bound to the recommendation's originator SVID and the case scope; cross-tenant refused. |

### 10.4 Trust-Boundary Violations

| Finding | Response |
|---|---|
| **F-7.** *Chain-of-custody event missed when Break-Glass used.* | Kernel-mediated Break-Glass generates custody events non-bypassably. |
| **F-8.** *Bundle contains upgraded classifications.* | Bundle manifests carry classification; export path composes with Policy; recipient must have receiving capability. |

### 10.5 Bottlenecks

| Finding | Response |
|---|---|
| **F-9.** *Signature verification on every retrieval is expensive.* | Cached verification for hot reads with TTL bounded by trust window; batch verification for bulk operations. |
| **F-10.** *Transformation ledger writes on high-volume normalization.* | Batched writes; per-batch signature; per-record integrity hash still preserved. |

### 10.6 Supply Chain

| Finding | Response |
|---|---|
| **F-11.** *A vulnerable JSON/CBOR library taints bundle parsing.* | Hash-pinned; SAST rules on unsafe patterns; verifier CLI has narrow input surface; open-source review invited. |
| **F-12.** *Compromised producer signs bad Facts as legitimate.* | Adaptive Trust (ARCH-11) decays producer trust on downstream verification failure; Detection Engine flags anomalous facts. |

### 10.7 AI-Specific Risks

| Finding | Response |
|---|---|
| **F-13.** *Explainability answers are themselves LLM-hallucinated.* | Every explanation answer references structured facts; Q2/Q3 must resolve (evidence + policy IDs), Q4 comes from Risk Engine, Q6/Q7 from a bank of scored alternatives; free-text is bounded and cross-verified. |
| **F-14.** *L3 provider retention exposes prompts including citations.* | Zero-data-retention modes preferred; classification-based routing; ARCH-09 §5-7. |

### 10.8 Operational Risks

| Finding | Response |
|---|---|
| **F-15.** *Evidence Engine's own confidence weights drift silently.* | Weights are catalog-versioned like Risk Engine (ARCH-11 §8); ceremony-governed. |
| **F-16.** *Bundle format churn breaks external verifiers.* | Versioned formats; verifier CLI supports all published versions; deprecation cycle. |

## 11. Adversarial Architect Review

| Attack path | Design response |
|---|---|
| **A-1.** Compromise Normalization Engine to forge a Derivation that omits an attacker's fact. | Raw preserved verbatim; Detection can re-run on raw; diff-monitor between two normalization versions on a fixture corpus alerts on drop discrepancies. |
| **A-2.** Insider tombstones a specific R evidence to erase a trace. | Tombstone is 4-eyes + break-glass + audit; a tombstone still leaves a hash-of-existence + reason; forensics can prove *what was here*. |
| **A-3.** Prompt-inject the Explainability Bundle to fool an auditor into misinterpreting a case. | Free-text answers are minimized and marked as advisory; structured fields (evidence IDs, policy IDs) are what auditors verify; verifier CLI presents structured view first, prose second. |
| **A-4.** Steal Case Engine export-signing key to forge a bundle claiming exculpatory evidence. | Key is short-lived and derived from a hierarchy under Z0; theft window bounded; compromise runbook revokes and re-issues; verifier CLI checks key freshness against Z0. |
| **A-5.** Cloud compromise reveals bulk raw evidence. | Envelope encryption per tenant DEK; egress detection; alert on unusual KMS use. |
| **A-6.** Supply chain compromise of the verifier CLI produces false-positive verification. | Verifier CLI is source-available, reproducible-build; multi-vendor verifier options; standardized bundle format enables third-party verifiers. |
| **A-7.** Plugin compromise attempts to submit forged Derivations. | Plugins never hold `SubmitFact`/`SubmitDerivation` capability by default; explicit grants required + audited + sandboxed. |
| **A-8.** Model provider silently trains on submitted evidence. | Redaction manifest pre-egress + zero-data-retention providers only for R+ classification; trust decay + provider audit; explicit tenant consent. |

## 12. Attacker's First-Target Analysis and Redesign

**"If I were an experienced attacker, what part of this design would I target first?"**

**The transformation ledger.** Reason: it's the single artifact that must be *complete and correct* for reproduction to work. If an attacker can corrupt a specific entry — replacing a Normalization transformer version reference with a benign-looking one — the reproduction pass would succeed (producing the wrong result deterministically) and no downstream check would catch it. This is the classic "poisoned build system" pattern applied to evidence.

**Redesign response.**

1. **Dual-store transformation ledger.** Every ledger entry is written to both PostgreSQL (control plane) and appended to a signed batch in immutable object storage. Evidence Engine, at each read, verifies both are consistent; divergence halts and alerts.
2. **Hash-committing transformers.** Every transformer publishes its build hash + config hash at startup to a signed registry; ledger entries reference this hash by ID. Substituting a "look-alike" version fails because its build hash won't match the registry entry created at transformer startup.
3. **Independent reproduction check.** Recovery Engine (in the Continuous Monitoring stage of the Pipeline) periodically reproduces a random-sample of recent recommendations; divergence alerts. This is a live canary against silent transformer substitution.
4. **Transformation ledger has its own SVID hierarchy** — the ledger-signing key is separate from other Evidence Engine keys; compromise of one does not compromise the other.
5. **Second target after redesign:** the *verifier CLI* itself. If everyone verifies with the same tool and that tool is compromised, all bundles look valid. Response: verifier CLI is source-available, reproducible-build, community-verifiable; AEGIS publishes a hash reference; a *specification* (not just the tool) is published so alternative verifiers can be built. Third-party verification is a first-class supported path.

Repeated: further target candidate: the *classification header on Facts*. If dropped or downgraded during transmission from Connector to Storage, downstream policy is misled. Response: classification is included in the signature payload — tampering breaks the signature.

## 13. Decisions

### D-12-1. Evidence Engine as first-class engine

- **Advantages.** Distributed trust; owns provenance/confidence/explainability separately from Case Engine; enables Zero SPOT for evidence.
- **Disadvantages.** One more engine.
- **Security Impact.** *Strongly positive.*
- **Performance Impact.** Neutral (registry writes are batched).
- **Operational Complexity.** Moderate.
- **Maintainability.** Positive.
- **Scalability.** Positive.
- **Alternatives.** *Merge into Case Engine.* Concentrates trust; rejected. *Merge into Storage Engine.* Couples data plane to evidence semantics; rejected.
- **Reason.** Mandate #3 (Distributed Trust) + #5 + #11.

### D-12-2. Dual-store transformation ledger with hash-committing transformers

- **Advantages.** Defends against the primary redesign target (§12).
- **Disadvantages.** Extra write cost.
- **Security Impact.** Strongly positive.
- **Performance Impact.** Small.
- **Operational Complexity.** Moderate.
- **Maintainability.** Positive.
- **Scalability.** Positive.
- **Alternatives.** *Single-store ledger.* Rejected by adversarial review.
- **Reason.** Ledger integrity is the reproduction guarantee; dual-store + hash commit is the structural defense.

### D-12-3. Tombstone-with-preserve for right-to-erasure

- **Advantages.** Reconciles compliance with audit-integrity requirements; proof-of-existence retained; cryptographic erasure available where content itself must go.
- **Disadvantages.** Not a full-delete for some regulators — must be explained in DPA (data processing agreement).
- **Security Impact.** Positive.
- **Performance Impact.** N/A.
- **Operational Complexity.** Moderate.
- **Maintainability.** Positive.
- **Scalability.** Positive.
- **Alternatives.** *Hard delete.* Breaks audit chain; rejected. *No erasure.* Non-compliant.
- **Reason.** Right compromise between conflicting regulatory demands.

### D-12-4. Structured-first explainability with bounded prose

- **Advantages.** Prose is not the source of truth; structured references are; auditor-friendly.
- **Disadvantages.** Less "natural" for casual reading.
- **Security Impact.** Positive.
- **Performance Impact.** Neutral.
- **Operational Complexity.** Neutral.
- **Maintainability.** Positive.
- **Scalability.** Positive.
- **Alternatives.** *Prose-primary.* Attacker-writable through AI; rejected. *Structured-only.* Loses intuition; rejected.
- **Reason.** Balance verifiability with usability.

## 14. Open Questions

- Q-12-1. Verifier CLI language + distribution channel. Recommendation: Rust (as per Q-06-1) for the verifier — narrow safe surface. Resolved in Phase C.
- Q-12-2. Bundle format canonical serialization (CBOR-primary + JSON secondary). Resolved in Phase C.
- Q-12-3. Transparency-log-style public witness for bundle hashes (long-term). Roadmap.

## 15. Change Log

- **0.1 (2026-07-10)** — Initial draft after two-reviewer discipline.