# AEGIS — Logging & Audit Standard

- **Document ID:** ARCH-20
- **Phase:** C — Standards & Operations
- **Status:** Draft for review (post four-reviewer discipline)
- **Version:** 0.1
- **Date:** 2026-07-10
- **Owner:** Chief Security Architect
- **Depends on:** ARCH-04, ARCH-06, ARCH-08, ARCH-09, ARCH-11, ARCH-12, ARCH-13
- **Consumed by:** ARCH-21 through ARCH-24 and every implementation-phase document.

---

## 1. Purpose

Codify how AEGIS logs and audits — schema, retention, redaction, verification, and separation. This standard implements Phase-A/B decisions (D-03-3 split audit, D-04-3 separate substrates, D-06-3 local integrity ring) as prescriptive rules producers must follow.

Vocabulary distinction (**MUST** be respected in code, docs, UX):
- **Log** — operational or security-relevant record kept for debugging/observability; not an evidentiary artifact.
- **Audit** — evidentiary record, hash-chained, signed, tamper-evident, produced when a *sensitive operation* occurs (per CLAUDE.md contract).

Logs are lossy under pressure. Audits are not.

## 2. Requirements

- **REQ-1.** Every sensitive operation produces a Layer-A audit record synchronously; Layer-B archival is asynchronous but guaranteed within a policy-defined window.
- **REQ-2.** Every engine emits structured logs, metrics, and traces to the Telemetry Engine (E-10) via OpenTelemetry.
- **REQ-3.** Logs and audit records are on separate substrates with distinct keys (D-04-3).
- **REQ-4.** Independent verification of the audit chain is possible using the published verifier CLI and public keys.
- **REQ-5.** Redaction of secrets and PII is applied *before* records leave the emitting process.

## 3. Mandatory Rules

### 3.1 Sensitive-operation Audit (Layer A + Layer B)

- **M-1.** Kernel-mediated operations produce a Layer-A record at the boundary crossing (per ARCH-05 BC-5, ARCH-06 §4.3).
- **M-2.** Layer-A record schema (v1) — required fields (per ARCH-02 F-6.6, extended):

  ```
  AuditRecord {
    schema_version   : uint
    event_id         : UUID
    tenant_id        : TenantId
    trace_id         : TraceId
    actor_id         : DID              // human, service, or agent
    operation        : OperationName
    resource_ref     : ResourceRef
    boundary_crossed : BoundaryLabel    // e.g., B-30
    policy_decision  : PolicyVerdict    // allow/deny/require-*/break-glass
    classification   : ClassificationLevel
    capability_id    : CapabilityId     // if capability was presented
    signer_key_id    : KeyId
    signer_key_alg   : AlgorithmId       // Cryptographic Agility, mandate #29
    previous_hash    : Sha256
    monotonic_seq    : uint64
    event_hash       : Sha256
    wall_time        : Time
    monotonic_time   : Time
    signature        : SignatureBytes
    extra            : { ... structured, redacted ... }
  }
  ```
- **M-3.** Every AuditRecord is signed by the Kernel-instance leaf. Layer-A hash chain per Kernel instance is a self-forming DAG rooted at instance-boot.
- **M-4.** Layer-B batches are signed by the Audit Engine's batch-signing key (separate hierarchy).
- **M-5.** Chain-gap detection: consumers of Layer-A verify `previous_hash + monotonic_seq`; any gap triggers a **KernelIntegrityAlarm** (ARCH-13 §5.3).
- **M-6.** Layer-B batches are written to Object Lock storage with compliance-mode retention (7 years default per ARCH-04, longer for legal-hold).
- **M-7.** Audit records MUST NOT be modified or deleted. Tombstone-with-preserve applies only to *content* fields under legal erasure — the record and its hash chain remain intact.

### 3.2 Sensitive-operation Enumeration

Operations that MUST produce Layer-A audit records include, non-exhaustively:
- Every Kernel-mediated boundary crossing (all `B-*` per ARCH-05).
- Every capability issuance, delegation, verification-denial, revocation, expiry, one-shot-consumption.
- Every identity lifecycle event (create, RBAC change, session establish, session revoke, break-glass request/approve/consume).
- Every policy change, catalog change, weight-catalog change (Risk, Trust, Cryptography, Detection, Redaction).
- Every configuration signed-bundle load or reload.
- Every AI request lifecycle: submitted, redacted, sent to tier, response received, Safety Layer verdicts (all eight), Multi-Engine Adjudication decision, Continuous Monitoring outcome.
- Every plugin lifecycle event: manifest submit, capability grant, install, enable, invoke (batched summary + anomaly detail), disable, kill, uninstall.
- Every Evidence Engine event: fact submit, derivation submit, tombstone, custody read.
- Every Recovery Engine event: backup, restore-drill, restore-real, recovery-integrity-attestation.
- Every Kernel Self-Verification event: beacon anomaly, watchdog alarm, isolation, recovery.

### 3.3 Operational Log Schema

- **M-8.** Logs are structured JSON (or Protobuf) with these required fields:
  - `time` (RFC 3339 with timezone + monotonic marker)
  - `level` (debug/info/warn/error/fatal)
  - `service` (engine name)
  - `instance_id`
  - `trace_id`, `span_id`
  - `tenant_id` (if applicable)
  - `event` (short slug identifying the log event class)
  - `message` (concise human string)
  - `attrs` (structured properties)
- **M-9.** No secrets or PII in logs. Redaction pipeline applied at boundary (§ 3.5).
- **M-10.** Log level `debug` is off by default in production; enabled per tenant with a time-boxed capability (`Enable-Debug-Logging`) and always audited.

### 3.4 Metrics and Traces

- **M-11.** Metrics: Prometheus/OpenMetrics naming (`aegis_<engine>_<metric>_<unit>`), bounded label cardinality.
- **M-12.** Every request emits an OpenTelemetry trace with span attributes: `tenant_id`, `trace_id`, `operation`, `classification`, `capability_id`, `boundary_crossed`.
- **M-13.** No PII in metric labels. `tenant_id` is opaque (UUID); tenant name is not a metric label.

### 3.5 Redaction

- **M-14.** Redaction runs *inside the emitting process*, before any log/metric/trace/audit leaves. Never in a downstream aggregator.
- **M-15.** Redaction pipeline:
  1. **Deterministic scanners** for well-defined secret patterns (JWTs, API keys per provider, RFC-defined tokens, RSA/EC key markers, session cookies, etc.).
  2. **Entropy-based scanner** for high-entropy strings likely to be tokens.
  3. **PII classifier** for structured PII fields (per tenant policy).
  4. **Deny-list** of exact-match patterns (per tenant).
  5. **Marker preservation** — redacted content replaced by `«REDACTED:<class>»` with a stable classifier tag; never dropped silently.
- **M-16.** Redaction failure = drop the record + emit a Redaction-Failure audit event (not a log; treat as sensitive-op) + alarm.
- **M-17.** A **RedactionManifest** is captured per record (structured summary of what was redacted) — mandatory for audit records; optional for logs where the volume is high.

### 3.6 Cardinality and Retention

- **M-18.** Log retention default: 30 days (Telemetry Engine), configurable per tenant + per class.
- **M-19.** Metric retention default: 90 days at fine resolution; downsampled at longer horizons.
- **M-20.** Trace retention default: 7 days full-fidelity; sampled at longer horizons.
- **M-21.** Audit retention default: 7 years (ARCH-04); Object Lock enforced.
- **M-22.** Cardinality caps: `< 50` distinct values per label; violations sampled + alerted.

### 3.7 Separation of Substrates

- **M-23.** Audit records go to a substrate independent of operational logs (D-04-3): separate storage keys, separate access paths, separate credentials, separate replication.
- **M-24.** No cross-substrate access; a compromise on logs cannot corrupt audit.

### 3.8 Time Integrity (Mandate #21)

- **M-25.** Every record carries both wall-clock (with source annotation) and monotonic markers.
- **M-26.** Clock drift monitored per instance; drift > policy threshold → alert; drift > hard threshold → refuse to sign new audit records (fail-secure).
- **M-27.** NTP or PTP source configured with fallback; time confidence recorded.

### 3.9 Independent Verifiability

- **M-28.** The AEGIS-published Rust verifier CLI (Q-12-1 resolution) verifies:
  - Layer-A hash chain integrity per Kernel instance.
  - Layer-B batch signature validity.
  - Public-key freshness against Z0-anchored release-hash registry.
  - Cross-instance stitching (per-tenant view across Kernel replicas).
- **M-29.** Public keys published; SDK examples in TS + Rust; documented cadence for key rotation.
- **M-30.** A third-party verifier is possible from the specification alone (D-12-4 posture).

## 4. Recommended Practices

- **R-1.** Log the *decision*, not the *input* — when in doubt, log outcome + IDs; avoid logging payloads.
- **R-2.** Use event slugs (`event: "capability.issued"`) rather than free text — enables structured search.
- **R-3.** Prefer one log line per event; avoid multi-line entries.
- **R-4.** Prefer counters over gauges where events are countable.
- **R-5.** Prefer tags/labels stable over releases; deprecate with a clear cycle.
- **R-6.** Never rely on the *order* of two independent audit records without a `previous_hash` link.
- **R-7.** For post-hoc reasoning, prefer `trace_id` correlation over string-matching messages.

## 5. Verification Process

- Per PR: schema-conformance tests for every emitted record type.
- Per merge: hash-chain property tests (monotonic_seq strictly increases, prev_hash matches, signatures verify).
- Per release: verifier CLI regression test suite against fixture audit sets across versions.
- Continuously: **audit-chain gap detector** (watchdog in Audit Engine, ARCH-13 §6.2).
- Weekly: sample independent verification by an auditor role.
- Monthly: end-to-end redaction canary — synthetic secrets injected; alarm if any leaks past redaction into logs/metrics/traces.

## 6. Operational Guidance

- **On-call runbook** for audit-chain gap: isolate suspected Kernel instance, capture forensics, verify replicated Layer-A copy is intact, initiate Kernel emergency recovery if warranted.
- **On-call runbook** for redaction-failure: rotate the redaction ruleset; back-scan affected retention window; incident review.
- **On-call runbook** for time-integrity failure: check NTP sources; escalate to hardware clock investigation; fail-secure on new signatures until resolved.
- **Tenant-facing** UI: audit view of one's own tenant with query + export.
- **Auditor-facing** UI: cross-tenant view (with break-glass) for regulator support.

## 7. Future Evolution

- **PQC migration.** `signer_key_alg` allows hybrid signatures per Cryptographic Agility.
- **Transparency log.** Layer-B batch hashes published to Sigstore Rekor + self-hosted witness (per Q-14-1 resolution). Enables public verifiability of continuity.
- **ZKP-based selective disclosure.** Long-term — auditors verify integrity of a window without accessing content (Future Readiness ARCH-09 §10).
- **Streaming audit query.** For very large deployments, columnar audit indexing (Phase 2).

## 8. Independent Architecture Review (Reviewer 1)

- **F-1.** *Logs are lossy under pressure; audit records get lost too.* Audit records are on the fast path with backpressure at the Kernel; if backpressure cannot be absorbed, the mediated operation fails-closed and the caller sees an error, not a silent audit skip.
- **F-2.** *Cardinality caps too strict for high-tenant deployments.* Caps are per-label per-tenant; per-tenant metrics scale with tenant count and are pre-aggregated.
- **F-3.** *Verifier CLI compromised → false confidence.* Rust reproducible build; open-source; hash published; specification enables third-party verifiers.
- **F-4.** *Sampled traces miss the important cases.* Sampling is adaptive: errors + high-classification ops always sampled 100%.

## 9. Adversarial Architect Review (Reviewer 2)

- **A-1.** *Insider mutates a Layer-A record on disk.* Signed records with hash chain; watchdog reads replicated copy; mutation on any single storage detected by chain-verification against archives.
- **A-2.** *Attacker suppresses audit records during an intrusion.* Beacon-chain gap + monotonic seq gap + Layer-B batch gap all alarm; watchdog cross-verification catches suppression.
- **A-3.** *Attacker floods logs to mask real events.* Structured search + rate-limited retention + tenant-scoped floors ensure audit is not drowned; log volume anomalies themselves alarm.
- **A-4.** *Insider changes redaction rules to leak secrets.* Redaction ruleset is a signed catalog (ARCH-22 will define config governance); changes ceremony-controlled; canary catches ineffective redaction.
- **A-5.** *Cloud compromise reads log storage.* Logs encrypted at rest (per NF-4.1); PII redacted before write; per-tenant DEK for structured contents.
- **A-6.** *Compromise NTP source → attacker rewrites time.* Multiple NTP sources; drift monitor; monotonic markers make ordering skew-detectable.

## 10. Operational Reliability Review (Reviewer 3)

- **O-1. Debug ergonomics under production.** Event-slug search + trace_id correlation + structured attributes → root cause locatable.
- **O-2. Retention cost management.** Tiered storage (hot → warm → cold) documented; per-tenant retention configurable.
- **O-3. Schema evolution.** Audit records schema-versioned; verifier CLI supports historical versions; deprecation cycle.
- **O-4. On-call fatigue.** Anomaly detection avoids constant alarms; verified-op events batched; issuance/denial/revoke/break-glass remain individual.
- **O-5. Sizing.** Documented per topology; audit volume vs. log volume ratio predictable.
- **O-6. Upgrade safety.** Verifier CLI ships with public key set; upgrade doesn't invalidate historical verification.
- **O-7. Air-gap operations.** Air-gap deployments produce audit locally; Layer-B batches optionally exported via secure media.

## 11. Self-Critique (Reviewer 4)

- **S-1.** *Retention defaults (30d logs, 90d metrics, 7d traces) are opinionated.* Some tenants will need longer traces for incidents. Made adaptive — retention configurable per tenant per class. Added to Rec Practices.
- **S-2.** *Verifier CLI is a large deliverable in itself.* Committed already (Q-12-1 → Rust, ARCH-14 transparency-log). Standard needs to ensure verifier CLI dev is not deprioritized against features. Added as a Security Metric (ARCH-24 will track).
- **S-3.** *Redaction-failure = drop is aggressive.* Alternative: retain redacted marker + flagged for review. But drop is fail-secure; keeping a bad-redaction record is worse than losing a log line. Standard keeps drop; added ARCH-19-EXCEPTION path for specific classes.
- **S-4.** *Monotonic seq per Kernel instance* — cross-instance ordering requires stitching (Q-in verifier CLI). Non-trivial; noted in verifier scope.
- **S-5.** *"Sensitive operation" list will grow.* Standard needs a governance rule for additions. Added: adding to the list is an ARCH-19 doc PR + two-approver.
- **S-6.** *"Audit is not lossy" claim depends on the Kernel-side backpressure being tested.* Added: mandatory failure-mode test — inject audit-substrate slowness → verify mediated operation fails-closed rather than silently succeeding without audit.

Revising in place: added the failure-mode test as a required CI check.

## 12. Attacker's First-Target Analysis and Redesign

**"If I were an experienced attacker, what part of this standard would I target first?"**

**The redaction pipeline.** Reason: same rationale as ARCH-15's first target, applied here — a redaction miss exposes secrets in *logs*, which have wide read access and long retention. A single missed pattern replicated across engines becomes a broad exposure.

**Redesign response.**

1. **Canonical redactor as a shared library.** All engines use the same redactor package (versioned, hash-pinned). No per-engine implementations to diverge.
2. **Redaction rule catalog is signed** (like Risk weight catalog, ARCH-11 §8) and loaded from Kernel-mediated config.
3. **Monthly canary** — synthetic secrets injected into ingest; if they surface in logs, alarm + incident + rule update.
4. **Failure ⇒ drop + alarm** (M-16).
5. **Log-consumer scanning.** Downstream log consumer (Telemetry Engine) runs a second redaction scan as belt-and-braces; hits alarm the emitting engine (and are re-redacted at the consumer).

**Second target after redesign.** Audit-substrate access credentials. Response: audit substrate accessible only by Audit Engine's SVID; access is Kernel-mediated; anomaly-detected; watchdog queries verify.

**Third target.** Time-integrity. Response: multi-source NTP; drift monitor; monotonic anchors; fail-secure on drift; ARCH-13 boot chain records time integrity as part of attestation.

## 13. Decisions

### D-20-1. Split audit vs. logs, separate substrates, separate keys

- **Advantages.** Audit is unhijackable via log compromise; distinct retention regimes; clarity.
- **Disadvantages.** Two systems to run.
- **Security Impact.** *Strongly positive.*
- **Performance Impact.** Neutral.
- **Operational Complexity.** *Moderate.*
- **Maintainability.** Positive.
- **Scalability.** Positive.
- **Alternatives.** *Unified pipe.* Rejected.
- **Reason.** D-04-3 (Asset Inventory decision).

### D-20-2. Mandatory `RedactionManifest` on every audit record

- **Advantages.** Proof of what was suppressed and why; enables independent verification of redaction claims.
- **Disadvantages.** Slightly larger records.
- **Security Impact.** Strongly positive.
- **Performance Impact.** Small.
- **Operational Complexity.** Moderate.
- **Maintainability.** Positive.
- **Scalability.** Positive.
- **Alternatives.** *No manifest.* Rejected — can't verify what was hidden.
- **Reason.** Transparency of redaction.

### D-20-3. Independent verifier CLI + published spec

- **Advantages.** Third-party verifiers possible; regulator can verify without AEGIS; long-term durability.
- **Disadvantages.** Spec maintenance overhead.
- **Security Impact.** *Strongly positive.*
- **Performance Impact.** N/A.
- **Operational Complexity.** Moderate.
- **Maintainability.** Positive.
- **Scalability.** Positive.
- **Alternatives.** *Verification only via AEGIS.* Rejected.
- **Reason.** Zero SPOT applied to verification.

## 14. Open Questions

- Q-20-1. Sampled trace policy per tenant tier. Resolved in ARCH-24.
- Q-20-2. Transparency-log witness cadence. Resolved in ARCH-22 / ARCH-24.
- Q-20-3. Long-horizon retention cost model. Resolved in ARCH-23.

## 15. Change Log

- **0.1 (2026-07-10)** — Initial draft after four-reviewer discipline.