---
name: ssp-csv-compile
description: Compile flat CSV structure manifests plus companion narrative markdown into a halo-action-pipeline actions.jsonl, with library-first payload resolution and fail-loud validation; project any action file back to plan-review.csv. Load whenever a user asks to "compile the SSP manifests", "generate actions from the spreadsheet", "give me a review CSV of this change plan", or authors/reviews SSP structure (components, capability-component mapping, ODPs, implementation statements) as spreadsheets before applying.
---

# ssp-csv-compile — Manifests are the review projection; JSONL is the source of truth

## References

| Reference | Type | Location | When to Use |
|---|---|---|---|
| `ssp-manifest-schemas` | File | `reference-docs/ssp-manifest-schemas.md` | Authoritative column contract for the four manifests + plan-review.csv — every enum, required field, AFUUID pattern, and library/off-library rule the compiler enforces |

Cross-reference in the body: `[[halo-ssp-asset-write-conventions]]` (the AFUUID, three-phase CF, and spine-order rules the compiler encodes), `[[halo-action-pipeline]]` (the executable `actions.jsonl` contract the compiler targets, unchanged), and `[[halo-asset-import-spreadsheet]]` (the spreadsheet-as-plan pattern this generalizes to the full SSP spine).

This skill hosts a self-contained, mirror-safe `scripts/` dir: `ssp_compile.py` (the deterministic compiler), `make_plan_review.py` (reverse projection), and the optional `make_review_xlsx.py` (hand-review workbook). The logic lives in those scripts. This document teaches the discipline behind them.

## What the manifest layer is

A full SSP's structure — component inventory, capability↔component mapping, ODP values, implementation-statement coverage — is naturally flat: hundreds of rows a reviewer wants to sort, filter, and diff in a spreadsheet. But the executable format is `[[halo-action-pipeline]]`'s `actions.jsonl`: nested args, `$ref` edges, multi-paragraph narratives — JSON blobs no spreadsheet can review.

The manifest layer resolves this tension: **flat CSV manifests are the human authoring/review projection; the JSONL stays the executable source of truth.** `ssp_compile.py` reads the manifests, validates them against the schema contract, and emits the `actions.jsonl` that `pipeline.py` runs via `/grcitsm-analyst:halo-apply`. The CSVs are regenerable and diffable; the JSONL is atomic-append and crash-safe-resumable. This is explicitly **not** a state-format swap and **not** a speed play — apply throughput is bounded by the Halo API and wave orchestration, never local file processing.

## The four manifests + companion markdown

Structural data goes in CSV cells; narratives never do. See `[[ssp-manifest-schemas]]` for the full column contract. In brief:

- **`components.csv`** — Component (asset type 173, `CMPN`). One row per component with `library_flag`, type, and short one-liner fields.
- **`cap-component-rel6.csv`** — Capability↔Component (rel 6). **Validated, not emitted:** rel 6 is owned by `reconcile_ssp_relationships_cap_component`, which derives the edge from each Component's `inventory_number` (`CMPN-<CFComponentAcronym>`) plus the library row's `CFComponentCapability`. The manifest still declares intent and its FKs are hard-checked. `--emit-rel6-fallback` restores explicit emission (one full-child-list `set_asset_relationships` per capability — rel 6 is full-replace on the parent, so per-pair emission would orphan its other children) for a tenant lacking that reconciler.
- **`odp-parameters.csv`** — Set Parameter / ODP (asset type 193, `SP`). One row per parameter value. `parent_afuuid` is **review/validation only** — the compiler confirms it resolves but emits no rel action (rel 11/12 are reconciler-owned).
- **`is-manifest.csv`** — Implementation Statement (asset type 191, `IS`). One row per (Component × control) with `library_flag`, `narrative_status`, and framework-CF attribution.

**Narratives live in AFUUID-keyed companion markdown, never a CSV cell.** The IS `AFImplementation Statement` payload, the SP long `AFParameter Value` prose, `AFValidation Process`, and the Site `system_information` blocks all resolve from companion `.md` files. A CSV cell holds at most a short `description` or `purpose`. Prose review needs prose format.

## Compile discipline

The compiler is deterministic and encodes `[[halo-ssp-asset-write-conventions]]` verbatim. Four rules govern emission:

**1. Spine emission order, reconciler steps excluded.** Actions emit in write-convention spine order:

1. `system_information` (from companion md, if present)
2. *(reconcile_ssp_assets — NOT emitted)*
3. Components: create (AFUUID guard) → `update_component_from_library` if library → update AFs → three-phase CF
4. rel 6 Capability↔Component — SKIPPED (reconciler-owned); `--emit-rel6-fallback` emits one full-child-list action per capability
5. Implementation Statements: create (guard) → update AFs (narrative from library | companion) → three-phase CF
6. rel 8 Component↔IS (one full-child-list action per component, derived from the IS AFUUID stem / `component_afuuid`)
7. Set Parameters: create (guard) → update AFs → three-phase CF *(no parent rel)*
8. *(reconcile_ssp_relationships_* — NOT emitted)*

**Compile emits only rel 8.** Reconciler-owned assets — Capabilities (178), Control Implementations (192), Implemented Requirements (190) — and links — Capability→Component (rel 6), CI→IR (rel 10), IR→IS (rel 7), IR→SP (rel 12), CI→SP (rel 11) — get **no** per-row actions. They are scaffolded by `reconcile_ssp_assets` / `reconcile_ssp_relationships_*`, which run separately. Hand-creating spine rows drifts off the catalog and re-introduces the duplicate-asset / missing-CF failures the reconcilers eliminate. `assert_no_reconciler_output` is a compile-time safety net: any emitted action targeting a reconciler-owned type or relationship is a compiler bug, not a warning.

**2. Library-first payload resolution.** Every Component and IS row carries `library_flag`. A `library` row instantiates a marketplace/library definition: it supplies the library key (`library_component_key` / `library_is_key`), and the compiler resolves the authoritative title, description, and (for IS) the narrative from the library-lookup CSV at compile time — the AFUUID uses the library acronym (`CMPN-<CFComponentAcronym>`, `IS-<CFISAcronym>`). An `off-library` row is tenant-specific: no library key, narrative from a companion file. Library-verbatim content is not review-worthy per row — the flag is — so it stays out of the manifest and the library remains the single source of framework CFs.

**3. Three-phase framework-CF write.** Framework Custom Fields land in three ordered transactions: (1) `CFBaseline` alone; (2) the **controls tier** — `CFAFedRAMPControls` / `CFAFedRAMPKSIs` / `CFAFedRAMPRules` / `CFACMMCControls`; (3) the **requirements tier** — `CFAFedRAMPRequirements` / `CFACMMCRequirements`, which must follow the controls tier because Halo filters the Requirements lookup by the asset's current Controls value. All six lists are covered (20x sites attribute by `CFAFedRAMPRules`). For a **library-flagged** row the six framework CFs and the narrative come library-verbatim from the library-lookup CSV, not the manifest; only `CFBaseline` (the instance's baseline envelope) is always manifest-authored. This ordering is a `[[halo-ssp-asset-write-conventions]]` requirement, not an optimization.

**4. AFUUID idempotency.** Every create carries an AFUUID guard (`{"search": afuuid, "kind": "asset"}`). An AFUUID already present in the tenant snapshot skips create and goes straight to update — re-applying a manifest updates changed rows rather than duplicating assets. This is what makes incremental recompiles safe.

## Narrative status + the review gate

The IS narrative is a collaborative surface — agents draft, humans edit, both iterate. Each `is-manifest.csv` row declares how its narrative resolves via `narrative_status`:

- **`library`** — compile pulls the narrative verbatim from the library lookup. No companion file, no `narrative_sha`.
- **`agent-draft`** / **`human-reviewed`** — the narrative reads from the AFUUID-keyed `companion_file`. Both require a `narrative_sha`: the content hash of the companion file captured at review time.

Transitioning a row off `library` means creating its companion file; from then on the compiler sources the narrative from the file, not the library. `ssp-brand-voice` applies to companion-file authoring — load it when drafting.

**The review gate is deterministic.** At compile, the live companion-file hash is compared against the stamped `narrative_sha`. A mismatch on any non-library row means the narrative changed since last review, and the batch routes through the `ssp-reviewer` CHANGE-PLAN chain. A pure-structure or all-library batch (every stamp matching) keeps the short path: manifest → compile → user approval → apply. This makes "changed since last review" machine-checkable instead of trust-based, honoring the MANDATORY SSP Review Chain for content mutations while removing ceremony from structural-only edits.

**Incremental recompiles (changed-only emit).** Against a **wide** tenant snapshot (manifest field columns present), the compiler classifies each existing row: if every field it would write matches the snapshot, the row is a **no-op** and its UPDATE + CF actions are skipped; if a field differs, that field is **drift** — reported (what will be overwritten), then the UPDATE is emitted (manifest wins). A **minimal** snapshot (`afuuid` + `asset_type` only) degrades gracefully to the old all-emit behavior. A row whose companion narrative changed (`narrative_sha` mismatch) is never treated as a no-op — it always emits and routes through the review gate. Fold-back — writing a tenant-adopted drift value back into the manifest CSV so "adopt-into-manifest" is actionable — is a deferred stretch; today the drift dispositions are **overwrite** (default, manifest wins) or **exclude** (`--exclude-file`, hold the AFUUID out of this run). Snapshot + library-lookup CSV column contracts: [[ssp-manifest-schemas]] §6–§7.

## Tenant-reads purity

`ssp_compile.py` is pure and offline: **it never calls MCP.** Every tenant-derived input arrives as a file the agent produced upstream (pattern: `correlation.py` — the agent runs the reads, the script does the math):

- **Snapshot CSV** (`--snapshot`) — existing tenant assets, keyed by AFUUID. Drives create-vs-update, rel-6 capability existence checks, SP parent resolution, orphan detection, and drift.
- **Library-lookup CSV** (`--library`) — resolved library payloads (`CF_lookup_implementation_statement_definition` results), keyed by `library_key`. Drives library-first resolution.
- **Field-map CSV** (`--field-map`) — `assettype_id,field_name,field_id`, generated from `asset_type_fields` per spine type. Drives `update_asset` payload emission: Asset Fields require the NUMERIC field id, which is tenant-specific — the map is how ids arrive per-tenant at run time (the tenant-ids-are-examples doctrine's mechanism), never hardcoded. An unmapped field name fails the compile loudly.

For **small targeted reads**, the agent step in `/grcitsm-analyst:ssp-compile` runs the read-only MCP lookups and writes these CSVs. For **bulk reads**, the user exports a CSV manually (e.g. a Halo report export) — no agent read fleet. This keeps the compiler testable offline and mirror-safe with zero tenant credentials.

## How to run

```
# 1. Validate only — fail loud with row-level errors, no output written.
#    --client-id/--site-id are REQUIRED whenever the compile emits creates:
#    check also validates the emitted actions (REQUIRED_ARGS gate), so an
#    under-specified create fails HERE, never in front of an executor.
python scripts/ssp_compile.py check --manifests ./manifests --companions ./companions \
    --snapshot ./snapshot.csv --library ./library.csv --client-id <client> --site-id <site>

# 2. Agent produces snapshot.csv + library.csv + field-map.csv from read-only MCP
#    lookups (field map: asset_type_fields per spine type; or user export)

# 3. Compile — manifests + companions → actions.jsonl (+ plan-review + drift/gate report).
#    --field-map is REQUIRED whenever updates touch Asset Fields: numeric field ids
#    are tenant-specific and arrive per-tenant at run time, never hardcoded.
python scripts/ssp_compile.py compile --manifests ./manifests --companions ./companions \
    --snapshot ./snapshot.csv --library ./library.csv --field-map ./field-map.csv \
    --out ./actions.jsonl --plan-review ./plan-review.csv --report ./report.json \
    --system-name ACME --client-id <client> --site-id <site>

# 4. Review plan-review.csv + report.json (no-op / drift / orphan / review-gate rows).
#    To hold a drifted row out of this run: list its AFUUID in exclude.txt and re-run step 3 with
#    --exclude-file ./exclude.txt  (default disposition is overwrite — manifest wins). On approval:
/grcitsm-analyst:halo-apply   # unchanged executor of the compiled action file
```

**Reverse projection** — any existing action file → a flat review CSV (one row per action):

```
python scripts/make_plan_review.py ./actions.jsonl -o ./plan-review.csv
```

**Optional xlsx** — a multi-tab hand-review workbook (openpyxl-gated). Plain CSV manifests remain the machine round-trip source of truth; the workbook is a read view only, edits go back through the CSVs:

```
python scripts/make_review_xlsx.py ./manifests ./review.xlsx --plan-review ./plan-review.csv
```

## Fail-loud vs gate

**Fail loud** — non-zero exit, **no partial `actions.jsonl`** — on any hard error:

- schema violation (unknown column, bad enum, missing required field);
- unknown AFUUID;
- dangling FK (rel-6 capability/component, IS `component_afuuid`, SP `parent_afuuid`);
- `library_flag=library` with no matching row in the library-lookup CSV.

These are contract breaks; there is no agent-improvisation fallback. Fix the manifest or the upstream reads and re-run.

**Gate, not fail** — the compile proceeds but flags a decision:

- `narrative_sha` mismatch on a non-library row → route the batch to the `ssp-reviewer` CHANGE-PLAN chain.
- Orphan (a manifest-owned asset in the snapshot with no manifest row) → a REPORT row, **never** a delete/retire action. Deletion is irreversible against a live tenant; the compile layer is incapable of emitting it. Retirement is a deliberate manual/ops action outside this pipeline.
- Drift on a manifest-owned row (a snapshot field differs from the manifest) → a `report.json` drift row naming the field + tenant value, so nothing is overwritten silently. Default disposition is **overwrite** (manifest wins, emitted); the user may **exclude** any AFUUID via `--exclude-file` to hold its UPDATE out of this run. *Adopt-into-manifest (writing the tenant value back into the CSV) needs fold-back — deferred.* The action file is not finalized until the drift rows are reviewed and approved.
