# halo-bulk-update — phase mechanics and operational detail

Deep operational detail for the six-phase bulk methodology. The phase map, gates, sizing heuristics, folder layout, and anti-patterns live in `skills/halo-bulk-update/SKILL.md`; platform write/read shape rules live in `reference-docs/halo-platform/write-semantics.md` and `reference-docs/halo-platform/api-mcp-semantics.md`.

## Phase 0 — the Array-Iteration runbook channel (mechanics)

For a **uniform mechanical write** — the same operation across every row, no per-row reasoning or payload composition — do NOT build an action file and dispatch an LLM executor fleet. Apply it server-side with a single **Array-Iteration runbook**, which runs entirely inside Halo with zero reasoning agents:

1. A **native-SQL step** (`auto_action:18`) SELECTs the target rows and binds the rowset into a named runbook variable (`runbook_variable_mappings: [{key: Targets, value: "<<response>>"}]`). Native SQL is read-only — it is the QUERY that drives the loop, never the write itself.
2. An **Array Iteration** (`auto_action:12` Start / `13` Next-End) loops `<<Targets>>` by name.
3. The loop **body is an Edit-Asset API Action** (`auto_action:8` + `auto_action_type:19`), parameterized per row via `<<iteration{StartId}^field!>>`.

```sql
-- iteration SOURCE (auto_action:18, read-only): the rows to fix
SELECT did FROM Device WHERE dtype IN (...) AND (dstatus=1 OR dstatus IS NULL);
-- loop body (auto_action:8 / auto_action_type:19 Edit Asset): set dstatus=12, dinactive=0 per row
```

One runbook execution applies the write to every row server-side: no agent fleet, no per-executor context bloat, no hang. Wrapping the same writes in dozens of reasoning executors is slower, more expensive, more fragile, and **can stall indefinitely** (see the hang failure mode in `[[halo-action-pipeline]]`): one real incident wrapped 2,068 identical column writes in 69-98 executors and froze silently at done count 108→111→flat.

The Edit-Asset bodies go through the API path, so they fire Halo "Asset Changed" events normally (CF rollups, derived-field runbooks) — correct for most writes; just account for rollup load at high row counts. Native SQL cannot write, so there is no raw-`UPDATE` runbook path that skips those events.

Reserve the action-file + executor fleet for writes that genuinely need agent-side judgment: a distinct, non-templatable payload per row, or a multi-tool sequence that can't be expressed as one iteration body. Authoring the iteration runbook itself is a runbook-creation task — see the Halo runbook-authoring doctrine for the Query→bind→iterate→body shapes.

## Phase 3 — approval-gate surface paths

Surface the plan + compact view to the user, then ask for approval. Two surface paths depending on context:

- **Main loop (Claude Code top-level)**: use `SendUserFile` to attach `plan.md` + `all-rows-compact.md`, then `AskUserQuestion` with the options below.
- **Subagent / skill context where `SendUserFile` is unavailable**: write both files to `outputs/<task-slug>/` (already done in Phase 2) and surface their absolute paths inline in the chat response, then `AskUserQuestion` with the options below. Tell the user "Plan: outputs/<task-slug>/plan.md | Compact: outputs/<task-slug>/all-rows-compact.md".

AskUserQuestion options:
- `approve` (recommended defaults on D1-Dn applied)
- `approve with [decisions]` (per-D overrides)
- `pause` / `cancel`

Approval can override individual decisions: bake the overrides into writer prompts.

## Phase 4 — pilot procedures

Main agent does ONE row manually to validate the full payload shape. Required for:
- New asset types or new field combinations
- Lookup-value uncertainty (validate-V scalar vs string)
- New relationship_id pairs
- **Any write to a lookup-backed framework CF** (CFAFedRAMPControls, CFAFedRAMPKSIs, CFACMMCControls, etc.) — silent drops happen when the value doesn't exist in the underlying control-catalog custom tables (CTControlDetails, CTControlBaselines, CTControlParameters, CTControlRequirements, CTBaselines — standard GRCITSM tenant schema; FedRAMP baselines use 2026 Class names — FedRAMP Rev5 Class A/B/C/D, FedRAMP 20x Class A/B/C)

Immediately `get_asset` to verify every field landed. **For framework-CF lookups, a silent drop shows as `display:""` with `id:"0"` unchanged on read-back** — the write didn't error but no row resolved. If you see this, the value isn't in the catalog at the expected baseline.

**Ticket-side framework-CF pilots need a DELAYED re-read (90s), not just an immediate one.** Event runbooks (`category-control-lookup`) fire asynchronously after the write returns — a loosely-conditioned event subscription can re-derive the framework CFs (six dimensions, including the derived-only `CFFedRAMPRules`) and silently revert the manual write within ~90 seconds while the write response looks correct. Rule: pilot ONE ticket → re-read after ≥90s → only then fan out. Never combine a category change and manual framework CFs in one payload — the category write retriggers auto-derive over the CFs. See `[[grcitsm]]` § KSI ticket organization.

**Always run the catalog-discovery procedure BEFORE the pilot when writing to a lookup-backed CF:**

```
baseline_catalog
  → confirms the baseline exists by Class name (FedRAMP Rev5 Class A/B/C/D,
    FedRAMP 20x Class A/B/C, CMMC L1/L2/L3, Drata) + baseline metadata
control_baselines_summary
  → confirms baseline membership + Row ID range
lookup_controls(control_id="X", baseline_name="Y")
  → empty result = value NOT in catalog → don't include in write
  → row returned = value IS in catalog → safe to write
  → (use lookup_control_details / lookup_control_parameters for canonical detail
     + per-class overrides)
lookup_requirements(requirement_id="X", baseline_name="Y")
  → same for sub-requirements
```

Catalog-scope value-format traps (KSI family stripping, zero-padding, enhancement format, Site.CFBaselines scope) are corpus-owned: `reference-docs/halo-platform/write-semantics.md` §3 and §5.2.

Log the pilot to `applied.jsonl` so resume logic skips it.

## Phase 5 — file-based checkpoint protocol (state/, mandatory)

Every bulk run maintains a `state/` folder; the FILES are the source of truth, never an agent's context. This makes work splittable, resumable, and hang-detectable.

```
outputs/<run>/state/
  rows-<step>.jsonl    # immutable work list (the plan rows; written ONCE by the planner)
  tasks.jsonl          # append-only checkpoint EVENT log (all writers share it, Bash >>)
```

Event schema — one line per (row × stage) transition:
```json
{"ts":"<iso>","step":"<step>","row":"<key|afuuid>","stage":"<stage>","status":"start|done|fail","ids":{"ticket_id":N},"owner":"<writer-name>","note":"<only on fail>"}
```

**Standard stages per entity:** ticket → `create, structure, assign, assets, cfs, status`; asset → `create, update1, update2, verify`; relationship batch → one stage per parent subject. A writer appends `start` BEFORE the first call of a stage and `done` (with returned ids) immediately AFTER it verifies — never batches checkpoints at the end.

**Slice sizing:** one writer claims ≤5 rows full-stack, or one stage across ≤15 rows. Small slices mean a dead agent loses minutes, not hours.

**Resume rule:** derive current state = last event per (row, stage). A row resumes at its first stage lacking `done`. A `start` without `done` is presumed dead after ~10 min — safe to retry every stage EXCEPT `create` blindly: before retrying a `create`, search Halo for the row's summary/AFUUID first (creates are the only non-idempotent stage).

**Orchestrator heartbeat:** poll `state/tasks.jsonl` growth (mtime/line count) every few minutes while writers run. Zero growth for ~10 min → kill the writer, reassign its unfinished rows to a fresh slice. Never wait on completion notifications alone — a hung agent is indistinguishable from a slow one, and writers can hang silently at tool-loading with zero writes.

**Append-only audit log:** every writer ALSO logs completed rows to the SHARED `applied.jsonl` via Bash `>>` (the run-level outcome ledger; `state/tasks.jsonl` is the fine-grained progress log). Schema:

```json
{"timestamp": "<iso>", "partition": "<name>", "afuuid": "<AFUUID>", "asset_id": <id>, "outcome": "created|updated|verified", "executor": "<writer-name>"}
```

**Resume logic:** if any writer halts, the next writer reads `applied.jsonl` first to skip already-done AFUUIDs.

## Staged-read discipline at bulk scale

The stage-to-disk rule itself (land bulk pulls in the run's data directory with a manifest checkpoint, query the file, never re-pull per row inside parallel workers) is corpus-owned: `reference-docs/halo-platform/api-mcp-semantics.md` §6. Bulk-run specifics:

```
outputs/<run>/data/                      # staged reads, parallel to state/
  tickets.json        # ticket_summary id set → looped get_ticket pull (query with jq/python, never whole-body Read)
  assets.json         # asset_summary id set → looped get_asset pull (active_inventory for Cloud/Hardware bulk)
  ir-current.jsonl    # current state of every target IR/IS/Component (one row per asset)
  manifest.json       # {source_tool, pulled_at, row_count, sha} per staged file — the read checkpoint
```

- **Pull once, to disk, up front.** Before analysis or a write run, fetch the whole target set ONCE and write it to `outputs/<run>/data/`. Record a `manifest.json` entry so a resumed run re-uses the staged pull instead of re-hitting Halo.
- The catalog has no full-detail bulk read — get the id set from a roll-up summary, then loop the point read: bulk asset = `asset_summary` (returns `<asset id>:<inventory_number>` pairs) → loop `get_asset`; bulk ticket = `ticket_summary` → loop `get_ticket`; `active_inventory` is the Cloud/Hardware bulk roll-up (it excludes the SSP spine, so it never returns Capability/Component/IR/IS/SP rows).
- **Merge-on-omit means you only pre-read for multi-select merges** (`reference-docs/halo-platform/write-semantics.md` §2.1). When pre-reads ARE needed at scale, stage them once: don't call `get_asset` inline per row inside parallel writers. Pull the current state of the full target set once (`asset_summary` id set → looped `get_asset` → `data/ir-current.jsonl`); each writer reads its row's current state from the staged file, merges the multi-selects, and writes. Fewer calls, resumable, no per-row context cost. (Per-write `get_asset` is still used for the ≤10% verification sample — that's a fresh read-back, a different purpose.)
- **Resume = re-use staged data.** A relaunched analysis or writer run reads `manifest.json`; if the staged file covers the needed rows and is fresh enough, skip the re-pull entirely.
- **Write responses split into two cases** (the read discipline above does not transfer uniformly). A **full-object** write response — the whole written entity echoed back — follows the same rule as a bulk read: confirm minimally, never pull the body into reasoning, targeted read-back if a written field is needed. A **slim-projection** response (`{id, inventory_number, client_id/name, site_id/name, status_id, inactive, written[]}` — `update_asset` v1.0.0+ and the two `*_from_library` tools) is a receipt, not a body: read it whole, assert the client/site NAMES as the tenant witness, read `inactive` as the duplicate-create soft-fail, and take written values from `written[]` instead of a read-back. Discriminate by the response the DEPLOYED tool returns, never by tool name — a tenant that has not imported the slim version still returns the full object. See `[[halo-bulk-update]]` anti-pattern #3.

## Phase 6 — batch sizing note

Even large batches land cleanly in one `set_asset_relationships` call: a ~190-row batch is a ~6 KB payload, and the response's full `child_assets[]` array stays well within the MCP relay token cap (~50-55K tokens).
