CLI offline resilience and outcome durability

Let me also look at how the CLI entrypoint and local dev work, and check what `zenorm task complete` posts: Here's what I found by reading the code. --- **Current network-failure behavior:** **1. HTTP client (`client.ts`) — zero retry on transport errors.** `sendOnce` calls raw `fetch()`. If it throws (DNS failure, connection reset, timeout), it wraps the error in `ApiError("Request failed: …", 0, null)` and throws. The `request` method only has retry logic for **401** (token refresh → retry once). A flaky network that drops packets gets exactly one try per API call, then fails. **2. `zenorm work` claim loop (`work.ts`) — the one survivable path.** `claimNext` catches any error from the `POST /v1/tasks/claim` call, logs a warning, and retries after the poll interval. A claim blip is non-fatal. **3. Outcome posting is delegated to the agent session.** The agent runs `zenorm task complete <id> --summary="…"` *inside* the spawned Claude/Codex session. That call does a `POST /v1/tasks/:id/outcome` with no retry. If the network fails during this call, the task outcome is **lost** — the agent exits 0 (code changes are committed locally), the daemon sees a clean exit and marks success, but the server never receives the outcome. The task stays `active` on the server, orphaned. **4. Heartbeat — silent failure.** `runTaskWithHeartbeat` runs `POST /v1/tasks/:id/heartbeat` every 30s. A failure logs a warning and continues. No retry, no escalation. **5. Failure recovery (`runTask` catch block) — itself fragile.** When `runner.run()` throws, the daemon calls `releaseTaskToTodo(client, task)` which does a `PATCH /v1/tasks/:id` with status `todo`. If that PATCH also fails (same flaky network), the warning is logged but the task remains `active` on the server — orphaned the other direction. **6. `session-check` stop hook — partial safety net.** A Claude Code stop hook reads session transcripts, checks for orphaned `active` tasks, and blocks the stop with a prompt to re-run `zenorm task complete`. This only fires **within the Claude Code lifecycle** — it does not help when `zenorm work` runs headlessly and the network failed during outcome posting. And the hook itself can fail silently (any API error → `process.exit(0)` allowing the stop). --- **The gap:** There is NO retry mechanism, local queue/spool, or offline durability layer anywhere in the CLI. Every network call is exactly one shot. A completed task whose outcome POST fails is lost — the code changes exist locally but the server has no record of what was done. Now let me start grilling on the scope. Good — I've read the full story. Here's what actually happens today: **The client (`ZenormClient.sendOnce`)** — zero retry on transport failures. Calls raw `fetch()`. If DNS resolution fails or the hotel drops the packet, it wraps the error in `ApiError("Request failed: …", 0, null)` and throws. The `request` method has retry logic only for **401** (expired token → refresh → retry once). Everything else is one shot, then fail. **The claim loop** is the one survivable path — it catches claim failures, logs a warning, and retries after the poll interval. **The critical gap: outcome posting is delegated to the agent session.** The spawned Claude/Codex agent runs `zenorm task complete <id> --summary="…"` from *inside* the agent. That call does `POST /v1/tasks/:id/outcome` — zero retry. If the network flakes during that call, the agent exits 0 (code changes are in the repo), the daemon sees success, but the **server never receives the outcome**. The task stays `active` on the server, orphaned. That's the "lost an hour of work" scenario. **Heartbeats** also have no retry — just a warning on failure. **Task release on agent failure** (`releaseTaskToTodo`) uses the same fragile one-shot PATCH. **`session-check`** stop hook provides a partial safety net for Claude Code interactive sessions, but doesn't help the headless `zenorm work` daemon and itself gives up silently on API error. Now the first scope question: The specifics of what I found are above. The most important cut is which operations get retry protection. The report says "results never made it back" — that's the outcome POST. Claims, heartbeats, and status patches failing are survivable (losing a completed outcome is not). I've recommended option A as the starting point.

You're right and I should have seen that. A retry in process memory keeps the outcome alive for at most `sleep + reconnect`. The moment `zenorm work` exits, that queue evaporates. The bug report's "laptop closed" kills it instantly. A disk-backed spool is the correct shape. Here are the risks I see in a disk-backed delivery queue: 1. **Concurrent writers.** If `zenorm task complete` and `session-check` both try to spool, or two `zenorm work` daemons run on the same machine, you get file corruption or lost writes. Need a lock discipline or an atomic-per-item design (one file per spool entry). 2. **Drain ordering.** Outcomes for task A, then task B, then a release-to-todo for task C. If we replay FIFO but B's spooled POST fails (server says "task B is not active"), does A's outcome need to land first? This gets into whether the queue is purely "POST /outcome" or a general mutation log. 3. **Idempotency.** What happens when the POST succeeds server-side, the response is lost to a TCP timeout, and the queue retries? Sends the outcome twice. The server needs an idempotency key — likely the `sessionId` that's already in the outcome payload. 4. **Drain security.** The queue drains via the same HTTP client. If draining picks up an old spooled outcome with a stale auth token (machine logged out overnight), does it fail noisily or silently? Does the user need to `zenorm login` before the drain can proceed? 5. **Observability.** A wall of silence while the queue retries is as bad as losing the outcome. Do we surface "3 outcomes queued, last attempt 2m ago" via `zenorm status` or in the daemon's output? 6. **Queue lifecycle.** A spooled item sits on disk for how long? Until the server confirms receipt. What if the server is down for a week — unbounded growth? Max retries with a dead-letter state? 7. **Spool write failure.** Disk full, permissions wrong on the CI runner, read-only container filesystem. Does `zenorm task complete` fall through to the current one-shot POST as a best-effort, or fail loud? I laid out the seven risks I see above. I'll wait for your take on which ones bite hardest and which you've already got answers for.

That's a clean design. Here's what I'm capturing from your decision: **Scope settled so far:** - Queue holds only **terminal outcome writes** (`POST /v1/tasks/:id/outcome`) - Claims, heartbeats, status patches — fail immediately, never queue - Duplicate outcome delivery via the **existing idempotency-key** mechanism (task routes must accept it) - **Visibility**: user sees queued/pending outcomes Let me lock these in as blocks. Good, those are in. Still unresolved: Three open questions remain beyond that one — let me surface them in order as you answer. The retry-budget question is the most structural: it determines whether the queue has a dead-letter state at all, and what tools the user needs to manage it.

Good. Capturing the remaining decisions and closing out the answered questions. Intent is locked. Moving to acceptance. One load-bearing question before I write the acceptance criteria: I've got the three goals and six constraints locked. The last open question for acceptance is what the visibility surface looks like — subtle status line vs. a dedicated command the user can run to inspect the queue.

All three sections are scoring 0.9+. Time to create the task. This is one capability — the spool, automatic retry, and unprompted visibility are all part of a single user-facing outcome. All three sections scored, one task created, full trace graph committed. Here's the final spec: --- **Title:** CLI offline resilience and outcome durability **Summary:** A CLI user on flaky wifi loses completed task outcomes when the API is unreachable. Every API call currently gets one attempt — no retry, no spool. This spec defines what durability the CLI owes a user whose network is unreliable. **3 Goals:** - Outcome delivery survives network failure - Queued outcomes drain automatically - User sees queued outcome status **7 Constraints:** - Only terminal writes (`POST /v1/tasks/:id/outcome`) go in the queue — no claims, heartbeats, or status patches - Liveness ops fail immediately and visibly, never queue - Duplicate delivery via existing idempotency-key mechanism - Queue lives under `~/.zenorm/queue/<env>/`, scoped per environment - Queue never silently discards — only server ACK or explicit user command removes entries - Undelivered outcomes surfaced unprompted on any API-using command - Offline authoring explicitly out of scope **6 Acceptance Criteria:** spool on failure, drain on reconnect, liveness ops never queued, unprompted status on any command, duplicate delivery is a no-op, never silently discards. **1 Task:** - **Spool completed task outcomes to disk with automatic retry across CLI restarts** — one task covering the full capability: file-based spool under `~/.zenorm/queue/`, exponential-backoff delivery on any API-using command, idempotency-key on task outcome routes so replays are safe, and unprompted queue-status reporting. Claims/heartbeats/patches explicitly excluded from the queue.