verify-spawn-recovery — WorksGood live mirror

Metadata

Status	done
Assigned	`agent-896`
Agent identity	`fbb2d89fe44d6b761bcb4b67748ab3d417f29df70a65f8d9403308f5bb2bab78`
Created	2026-07-26T17:58:15.410101377+00:00
Started	2026-07-26T18:09:01.142830158+00:00
Completed	2026-07-26T18:36:22.707399626+00:00
Tokens	12669273 in / 27084 out

Description

Objective

Independently review and harden emergency main commit 8419f909, which stopped the live spawn storm. The observer rejected this valid Linux checkout because it contains tracked case-distinct paths; the failed reservation audit then reused the same agent ID/idempotency key, leaving Open+assigned and causing endless rollback/reconcile/breaker loops across every ready task.

Required outcomes

Verify exact case-distinct paths are safely represented on filesystems that materialize both, while case-insensitive checkout/materialization failures remain explicit.
Verify a preparation failure followed by retry with the same uncommitted agent ID creates a new reservation/attempt event rather than replaying the rolled-back idempotency key.
Harden rollback so every pre-permit failure leaves Open, unassigned, no process/worktree/registry phantom, and one breaker-neutral actionable diagnostic; no coordinator tick can create a storm.
Add a permanent real-daemon regression: inject observer-baseline failure, repair cause, retry same allocation, spawn exactly once; multiple other ready tasks remain untouched.
Review cross-platform implications and retain exact-path manifest identity.

Validation

Review 8419f909 rather than replacing it blindly.
Focused case-path and reservation-reuse tests pass.
Real daemon scenario proves zero stale claims and zero repeated breaker charges.
cargo fmt --check, cargo clippy, focused suites, build, install, and owned smoke pass.

## Objective
Independently review and harden emergency main commit `8419f909`, which stopped the live spawn storm. The observer rejected this valid Linux checkout because it contains tracked case-distinct paths; the failed reservation audit then reused the same agent ID/idempotency key, leaving Open+assigned and causing endless rollback/reconcile/breaker loops across every ready task.

## Required outcomes
- Verify exact case-distinct paths are safely represented on filesystems that materialize both, while case-insensitive checkout/materialization failures remain explicit.
- Verify a preparation failure followed by retry with the same uncommitted agent ID creates a new reservation/attempt event rather than replaying the rolled-back idempotency key.
- Harden rollback so every pre-permit failure leaves Open, unassigned, no process/worktree/registry phantom, and one breaker-neutral actionable diagnostic; no coordinator tick can create a storm.
- Add a permanent real-daemon regression: inject observer-baseline failure, repair cause, retry same allocation, spawn exactly once; multiple other ready tasks remain untouched.
- Review cross-platform implications and retain exact-path manifest identity.

## Validation
- [ ] Review `8419f909` rather than replacing it blindly.
- [ ] Focused case-path and reservation-reuse tests pass.
- [ ] Real daemon scenario proves zero stale claims and zero repeated breaker charges.
- [ ] `cargo fmt --check`, `cargo clippy`, focused suites, build, install, and owned smoke pass.

Depends on

done amend-pi-watchdog

Required by

Log

2026-07-26T17:58:15.391018219+00:00 Task paused
2026-07-26T17:58:15.637341739+00:00 Task published
2026-07-26T18:09:01.460457632+00:00 Spawned by coordinator --executor pi --model openai-codex:gpt-5.6-sol --isolation required-worktree
2026-07-26T18:09:17.498984879+00:00 Starting review of emergency commit 8419f909 and spawn preparation recovery paths
2026-07-26T18:13:37.046657407+00:00 FANOUT_DECISION: stay inline — observer identity, spawn rollback classification, coordinator coalescing, and daemon regression are one tightly coupled recovery path sharing the same files/tests
2026-07-26T18:13:37.090822510+00:00 Reviewed 8419f909: exact-path fold removal and attempt-sequence idempotency direction are correct; hardening gaps are exact manifest assertions, explicit materialization mismatch, breaker-neutral pre-permit classification, and real-daemon regression
2026-07-26T18:26:56.069448843+00:00 Implemented hardening: exact case-path manifest assertions and explicit non-exact materialization hold; exact fenced rollback ownership; typed breaker-neutral pre-permit deferral coalescing; stop-current-tick storm containment; real-daemon fail/repair/same-agent retry smoke passes
2026-07-26T18:29:24.831295720+00:00 Full cargo test ran 3000 passing tests but current-main stale dispatch test failed deterministically; profile failures were poison cascade and pass alone. Published follow-up fix-external-worker-compat-test (non-overlapping src/dispatch/plan.rs).
2026-07-26T18:35:59.246329259+00:00 Committed: 5360bc4a — pushed to remote
2026-07-26T18:36:09.857427953+00:00 Validated task criteria: reviewed 8419f909; exact case-distinct baseline keys/mutations pass; same-agent rolled-back reservation appends attempt-0-2 with a distinct idempotency key; all injected pre-permit boundaries leave Open/unassigned with no process/worktree/output/registry/cache phantom
2026-07-26T18:36:09.910170163+00:00 Validated real daemon: observer baseline fails on tracked escaping symlink; repeated ticks coalesce one actionable breaker-neutral diagnostic and leave two ready siblings untouched; repairing checkout reuses agent-1 and launches Fake-Pi exactly once
2026-07-26T18:36:10.694126489+00:00 Validated cargo fmt --check, cargo clippy, cargo build, focused unit/integration suites, candidate real-daemon smoke, and cargo install --path . --locked pass. Full cargo test reached 3000 passes; one deterministic pre-existing stale handler-compat assertion fails alone, tracked by published fix-external-worker-compat-test; seven profile failures were mutex-poison cascade and the root profile test passes alone.
2026-07-26T18:36:22.707399936+00:00 Task marked as done