fix-evaluation-satellite — WorksGood live mirror

Metadata

Status	done
Assigned	`agent-854`
Created	2026-07-25T14:34:58.695840950+00:00
Started	2026-07-26T07:35:35.327590932+00:00
Completed	2026-07-26T07:58:42.365513337+00:00
Tags	`bug`, `evaluation`, `lifecycle`
Tokens	16560569 in / 60096 out

Description

Objective

During the resolve-prophage-source incident, .evaluate-resolve-prophage-source kept being spawned and failing INSTANTLY with "task is open — can't evaluate", and those failures were CHARGED to resolve-prophage-source (driving its spawn_failures up). An evaluation satellite must never be scheduled against a task that is still open/incomplete, and a child eval failure must never be billed to a parent that is not itself in an eval-eligible state.

Relationship to existing work

fix-failedpendingeval-and (DONE) fixed the FailedPendingEval FLIP/eval RESPAWN STORM and children blocked by an UNRESOLVED parent. This is the adjacent facet: an eval scheduled against a task that is merely OPEN (not FailedPendingEval) and the failure-charge coupling. Reconcile/de-dupe with that fix; if its guard already covers the open-task case, close this as duplicate with evidence.

Work

Eval/FLIP scheduling must skip (or defer) tasks not in an eval-eligible terminal state; never spawn .evaluate-* for an open/incomplete task.
A child eval/FLIP failure must not increment the parent's spawn_failures / failure counters when the parent is not eval-eligible.
Resetting a task (or stripping stale .assign-*/eval meta) must reliably cancel any already-scheduled stale eval satellite (so a bulk-retry can't leave a zombie eval behind).

Validation

An open task never has a .evaluate-* satellite spawned against it; attempting it is a no-op or deferred, not a charged failure.
A child eval failure does not bill spawn_failures to a non-eval-eligible parent.
After a bulk retry/reset, no stale .evaluate-*/.assign-* satellite respawns against the reset task.
Reproduces the resolve-prophage-source scenario and shows no spurious failures charged to the parent.

Reference

Root-cause layer 2 from the resolve-prophage-source incident (stale .evaluate-* + .assign-* meta; "task is open — can't evaluate" loop).

## Objective
During the resolve-prophage-source incident, `.evaluate-resolve-prophage-source` kept being spawned and failing INSTANTLY with "task is open — can't evaluate", and those failures were CHARGED to `resolve-prophage-source` (driving its spawn_failures up). An evaluation satellite must never be scheduled against a task that is still open/incomplete, and a child eval failure must never be billed to a parent that is not itself in an eval-eligible state.

## Relationship to existing work
`fix-failedpendingeval-and` (DONE) fixed the FailedPendingEval FLIP/eval RESPAWN STORM and children blocked by an UNRESOLVED parent. This is the adjacent facet: an eval scheduled against a task that is merely OPEN (not FailedPendingEval) and the failure-charge coupling. Reconcile/de-dupe with that fix; if its guard already covers the open-task case, close this as duplicate with evidence.

## Work
1. Eval/FLIP scheduling must skip (or defer) tasks not in an eval-eligible terminal state; never spawn `.evaluate-*` for an open/incomplete task.
2. A child eval/FLIP failure must not increment the parent's spawn_failures / failure counters when the parent is not eval-eligible.
3. Resetting a task (or stripping stale `.assign-*`/eval meta) must reliably cancel any already-scheduled stale eval satellite (so a bulk-retry can't leave a zombie eval behind).

## Validation
- [ ] An open task never has a `.evaluate-*` satellite spawned against it; attempting it is a no-op or deferred, not a charged failure.
- [ ] A child eval failure does not bill spawn_failures to a non-eval-eligible parent.
- [ ] After a bulk retry/reset, no stale `.evaluate-*`/`.assign-*` satellite respawns against the reset task.
- [ ] Reproduces the resolve-prophage-source scenario and shows no spurious failures charged to the parent.

## Reference
Root-cause layer 2 from the resolve-prophage-source incident (stale `.evaluate-*` + `.assign-*` meta; "task is open — can't evaluate" loop).

Depends on

done .assign-fix-evaluation-satellite

Required by

✉ Messages 2 messages (2 unread)

#1user2026-07-26T07:32:02.291491996+00:00delivered

Emergency recovery mode: Pi Codex Sol/high, evaluation gates temporarily disabled. Finish/land the existing satellite scheduling/rearm fix without spawning downstream work. Do not re-enable evaluation settings.

#2fix-evaluation-satellite2026-07-26T07:35:44.503440021+00:00read

Acknowledged emergency recovery instruction — I will finish and land the existing satellite scheduling/rearm fix inline, without spawning downstream work or changing evaluation settings.

Log

2026-07-25T15:27:11.693227514+00:00 Task unclaimed: agent 'agent-755' (PID 1714601) process exited
2026-07-25T15:27:12.700076185+00:00 Minted evaluation pipeline evalp-3a5436033d6195db53cc3a98 for source attempt 2 (coordinator dead-agent retry)
2026-07-25T15:29:30.533551480+00:00 Spawned by coordinator --executor pi --model zai:glm-5.2 --isolation required-worktree
2026-07-25T15:29:57.446412660+00:00 Reconciliation: task recovered from orphaned state (was InProgress, agent: agent-758)
2026-07-25T15:29:57.446424020+00:00 Minted evaluation pipeline evalp-d7d4c9be417e05347b01131e for source attempt 3 (coordinator orphan reconciliation retry)
2026-07-25T15:29:58.054550617+00:00 Spawn failed (attempt 1/5): isolated worktree /home/bot/wg/.wg-worktrees/agent-755 is owned by live attempt agent-758 for task 'fix-evaluation-satellite' (status Working); no process launched. Recover by terminating/reaping that attempt or archive the worktree explicitly after inspection. exec_mode=default, executor=pi
2026-07-25T15:29:59.090214073+00:00 Spawn failed (attempt 2/5): isolated worktree /home/bot/wg/.wg-worktrees/agent-755 is owned by live attempt agent-758 for task 'fix-evaluation-satellite' (status Working); no process launched. Recover by terminating/reaping that attempt or archive the worktree explicitly after inspection. exec_mode=default, executor=pi
2026-07-25T15:30:04.177784657+00:00 Spawned by coordinator --executor pi --model zai:glm-5.2 --isolation required-worktree
2026-07-25T15:30:30.928273105+00:00 Reconciliation: task recovered from orphaned state (was InProgress, agent: agent-761)
2026-07-25T15:30:30.928288265+00:00 Minted evaluation pipeline evalp-f7b24ea34e30f56a4ea96bda for source attempt 4 (coordinator orphan reconciliation retry)
2026-07-25T15:30:46.442568524+00:00 Spawned by coordinator --executor pi --model zai:glm-5.2 --isolation required-worktree
2026-07-25T15:31:10.906405281+00:00 Reconciliation: task recovered from orphaned state (was InProgress, agent: agent-764)
2026-07-25T15:31:10.906419092+00:00 Minted evaluation pipeline evalp-56cec7c08db5c1515bc16893 for source attempt 5 (coordinator orphan reconciliation retry)
2026-07-25T15:31:16.368504419+00:00 Spawn failed (attempt 3/5): isolated worktree /home/bot/wg/.wg-worktrees/agent-755 is owned by live attempt agent-764 for task 'fix-evaluation-satellite' (status Working); no process launched. Recover by terminating/reaping that attempt or archive the worktree explicitly after inspection. exec_mode=default, executor=pi
2026-07-25T15:31:21.471537274+00:00 Spawned by coordinator --executor pi --model zai:glm-5.2 --isolation required-worktree
2026-07-25T15:31:45.956171322+00:00 Reconciliation: task recovered from orphaned state (was InProgress, agent: agent-767)
2026-07-25T15:31:45.956186792+00:00 Minted evaluation pipeline evalp-0b3b2734f6ac6ba3d7380890 for source attempt 6 (coordinator orphan reconciliation retry)
2026-07-25T15:31:46.720613338+00:00 Spawn failed (attempt 4/5): isolated worktree /home/bot/wg/.wg-worktrees/agent-755 is owned by live attempt agent-767 for task 'fix-evaluation-satellite' (status Working); no process launched. Recover by terminating/reaping that attempt or archive the worktree explicitly after inspection. exec_mode=default, executor=pi
2026-07-25T15:32:11.490310127+00:00 Spawned by coordinator --executor pi --model zai:glm-5.2 --isolation required-worktree
2026-07-25T15:32:36.121601021+00:00 Reconciliation: task recovered from orphaned state (was InProgress, agent: agent-771)
2026-07-25T15:32:36.121615541+00:00 Minted evaluation pipeline evalp-a82a3c3c7ed0199b229a9886 for source attempt 7 (coordinator orphan reconciliation retry)
2026-07-25T15:32:36.592145734+00:00 Spawn failed (attempt 5/5): isolated worktree /home/bot/wg/.wg-worktrees/agent-755 is owned by live attempt agent-771 for task 'fix-evaluation-satellite' (status Working); no process launched. Recover by terminating/reaping that attempt or archive the worktree explicitly after inspection. exec_mode=default, executor=pi
2026-07-25T15:32:36.592145734+00:00 Circuit breaker tripped: spawn failed 5 times. Last error: isolated worktree /home/bot/wg/.wg-worktrees/agent-755 is owned by live attempt agent-771 for task 'fix-evaluation-satellite' (status Working); no process launched. Recover by terminating/reaping that attempt or archive the worktree explicitly after inspection. exec_mode=default, executor=pi. Task marked incomplete for evaluator review.
2026-07-25T19:04:18.696284275+00:00 Task reset for retry from incomplete (attempt #7) — reason: spawn breaker tripped (spawn_failures=5) after orphaned-worktree/profile-invalid fallout; retrying on zai
2026-07-25T19:04:18.696324296+00:00 Minted evaluation pipeline evalp-965a387a31ae268222f94e01 for source attempt 8 (resume-in-place retry)
2026-07-25T19:21:15.377289192+00:00 Spawned by coordinator --executor pi --model zai:glm-5.2 --isolation required-worktree
2026-07-25T19:28:35.849532351+00:00 Reconciliation: task recovered from orphaned state (was InProgress, agent: agent-785)
2026-07-25T19:28:35.849581853+00:00 Minted evaluation pipeline evalp-25a4984664e093095764d1ba for source attempt 9 (coordinator orphan reconciliation retry)
2026-07-25T20:24:55.767655427+00:00 Spawned by coordinator --executor pi --model zai:glm-5.2 --isolation required-worktree
2026-07-25T20:41:58.858302192+00:00 Resumed prior WIP: query.rs Failed-bypass + coordinator eligibility gate + reset stale-satellite cancellation were all in place but uncommitted, and the smoke scenario had two helper bugs (enriched 'after' objects, missing spawn_failures in wg show --json). Fixed smoke helpers to read raw graph.jsonl; installed fresh wg binary; all 5 smoke checks + unit tests (query 92, reset 17, coordinator 79, eval_lifecycle 27) green; cargo fmt clean; no new clippy warnings.
2026-07-25T20:42:25.393505871+00:00 Committed: 56abb531 — pushed to remote (wg/agent-755/fix-evaluation-satellite)
2026-07-25T20:42:45.361360848+00:00 Task pending eval (agent reported done; awaiting `.evaluate-*` to score)
2026-07-25T20:44:33.199609440+00:00 Consumed durable verdict verdict-evalp-25a4984664e093095764d1ba-evaluate-f6e54ddb2a40fa7e exactly once: score=0.65, outcome=open
2026-07-25T20:44:33.199628350+00:00 Minted evaluation pipeline evalp-881a05c6e6ddb15d5d4cb394 for source attempt 10 (automatic in-place rescue after rejected evaluation)
2026-07-25T20:44:33.199630410+00:00 error[WG-EVAL-PIPELINE-REARM]: source attempt 10 could not atomically rearm its evaluation pipeline: .flip-fix-evaluation-satellite is still active on the previous source attempt; .evaluate-fix-evaluation-satellite is still active on the previous source attempt
2026-07-26T07:27:36.043647535+00:00 Task paused
2026-07-26T07:31:35.623126060+00:00 Task resumed
2026-07-26T07:31:38.209520097+00:00 Spawned by coordinator --executor pi --model openai-codex:gpt-5.6-sol --isolation required-worktree
2026-07-26T07:31:59.296433391+00:00 Resuming committed implementation 56abb531 from prior attempt; no decomposition because work is complete and only final validation/completion remains.
2026-07-26T07:35:20.865247814+00:00 Task unclaimed (was assigned to @agent-850)
2026-07-26T07:35:20.865301576+00:00 Minted evaluation pipeline evalp-6d306a8c8bc857bc8acf3788 for source attempt 11 (explicit unclaim redispatch)
2026-07-26T07:35:20.865317136+00:00 error[WG-EVAL-PIPELINE-REARM]: source attempt 11 could not atomically rearm its evaluation pipeline: .flip-fix-evaluation-satellite is still active on the previous source attempt; .evaluate-fix-evaluation-satellite is still active on the previous source attempt
2026-07-26T07:35:35.408054831+00:00 Spawned by coordinator --executor pi --model openai-codex:gpt-5.6-sol --isolation required-worktree
2026-07-26T07:35:44.551369947+00:00 Recovery resumed: inspecting prior WIP and completion state; no decomposition per emergency instruction.
2026-07-26T07:47:31.392138326+00:00 Validation checkpoint: cargo fmt --check and cargo build --locked passed. Full cargo test ran 2,970 passing tests; 7 unrelated profile::named tests failed from one shared-lock/home contamination cascade (first failure: dry-run profile file unexpectedly existed; remaining six PoisonError). Re-running the root test isolated, then task-specific suites/smoke and clippy.
2026-07-26T07:49:49.579117018+00:00 Validated fix: fmt/build pass; task-specific query/reset/coordinator tests pass; credential-free 5-step resolve-prophage-source smoke passes; cargo clippy completes with pre-existing warnings and no warning from changed code. Full suite reached 2,970 pass / 7 unrelated profile shared-state failures; isolated root failure passes. Commits: 56abb531 + 54329c7d, pushed.
2026-07-26T07:53:16.886933103+00:00 Installed validated branch globally with cargo install --path . --locked; evaluation settings were not changed or re-enabled. Final HEAD 54329c7d is pushed.
2026-07-26T07:53:26.973496946+00:00 Owned smoke gate passed. wg done merge-back found a conflict in src/commands/reset.rs because main advanced; resolving inline without downstream work.
2026-07-26T07:57:39.636707279+00:00 Resolved merge-back cleanly: main already carried implementation as ecb3398a; removed mechanically duplicated coordinator guard/tests introduced by merging equivalent histories. Post-merge build, exact satellite/reset unit tests, and 5-step smoke pass. Full integration test compile is currently blocked on main by unrelated tests/test_cron_serialization.rs missing new last_spawn_failure_at field.
2026-07-26T07:58:42.365515027+00:00 Task marked as done; scheduled evaluator is advisory evidence only (execution is not a quality pass)