study-long-lived

Study: long-lived supervisor hard-agent for graph health and auto-reset

Metadata

Statusdone
Assignedagent-740
Created2026-07-25T13:23:19.056075522+00:00
Started2026-07-25T14:08:29.460557366+00:00
Completed2026-07-25T14:17:18.171756997+00:00
Tagsresearch, design, daemon, lifecycle
Tokens3004091 in / 19022 out

Description

Objective

Study and design a long-lived, periodically-waking "supervisor" (hard) agent inside the WG daemon — a Casa-style persistent persona (see docs/design-casa-wgfed-adapter.md) rather than an ephemeral per-task worker. It wakes on a tick (every N minutes), scans the task graph for tasks failed/stuck "for dumb reasons," and safely resets/requeues them. It keeps its own persistent memory of activity so it does not flap and can recognize recurrence.

Research questions

  • Wake cadence: tick interval vs event-driven vs hybrid; avoid thundering herd / double-acting with the existing coordinator reaper and auto_rescue_on_eval_fail.
  • What counts as a "dumb failure" worth auto-resetting? Inventory the classes seen in this graph: failed-pending-eval / pending-eval (eval-pipeline exhaustion), agent-exit-nonzero crashes, dead-agent orphans, intentional-crash test tasks (e.g. storm-source), eval-satellite debris, respawn storms. For each: safe reset policy vs leave-for-human.
  • Memory model: what activity history the supervisor keeps, where (graph task? sidecar file?), how it dedupes/recalls, and how it avoids reset loops (max-attempts, backoff, escalation to human).
  • Composition with existing lifecycle: reaper, wg recover, auto_rescue_on_eval_fail, eval-lifecycle-reconcile. The supervisor must not fight them; define the boundary (what only the supervisor does).
  • Failure mode of the supervisor itself: what if it hangs or mis-resets? Liveness, bounded blast radius, dry-run / audit-first mode.
  • How detection results feed the rate-limit/parallelism controller (sibling studies).

Inputs

  • docs/design-casa-wgfed-adapter.md (long-lived persona pattern)
  • src/ (coordinator, reaper, recover, eval-lifecycle) — survey current health/recovery code
  • The hand-cleared incidents in this session (failed-pending-eval tar pits, storm-source loop) as worked examples

Deliverable

docs/studies/supervisor-hard-agent-design.md

Validation

  • Design doc committed at docs/studies/supervisor-hard-agent-design.md
  • Enumerates the dumb-failure classes present in this graph with a per-class reset/leave policy
  • Defines wake cadence, memory model, loop-prevention, and the boundary vs existing reaper/recover/eval-reconcile
  • Includes a fail-safe / dry-run / audit-first rollout plan and the supervisor's own failure handling
  • Cites the specific source files/functions it would touch or add

Depends on

Required by

Log