Assert Agent Story
assert-agent-story · always_apply: yes
Content
Mimir metadata: slug do-assert-agent-story; always_apply: false (link to BPE-02 / MIN-04 / TFK-02 / DTA-19).
When a plan, iteration manifest, or SAO §17 row declares Agent proof scenarios (PRF-*, agent_proof_script[], or agent_proof_command), you MUST prove orchestration with pytest integration tests using CAP-004 ScriptedLLM and the shared trace helper — not by asserting chat prose, grepping logs/app.log alone, or "the agent loop exists" checklists.
Requirements
- For each PRF row mapped in SAO §17, a test asserts run trace beats (workflow, tools offered/called, domain effects, blackboard/plan state as applicable) via
tests/support/agent_story.py→assert_agent_story(...). - Cover happy and reject paths when the PRF is marked adverse or has a reject sketch (
test_*_agent_story_happy/test_*_agent_story_reject). - Use ordered LLM scripts from
tests/fixtures/llm_scripts/<prf-id>/injected through the agent factory — only CAP-004 ScriptedLLM may stand in for the LLM in integration tests. - Agent-proof tests ship in the same commit as the CAP/worker behavior they prove — never a deferred "agent testing pass".
- Never assert on raw secrets, tokens, or passwords in trace or log lines.
- Do not mock domain services, tools, or brokers in
@agent_prooftests — real DB and real tool registry per Test Trophy.
Relationship to log story
- Log story (
do-assert-log-story): proves logging beats at decision points (caplog). - Agent story: proves control-plane behavior (trace + domain) under scripted LLM turns.
- Both may apply to the same slice; both must green in the same commit when declared.
Anti-patterns
- Behavior test green while PRF trace absent or weakened to "agent completed"
FakeLLMthat returns[]on exhaustion instead of raising (hides missing script turns)- Mocking ToolExecutor or domain services in
@agent_proofintegration tests - Skipping
checkpoint.agent_proof_commandwhen the manifest lists PRF proofs - Using live LLM in PR merge gate (belongs to lane 3/4 — see TFK-01 §6)
Details
- Slug:
assert-agent-story- Always apply:
- Yes
- Created:
- Aug 21, 2026
- Updated:
- Aug 21, 2026
Playbook
Activities 4
- Plan Feature Build Feature
- Implement Backend Build Feature
- TL Dispatches Workers to Build Manage Iteration
- Bootstrap Test Harness Test Automation Framework