Construction - Junior Edda

Description

PIN → MIN → BPE until done

Activities in this Phase

Order	Name	Actions
Build Feature
#1	Plan Feature	Read the 'BPE-1 Plan Feature' activity from playbook 'Junior Edda' (workflow 'Build Feature'). Follow its guidance when assisting me. ## Purpose Plan a new feature implementation by analyzing requirements, assessing codebase state, creating detailed implementation plan, and preparing for execution. ## Prerequisites - Feature specification or idea for what needs to be built - Access to codebase and documentation ## Steps ### Step 0: Reset and Prepare Reset your task plan. If there was work in progress on another task, create a TODO.md file documenting what was being worked on, what was completed, what remains to be done, and any blockers or notes. ### Step 1: Understand Architecture Read `docs/architecture/SAO.md` (if exists) to identify architectural patterns and principles relevant to this implementation. Note which section headings govern this feature — you will need these in Step 6. ### Step 2: Understand User Journey Check `docs/features/user_journey.md` (if exists) to understand what you're building and how it integrates with other parts of the system. ### Step 3: Analyze Feature Specification Read the feature specification thoroughly. If none exists, propose creating one first. Follow BDD/Gherkin format: one scenario = one user goal; 5–10 steps max; GIVEN-WHEN-THEN structure; independently executable. Step phrases must reuse patterns from `docs/features/CATALOG.md` or explicitly add a new catalogued step via TFK-07. ### Step 4: Assess Codebase State a) Identify reusable components: list components/services/models that can be reused; verify they're not stubs. b) Check test coverage: for any reusable components without tests, add test creation to the plan. c) Build context map: identify 3–5 `file:line_range` locations an implementor must read. Format: ``` \| file \| lines \| one-line note \| ``` If you cannot identify 3 meaningful references, the codebase assessment in step (a) is incomplete. d) Detect MCP / Agent surface: - Check SAO §18 — is MCP in scope for this feature? If yes: locate `mcp/server.py` (or equivalent). Verify `initialize_mcp()` exists and is called on startup. If missing: add initialization to the plan before any tool registration step. - Check SAO §17 — does this feature add or extend agent capabilities? If yes: locate `ToolExecutor` (artifact 56 §4.3). Verify it is instantiated and injected into the agent loop. If missing: add ToolExecutor initialization to the plan before service wiring. - Record findings as additional one-liners in the Context Map: `mcp/server.py \| <line> \| FastMCP singleton — register new tools here` `llm/executor.py \| <line> \| ToolExecutor — wire service callables here` Skip a row if that surface is not in scope. ### Step 5: Clarification Questions Ask clarification questions scenario-by-scenario. If >5 questions total, create `FEAT_X.Y.Z_Clarifications.md` for batch answers. ### Step 6: Create Implementation Plan Mandatory Section A — Context Map (from Step 4c) Mandatory Section B — Do-Not-Do List (derived from SAO.md sections) Mandatory Section C — SAO.md Sections That Apply Mandatory Section D — Tests to Create (each row must state what it asserts). When Section E has rows, also list `test__log_story_happy` and/or `test__log_story_reject` (pytest + caplog; integration or view level). When Section G has rows (SAO §17 in scope), also list `test__agent_story_happy` and/or `test__agent_story_reject` (`@pytest.mark.agent_proof`; ScriptedLLM + `assert_agent_story`). Mandatory Section E — Log Story Script (formerly Logs to Emit). Each row is a beat the implementor must emit and prove with caplog. Never defer logging to a final slice. ``` ## Log Story Script \| Where \| Beat \| Trigger \| Must include \| \|-------\|------\|---------\|--------------\| \| `LoginView.post` \| entry \| method called \| email= (never password) \| \| `LoginView.post` \| branch \| bad credentials \| authentication failed \| \| `LoginView.post` \| exit \| success \| login success, user_pk= \| ``` Beats vocabulary: `entry \| config \| validation \| processing \| branch \| exit \| error`. Do not cite dropped rule `add-logging`. Mandatory Section F — MCP Tools to Expose (skip with "Not applicable" if MCP not in scope): ``` ## MCP Tools to Expose \| Tool name \| Service method \| Write? \| HITL? \| Auth injection \| \|-----------\|---------------\|--------\|-------\|----------------\| \| `create_foo` \| `FooService.create` \| Yes \| No \| server-side user_id \| ``` Rules: artifact 57 §3.2 (tool schema), §3.3 (auth injection — never accept user_id from args), §3.4 (write-tool policy) Mandatory Section G — Agent proofs (skip with "Not applicable" if SAO §17 is N/A or this feature does not touch agent capabilities): ``` ## Agent Proof Table \| PRF ID \| Workflow \| Script path \| Trace beats to assert \| Adverse? \| \|--------\|----------\|-------------\|----------------------\|----------\| \| PRF-SC01-04 \| munin_chat \| tests/fixtures/llm_scripts/PRF-SC01-04/happy.json \| tools_blocked: list_entities \| yes \| ``` Rules: - Map only PRF rows from SAO §17 / DTA-19 §13 that this feature affects. - Script paths under `tests/fixtures/llm_scripts/<prf-id>/` (`happy.json`, `reject_.json`). - Section D must name matching `test__agent_story_` tests for every selected PRF row. - TASK- golden tasks (lane 4) belong in SAO §17 / nightly eval — list in plan only when this slice introduces or changes a golden task fixture. - Reference: artifact 56 Part 4.5, skill Agent Integration Proof Patterns, rule `do-assert-agent-story`. Implementation Steps: 1. Backend Implementation: models, admin, services, views, URLs. Follow the Rules section below. 6. MCP Tool Registration (if SAO §18 in scope): - Declare the tool in `mcp/tools/` wrapping the service method; apply server-side auth injection. - Register with `initialize_mcp()`. - Apply stdout hygiene if stdio transport (artifact 57 §6). - Reference: artifact 57 §3–§4 7. ToolExecutor Wiring (if SAO §17 in scope): - Register the service callable in `ToolExecutor` under the tool name expected by the agent loop. - Ensure the return envelope matches the stable format. - Mark as HITL if destructive. - Reference: artifact 56 §4.3 2. Frontend Implementation: Django templates, HTMX, partials, `data-testid` attributes. 3. Testing — prove what's working: behavior tests red → green → refactor. Unit, integration (no mocks), view tests. Then log-story tests red → green in the same slice (caplog / `assert_log_story`). When Section G has rows: agent-proof tests red → green in the same slice (`@agent_proof` / `assert_agent_story` / CAP-004 ScriptedLLM). MCP tools (if Section F populated): Tests to Create table must include T1 + T2 per new tool; T3 required if stdio: - T1 (FastMCP Client): `async with Client(transport=mcp) as c: await c.call_tool(...)` — proves schema, registration - T2 (Direct service): call service with real DB — proves service/ORM wiring - T3 (subprocess JSON-RPC): proves entrypoint cleanliness (no stdout noise) - Full recipes: artifact 57 §7 4. Observability: emit INFO at decision points matching the Log Story Script (who/what/when/why, never raw secrets). Prove with caplog tests — ban deferred "informative logging pass" slices. 5. Commit Strategy: Angular convention after every principal step. Behavior + log-story (+ agent-proof when Section G populated) green in the same commit. When GitHub issue #N will hand off this plan: list one commit per implementation slice in the plan; first commit is plan/docs handoff only (`docs(plan): … Refs #N`); slice commits use `Refs #N`; final slice uses `Closes #N`. Each slice requires `gh issue comment N` with commit SHA (rule `do-github-issues`). ### Steps 7–10: Rule Confirmation, No Time Estimates, Submit for Approval, GitHub Issue Issue body must contain all seven mandatory sections inline (A–G, not linked). Each section must be self-sufficient for a cold-start implementor. When used under PIN, note that `checkpoint.log_story_command` and `checkpoint.agent_proof_command` (when Section G populated) must pass alongside the behavior checkpoint. ### Step 7: Write Lessons Learned Before creating the GitHub issue, add a `## Lessons Learned` section to the issue body. The question is open-ended: Completing this story — what difficulties did you encounter? How could the task definition, tools, and artifacts you received have been improved to make accomplishing this task easier, without creating extra duplicates or contradictions elsewhere? This is not a finite list of categories. Write freely. If something matters, write it down. Areas that often surface signal (use them if they apply, ignore if they don't): - Missing or unclear artifacts (mockups, SAO sections, feature files, skills) - Activity steps that contradicted each other, the feature spec, or `user_journey.md` - Technical decisions made during planning not yet in `docs/architecture/SAO.md` - Deviations from the written BDD scenarios (scope narrowed, step reworded, constraint added) If nothing to note: `## Lessons Learned — None.` One honest sentence beats four empty bullet points. The reader is MIN-05 closing the iteration and PIN-02 opening the next one. ### Step 8: GitHub Issue Handoff (hard gate — blocks BPE-02 / MIN-04 code) After user approves the plan and before any implementation slice (including spec/Gherkin): 1. Create GitHub issue #N — body inlines sections A–G + `## Lessons Learned` placeholder + checkpoint YAML when used under PIN. 2. Update plan INDEX with issue #N and plan link. 3. Commit docs/plan only: `docs(plan): {wave} plan and GitHub issue #N` with footer `Refs #N`. 4. Post issue comment citing handoff commit SHA and listing slice order from the plan. Do not modify `src/`, `tests/` implementation, or production templates until step 3 is committed and step 4 is posted. ## Rules Before planning and writing the implementation plan, read each Rule below in this playbook (by slug), then apply it when filling Mandatory Sections D–G and the Implementation Steps. Do not rely on memory of the rule text. Required: - `do-plan-before-doing` - `do-skeletons-first` - `do-test-first` - `do-not-mock-in-integration-tests` - `do-informative-logging` - `do-assert-log-story` - `do-assert-agent-story` (when SAO §17 in scope) - `do-write-concise-methods` - `do-docstring-format` - `do-semantic-versioning-on-ui-elements` - `do-follow-commit-convention` - `do-small-increments` - `pytest` - `do-github-issues` Activity-specific (not a substitute for the rules above): - Mandatory Section E is a Log Story Script (Where / Beat / Trigger / Must include); Section D must name matching `_log_story_` tests when Section E is non-empty. - Mandatory Section G is an Agent Proof Table when SAO §17 applies; Section D must name matching `_agent_story_` tests when Section G is non-empty. - Logging and agent proofs ship in the same green slice as behavior — ban deferred logging or agent-testing passes. ## Success Criteria - Feature specification exists and is clear - Codebase assessment complete with context map (3–5 file:line_range references) - Plan contains all seven mandatory sections: Context Map, Do-Not-Do, SAO Sections, Tests to Create, Log Story Script, MCP Tools to Expose, Agent proofs (or explicit N/A for G) - All tests explicitly listed with what they prove, including `_log_story_` when Section E has rows and `_agent_story_` when Section G has rows - All log decision points listed with Beat + required context fields - When SAO §17 in scope: PRF rows mapped with script paths and trace beats; Section D lists agent_story tests - If MCP tools added: Tests to Create table contains T1+T2 rows per new tool (T3 if stdio) - Plan reviewed and approved by user - GitHub issue created/updated with all mandatory sections inline - `## Lessons Learned` section present in the issue body - Handoff commit `docs(plan): … Refs #N` exists before any implementation commit for this feature - Plan lists slice commit boundaries; issue #N created with A–G inline before BPE-02
#2	Implement Backend	Read the 'BPE-2 Implement Backend' activity from playbook 'Junior Edda' (workflow 'Build Feature'). Follow its guidance when assisting me. ## Purpose Implement backend services, models, and views following test-first development and small increments approach. ## Steps ### 1. Create Skeletons Create all class and method skeletons with full docstrings following your language's documentation conventions. The core principle: the developer who has no knowledge of the system can implement methods/properties etc. following only documentation in the skeleton. - Create class and method/function stubs and document them - Include full docstrings, return types, and sample return values - Use appropriate placeholder (e.g., `NotImplementedError`, `TODO`, `panic!`, etc.) - Do not skip type hints or documentation - Add comments inside the methods pointing attention to the logic flow, exception handling, logging etc. ### 2. Write Behavior Tests Before Logic Write unit tests before writing method logic using your test framework. Use real dependencies in integration scenarios - no mocking. ### 3. Implement Incrementally (behavior green) Work method-by-method. Each method/property should be: implemented → behavior-tested → log-story tested (step 4) → agent-proof tested (step 4b when declared) → committed. ### 4. Log Story — Caplog in the Same Slice For each footprint method covered by the plan's Log Story Script: 1. Write `test__log_story_happy` and/or `test__log_story_reject` (red) 2. Emit INFO logs that satisfy story beats (`entry → config → validation → processing → branch → exit → error`) per `do-informative-logging` 3. Make log-story tests green using skill Pytest Log Story Assertions (`tests/support/log_story.py` → `assert_log_story`) once TFK-02 has bootstrapped the helper — per `do-assert-log-story` 4. Never defer logging to a later "informative logging pass" slice ### 4b. Agent Proof — Same Slice (when SAO §17 / plan Section G in scope) For each PRF row in the plan's Agent Proof Table: 1. Write `test__agent_story_happy` and/or `test__agent_story_reject` marked `@pytest.mark.agent_proof` (red) 2. Queue ordered LLM scripts from `tests/fixtures/llm_scripts/<prf-id>/` via CAP-004 ScriptedLLM — only the LLM may be scripted; real DB, real tool registry 3. Make agent-story tests green using skill Agent Integration Proof Patterns (`tests/support/agent_story.py` → `assert_agent_story(...)`) per `do-assert-agent-story` 4. Assert control-plane trace beats (workflow, tools offered/called, domain effects, blackboard/plan state) — not assistant prose 5. Never defer agent proofs to a later "agent testing pass" slice ### 5. Commit After Each Step Write → run → test (behavior + log story + agent proof when declared) → evaluate → fix. Commit using Angular-style commit messages. Behavior, log-story, and agent-proof green in the same commit when declared. Issue #N (when active): footer `Refs #N` on every slice commit; immediately `gh issue comment N` with `` `{short-sha}` ``, slice summary, next slice. Do not start the next plan slice until both are done. See rule `do-github-issues`. ### 6. Backend Architecture - Services Layer: Business logic shared between different interfaces (API, Web UI, CLI, etc.) - Repository Pattern: Data access abstraction (can be swapped) - Views/Controllers: Return appropriate responses for your framework - API Endpoints: Follow RESTful or your framework's conventions - Context/State: Always validate and document ### 7. Route Registration Register new routes/endpoints with descriptive names. Follow your framework's conventions for URL/route structure. ### 8. Testing Views/Controllers Use your framework's test client. Test responses, validate context/state, check templates/views used, test dynamic endpoints. Include caplog assertions for Log Story Script rows on those views. ### 9. Scan Skills Query Playbook Skills where `capability_domain` in: - `BACKEND_FRAMEWORK` — Backend implementation patterns for your tech stack - `TEST_FRAMEWORK` — Testing patterns and best practices - `LOGGING_PATTERN` — Logging implementation for your language - `LOG_STORY_TESTING` — Pytest Log Story Assertions / caplog helpers - `AGENT_PROOF_TESTING` — Agent Integration Proof Patterns / agent_story helpers - `DOCSTRING_FORMAT` — Documentation format for your language Apply reference implementations and patterns from matched Skills. ## Rules Before implementing, read each Rule below in this playbook (by slug), then apply it to every change in this activity's footprint. Do not rely on memory of the rule text; do not paraphrase the rule body into this activity. Required: - `do-skeletons-first` - `do-test-first` - `do-not-mock-in-integration-tests` - `do-informative-logging` - `do-assert-log-story` - `do-assert-agent-story` (when plan Section G / SAO §17 PRF rows in scope) - `do-import-on-module-level` - `do-write-concise-methods` - `do-docstring-format` - `do-follow-commit-convention` - `do-small-increments` - `pytest` - `do-github-issues` Activity-specific (not a substitute for the rules above): - Log-story tests (`_log_story_` / caplog) ship in the same red→green slice as behavior — no deferred logging pass. - Agent-proof tests (`_agent_story_` / `@agent_proof`) ship in the same red→green slice as behavior when Section G declares PRF rows — no deferred agent-testing pass. ## Success Criteria - All skeletons created with full documentation - Behavior tests written before implementation and passing - Log Story Script rows proven by passing `_log_story_` caplog tests in the same commit as behavior - Agent Proof Table rows proven by passing `_agent_story_` tests in the same commit as behavior (when Section G populated) - No deferred logging or agent-proof slice in the plan - Code committed with proper messages - Every slice commit for issue #N referenced in a GitHub issue comment with SHA - Routes/endpoints properly registered - Services layer properly structured
#3	Implement Frontend	Read the 'BPE-3 Implement Frontend' activity from playbook 'Junior Edda' (workflow 'Build Feature'). Follow its guidance when assisting me. ## Purpose Implement frontend templates and interactions following your framework's patterns with appropriate rendering approach. ## Prerequisites - Backend implementation completed with passing tests - Routing defined - Review UX/design guidelines to identify sections applicable to the page/component ## Steps ### 1. Review Routing and Template Structure Check routing patterns - does anything need to be added or changed? Plan template/component hierarchy (base templates, partials, pages/components). Identify dynamic interactions needed. ### 2. Implement Templates/Components A. Page Templates/Components: Full pages (inherit from base or layout) B. Partials/Components: Reusable fragments for dynamic updates C. Dynamic Attributes: Use your framework's patterns for interactions D. Forms: Forms with appropriate validation and submission handling E. Visualizations: Embed visualizations as appropriate for your stack Add test IDs (`data-testid` or equivalent) to all interactive elements. Work in small increments. ### 3. Interaction Patterns - Navigation: Follow your framework's navigation patterns - Forms: Handle submission with validation feedback - Dynamic Lists: Refresh/update lists dynamically - Modals/Dialogs: Implement dialogs with dynamic content loading Add minimal client-side code only for: * Framework-specific enhancements * Client-side validation enhancement * Interactive effects (tooltips, hover, etc.) ### 4. Development Testing Start your development server Test in browser Check console for errors Verify forms submit correctly Test visualizations render properly Validate dynamic updates work as expected ### 5. Add Semantic Test IDs Add `data-testid` attributes using kebab-case. Use semantic names that describe purpose, not structure. Required on: buttons, links, form inputs, containers, error messages. ### 6. Styling Read `docs/ux/IA_guidelines.md` to identify what applies to the page/component and apply them. Use simple CSS, keep styles in static/css/, use semantic HTML elements, ensure responsive design. ### 7. Commit Changes Commit frontend changes using Angular-style commit messages. ### 8. Integration Validation Verify all views/controllers return correct templates/components, check dynamic interactions update correct elements, ensure forms handle validation errors properly, test visualizations render correctly, validate test IDs are present for testing. ### 9. Scan Skills Query Playbook Skills where `capability_domain` in: - `FRONTEND_FRAMEWORK` — Frontend implementation patterns for your tech stack - `UI_TESTING` — UI testing patterns and semantic naming conventions Apply reference implementations and patterns from matched Skills. ## Rules to Follow ### I. Semantic Naming for UI Elements Test Selector Hierarchy: 1. Test IDs (`data-testid` or framework equivalent) - Primary choice for all components 2. Semantic attributes (`aria-label`, role) - Secondary for semantic elements 3. Label association - For form inputs 4. Text content - Only for static, stable content 5. Avoid CSS selectors or XPath when possible Every interactive element MUST have: - Proper test ID attribute - `aria-label` or descriptive text for buttons - Associated label elements for form inputs - Unique accessible names for dialogs Hierarchical Test ID Naming: - Page-Level: `{domain}-page` - Feature Components: `{domain}-card` - Action Elements: `{action}-{domain}-button` - Form Elements: `{domain}-{field}-input` - Status/Feedback: `{type}-message` ### II. Small Increments Work in small vertical slices. After every change: write → run → test → evaluate → fix. No large PRs or 1000-line commits. ### III. UI Design Rules Action Buttons and Links: - Every action button/link should have an appropriate icon - If no icon specified in requirements, propose 3 icon options for user to choose - Icons should be semantically appropriate for the action Tooltips/Help Text: - Every action button should have helpful tooltip or help text - Active buttons: Explain clearly what will happen when clicked - Disabled buttons: Explain why it's disabled and what needs to be done to enable it - Use your framework's tooltip/help text patterns - Keep text concise but informative ### IV. Validation Checklist Before committing any component, verify all interactive elements have proper attributes and accessibility features. ## Success Criteria - All templates/components implemented with semantic markup - Dynamic interactions working as expected - All interactive elements have test ID attributes - Forms handle validation errors properly - Responsive design verified - Code committed with proper messages ## Inputs Read these before starting this activity. They are produced earlier in the playbook and are authoritative — raise a drift event instead of deviating. - User Journey (Document, Required) — produced by Define User Journey (#36). - IA Guidelines (Document, Required) — produced by Define Information Architecture (#37). - Screen Flow / Dialogue Map (Diagram, Required) — produced by Create Dialogue Maps (#38). - Feature Files (Document, Required) — produced by Write Feature Files (#39). - HTML Mockups (Code, Optional) — produced by Create Mockups (#40). - System Architecture Overview Template (Template, Required) — produced by Write SAO.md (#59). - Implementation Plan Template (Template, Required) — produced by Plan Feature (#96). - Definition of Done Checklist Template (Template, Required) — produced by Check Definition of Done (#101).
#4	Implement Feature Acceptance Tests	Read the 'BPE-4 Implement Feature Acceptance Tests' activity from playbook 'Junior Edda' (workflow 'Build Feature'). Follow its guidance when assisting me. APPEND TO GUIDANCE: --- ## TFK Integration: Behave-Based Acceptance Tests Feature acceptance tests are executed via behave-django against `.feature` files in `docs/features/` (spec + AT runner — single source of truth). There is no separate `tests/acceptance/` tree and no promote/copy step. ### Updated Process 1. Locate spec: Find the feature's BDD scenarios in `docs/features/act-/`. 2. Verify steps exist: Check that all steps used in the spec have implementations in `docs/features/steps/`. If any are missing, invoke TFK-07. 3. Remove `@wip` when ready: Tag unimplemented scenarios `@wip` during ESM; remove the tag when implementation is complete so CI runs them (`behave.ini` uses `tags = ~@wip`). 4. Prepare fixtures: Check the Fixture Library Catalog for needed test data. If presets are missing, invoke TFK-07. 5. Run acceptance tests: `make test-at` (runs `docs/features/` via `behave.ini`). 6. The dual approach remains: Django test client unit/integration tests remain valuable for fast feedback. Behave AT runs the same scenarios through the BDD layer. Both must pass. ### Key Principle Acceptance tests are the real thing minus the GUI (Django test client — not Playwright). No mocking. The `.feature` file in `docs/features/` IS the grounding executable requirements document for AT. ## Skills (additional) - Behave-Django BDD Runner* (BDD_RUNNER) ## Inputs (additional) - Step Library Catalog (Document, Required) — produced by TFK-03. - Fixture Library Catalog (Document, Required) — produced by TFK-04. - Behave Configuration (Code, Required) — produced by TFK-02 (Artifact #51). --- ## TFK Integration: Behave-Based Acceptance Tests Feature acceptance tests are executed via behave-django against `.feature` files in `docs/features/` (spec + AT runner — single source of truth). There is no separate `tests/acceptance/` tree and no promote/copy step. ### Updated Process 1. Locate spec: Find the feature's BDD scenarios in `docs/features/act-/`. 2. Verify steps exist: Check that all steps used in the spec have implementations in `docs/features/steps/`. If any are missing, invoke TFK-07. 3. Remove `@wip` when ready: Tag unimplemented scenarios `@wip` during ESM; remove the tag when implementation is complete so CI runs them (`behave.ini` uses `tags = ~@wip`). 4. Prepare fixtures: Check the Fixture Library Catalog for needed test data. If presets are missing, invoke TFK-07. 5. Run acceptance tests: `make test-at` (runs `docs/features/` via `behave.ini`). 6. The dual approach remains: Django test client unit/integration tests remain valuable for fast feedback. Behave AT runs the same scenarios through the BDD layer. Both must pass. ### Key Principle Acceptance tests are the real thing minus the GUI (Django test client — not Playwright). No mocking. The `.feature` file in `docs/features/` IS the grounding executable requirements document for AT. ## TFK boundary: AT (user-visible) vs `@agent_proof` pytest Do not* express agent orchestration proofs as Behave scenarios. \| Concern \| Layer \| Runner \| Marker / location \| \|---------\|-------\|--------\|-------------------\| \| User-visible behavior, screens, HTMX, API responses humans see \| AT \| behave-django (`make test-at`) \| `.feature` in `docs/features/` \| \| Agent control-plane wiring (PRF-): tool allowlists, blackboard, plan handoff, HITL gates, 429 retry \| Integration \| pytest `@pytest.mark.agent_proof` \| `tests/integration/agent/` \| \| Deterministic shell (lane 2): D0 pre-filter, JSON parse, validate-before-LLM \| Integration \| pytest (no `@agent_proof`) \| `tests/integration/` \| \| Model quality / golden tasks (TASK-, lane 4) \| Eval \| pytest `@pytest.mark.quality` + `@pytest.mark.live_llm` \| `tests/fixtures/agent_tasks/` — nightly, not AT \| Rules: - AT proves what the user sees — forms, redirects, ChangeSet approval UI, error messages. - `@agent_proof` proves what the agent loop did under CAP-004 ScriptedLLM — trace beats via `assert_agent_story`, not chat prose assertions in Gherkin. - When a feature touches both surfaces, BPE-04 adds/updates `.feature` scenarios and BPE-02 (or the agent slice) adds `@agent_proof` tests — both green before DoD; neither substitutes for the other. - Skip `@agent_proof` rows in BPE-04 when SAO §17 is N/A. ## Skills (additional) - Behave-Django BDD Runner (BDD_RUNNER) - Agent Integration Proof Patterns (TEST_INTEGRATION) — when SAO §17 in scope; AT does not replace this skill ## Inputs (additional) - Step Library Catalog (Document, Required) — produced by TFK-03. - Fixture Library Catalog (Document, Required) — produced by TFK-04. - Behave Configuration (Code, Required) — produced by TFK-02 (Artifact #51). - Agent Proof Table (from BPE-01 Section G, when SAO §17) — PRF rows map to pytest, not `.feature` files.
#5	Implement Journey Certification Tests	Read the 'BPE-5 Implement Journey Certification Tests' activity from playbook 'Junior Edda' (workflow 'Build Feature'). Follow its guidance when assisting me. APPEND TO GUIDANCE: --- ## TFK Integration: E2E Infrastructure Journey certification tests that require browser-based Playwright execution use TFK's E2E infrastructure under `tests/e2e/`. ### E2E vs AT (clarification) - AT (BPE-04): Proves one particular feature works. `docs/features/` via Django test client (`behave --simple`). Fast. Every CI run. - E2E (BPE-05): Proves the ability to fulfill a goal using system means. `tests/e2e/` via Playwright + LiveServer. Sequential. Screenshots. Staging / release certification. Same Gherkin phrases (catalog); separate step implementations and `environment.py`. ### Using TFK E2E Infrastructure 1. Locate journey intent: Read `docs/features/user_journey.md` for the multi-screen goal. 2. Write E2E `.feature`: Add journey scenarios to `tests/e2e/` (optionally tag `@e2e`). 3. Write as multi-phase .feature: One long sequential `.feature` file with multiple scenarios. 4. Environment targeting: Use `--base-url` to point at any environment. 5. Screenshots: `tests/e2e/environment.py` automatically captures screenshots after every step. 6. Data preparation: E2E tests use `--db-uri` to load initial data via loaddump. 7. Run: `make test-e2e` → `manage.py behave tests/e2e/`. ## Inputs (additional) - E2E Test Configuration (Code, Required) — produced by TFK-06. - Fixture Library Catalog (Document, Required) — produced by TFK-04.
#6	Check Definition of Done	Read the 'BPE-6 Check Definition of Done' activity from playbook 'Junior Edda' (workflow 'Build Feature'). Follow its guidance when assisting me. ## Purpose Validate that feature/story implementation complies with all project rules and standards by examining current state and outputs. ## Core Development Rules Checklist ### Test-First Development - [ ] Every function/method has corresponding test(s) - [ ] Feature files in `docs/features/` exist and comply with scenarios - [ ] Tests use pytest framework - [ ] Mocking is minimal ### Continuous Testing - [ ] All tests are runnable via `pytest tests/` - [ ] Tests are pytest compatible with proper fixtures - [ ] `tests.log` file exists and contains test output ### Concise Methods - [ ] Top-level (public) methods are 20-30 lines maximum - [ ] Supporting logic is in well-named private methods - [ ] Helper methods have single, focused responsibilities - [ ] Method names are descriptive and clear ## Code Quality Rules Checklist ### Import Management - [ ] All imports are at module level - [ ] No imports inside functions/methods - [ ] Dependencies are properly declared ### Log Story Verification - [ ] Implementation plan / issue contains a Log Story Script (Where / Beat / Trigger / Must include) - [ ] Every script row is proven by a passing `_log_story_` caplog test (or `checkpoint.log_story_command` from the iteration manifest) - [ ] Happy and reject paths covered when the scenario has both - [ ] Log levels are appropriate (DEBUG, INFO, WARNING, ERROR) - [ ] No secrets/tokens/passwords appear at INFO in asserted lines - [ ] No deferred logging slice remains in the plan Grep of `logs/app.log` is an audit aid, not the DoD gate. Caplog (or `log_story_command`) is the gate. ### Agent Proof Verification (when SAO §17 / plan Section G in scope) - [ ] Implementation plan / issue contains an Agent Proof Table (PRF ID, workflow, script path, trace beats) - [ ] Every PRF row is proven by a passing `_agent_story_` test marked `@pytest.mark.agent_proof` (or `checkpoint.agent_proof_command` from the iteration manifest) - [ ] Happy and reject paths covered when the PRF is adverse or has a reject sketch - [ ] Tests use CAP-004 ScriptedLLM only — no `FakeLLM`, no `@patch` on ToolExecutor or domain services in `@agent_proof` tests - [ ] LLM scripts exist under `tests/fixtures/llm_scripts/<prf-id>/` for every asserted path - [ ] Trace asserts control-plane beats (tools, domain writes, plan/blackboard state) — not assistant wording - [ ] No secrets/tokens/passwords in trace or asserted log lines - [ ] No deferred agent-proof slice remains in the plan - [ ] TASK-* lane 4 evals (if any) are not required for story DoD — confirm they remain on nightly/promotion band only Grep of run logs is an audit aid, not the DoD gate. `assert_agent_story` (or `agent_proof_command`) is the gate. ## Testing and Quality Assurance Checklist ### Integration Test Standards - [ ] Integration tests in `tests/integration/` exist - [ ] Integration tests avoid mocking (except CAP-004 ScriptedLLM in `@agent_proof` tests) - [ ] Real dependencies are used in integration scenarios ### Commit Conventions - [ ] Recent commit messages follow Angular conventional format - [ ] Commits are atomic and focused - [ ] Breaking changes are documented in commit messages ## UI and Frontend Rules Checklist ### Django Views + HTMX - [ ] No DRF views exist for new web UI features - [ ] Django views return HTML templates - [ ] HTMX attributes used for dynamic interactions - [ ] Services layer is shared between MCP and Web UI ### Semantic Naming - [ ] All interactive elements have `data-testid` attributes - [ ] Naming follows kebab-case convention - [ ] Form inputs have proper name and id attributes ## Documentation Checklist ### Scenario Writing - [ ] BDD scenarios exist for features - [ ] Feature files are well-structured - [ ] Scenarios cover edge cases and error conditions - [ ] Review GUI - do scenarios match behavior, fields, URLs, design rules? Report inconsistencies to user ### TODO Management - [ ] TODO comments exist for incomplete implementations - [ ] TODO items have clear descriptions - [ ] TODOs in dependencies can be ignored ### Document Updates - [ ] Review code: new packages, patterns, approaches worth documenting? - [ ] Review conversation: need to update feature files/corrections? - [ ] Review modus operandi against workflows and rules - can we improve? ## Final Validation Checklist ### Overall Quality Check - [ ] Feature meets acceptance criteria - [ ] Code is production-ready - [ ] Documentation exists and is accurate ### Integration Validation - [ ] Feature integrates with existing system - [ ] No breaking changes introduced - [ ] Dependencies properly declared in requirements.txt ### Deployment Readiness - [ ] Database migrations exist if needed - [ ] Environment variables are documented - [ ] Configuration changes are documented ### Cleanup - [ ] Remove temporary files like debug_.py - [ ] Scan file structure for stray misplaced files - [ ] Remove .log files from repository ## Rules Before validating DoD, read each Rule below in this playbook (by slug), then apply it as the acceptance criterion for the matching checklist items. Do not rely on memory of the rule text. Required: - `do-test-first` - `do-continuous-testing` - `do-write-concise-methods` - `do-import-on-module-level` - `do-informative-logging` - `do-assert-log-story` - `do-assert-agent-story` (when SAO §17 / Section G in scope) - `do-not-mock-in-integration-tests` - `do-follow-commit-convention` - `do-semantic-versioning-on-ui-elements` - `do-write-scenarios` - `do-add-todos-for-incomplete-items` - `pytest` Activity-specific (not a substitute for the rules above): - Observability DoD is Log Story Verification (caplog / `log_story_command`), not "logging statements exist". - Agent DoD is Agent Proof Verification (`agent_proof_command` / `@agent_proof`), not "the agent loop exists" or Behave AT alone. ## Actions All checkboxes must be completed before considering the story "Done". Any deviations must be presented to user. If user says "collect for cleanup but defer" - create a Backlog item in GitHub as Issue with "deferred" tag. Otherwise resolve deviations as directed by user, commit following Angular convention, and send Pull Request. ## Success Criteria - All checklist items verified - No deviations or all approved by user - Code production-ready - Ready for PR ## Inputs Read these before starting this activity. They are produced earlier in the playbook and are authoritative — raise a drift event instead of deviating. - Feature Files (Document, Required) — produced by Write Feature Files (#39). - Implementation Plan Template (Template, Required) — produced by Plan Feature (#96). ## Defects Found During DoD When DoD review reveals a defect (behavior does not match feature files or acceptance criteria) — not a deferred cleanup item: 1. File a Bug Report via `report_bug` MCP tool or Feedback UI (see Bug Report artifact for body structure) 2. Invoke Fix Bug (BPE-09) with the Bug Report as input 3. Do not mark the story done until blockers are resolved and BPE-09 success criteria are met
#7	Finalize Feature	Read the 'BPE-7 Finalize Feature' activity from playbook 'Junior Edda' (workflow 'Build Feature'). Follow its guidance when assisting me. ## Purpose Finalize feature with testing, validation, and deployment preparation. ## Steps ### 0. Run All Tests Run all tests and ensure none are broken. If there are discrepancies between test expectations and implementation - ask user to clarify what takes precedence. Mark scenarios with "done" emoji and add "completed" tag. ### 1. Identify E2E Test Identify Playwright E2E test for this feature/scenario. If there is none - ask user if they want to create one. ### 2. Run Development Server and Validate Auto-run: Start the server: `python manage.py runserver` Execute Playwright tests to ensure they pass. Fix any issues found during testing. ### 3. Run Full Test Suite Auto-run: Execute all unit tests, integration tests, and E2E tests. Ensure no regressions were introduced. Fix any failing tests. ### 4. Update Project Dependencies Add any new packages to requirements.txt that were added during feature development. Use `pip install <package>` to add new dependencies. Ensure version constraints are appropriate. Test that `pip install -r requirements.txt` works in a fresh venv. ### 5. Present Completed Work Summarize implemented features and changes. Show test results and coverage. Demonstrate functionality if possible. ### 6. Final Commit Check if there is a corresponding issue. Update the status with the latest changes and associate commit. Commit all remaining changes using Angular-style commit messages. Ensure commit message clearly describes the completed feature. ### 7. Handle Feature Branch If on feature branch - ask user if they want to send a PR or merge into main. ### 8. Update Screen Flow Diagram Update `docs/ux/2_dialogue-maps/screen-flow.drawio` to mark completed screens with green borders: - Change `strokeColor=#1565c0` to `strokeColor=#22c55e` (green) - Add `strokeWidth=3` if not present - Add ✅ emoji prefix to screen labels - This indicates feature completion per the diagram legend ### 9. Close and Mark Complete Close issue if exists, review and mark all todos as "done" in the implementation plan, and update scenario and/or feature with "done" emoji. ## Rules to Follow ### I. 100% Test Pass Rate Required Only 100% test pass rate can be reported as "success" or "complete". 92%, 95%, 99% are NOT success. Any failing tests must be fixed before declaring feature complete. Cannot mark features as "done" or "production-ready" with failing tests. Test failures must be resolved, not deferred or ignored. Exception: Only if user explicitly approves deferring specific test scenarios can they be excluded from the count. ### II. Fix Tests Immediately Failing tests are major problem - we don't start new development until we fix them. ### III. Commit Convention Follow Angular convention with proper type, scope, subject, body, and footer. ## Success Criteria - All tests passing (100% pass rate) - Dependencies updated in requirements.txt - Final commit made with clear message - Issue updated and closed - Implementation plan marked complete - Feature file updated with completion markers ## Inputs Read these before starting this activity. They are produced earlier in the playbook and are authoritative — raise a drift event instead of deviating. - Screen Flow / Dialogue Map (Diagram, Optional) — produced by Create Dialogue Maps (#38). - Implementation Plan Template (Template, Required) — produced by Plan Feature (#96). - Definition of Done Checklist Template (Template, Required) — produced by Check Definition of Done (#101). ## Defects Found During Finalization When testing or validation reveals a defect (failing test that exposes a real bug, user-reported issue during demo, or ERROR in `logs/app.log` tied to broken behavior): 1. File a Bug Report via `report_bug` MCP tool or Feedback UI (see Bug Report artifact for body structure) 2. Invoke Fix Bug (BPE-09) — do not patch ad hoc without the test-first workflow 3. Do not declare the feature complete until BPE-09 success criteria are met (100% regression green)
#8	Process Change Request	Read the 'BPE-8 Process Change Request' activity from playbook 'Junior Edda' (workflow 'Build Feature'). Follow its guidance when assisting me. ## Purpose Process an enhancement or change request by reconciling requirements across the spec stack, agreeing a target state with the user, then closing with an approved reconciliation record and in-place spec updates. This activity does not implement code. ## Requirements Sources Reconcile across all layers that exist in this project. Read what is present; skip absent layers and note the gap in the matrix. For each affected Screen ID, build a reconciliation matrix: \| Layer \| Typical source \| Activity \| Current state \| Drift? \| Notes \| \|-------\|----------------\|----------\|---------------\|--------\|-------\| \| User journey \| User Journey document \| Define User Journey \| — \| Y/N \| Screen narrative \| \| Scenarios \| Feature files (Gherkin) \| Write Feature Files \| — \| Y/N \| Executable requirements \| \| Mockups \| HTML mockups / prototypes \| Create Mockups \| — \| Y/N \| UX validation screens \| \| Screen flow \| Dialogue map \| Create Dialogue Maps \| — \| Y/N \| Navigation flow \| \| IA guidelines \| IA guidelines \| Define Information Architecture \| — \| Y/N \| Design system rules \| \| Prior plan \| Implementation plan \| Plan Feature \| — \| Y/N \| What was planned \| \| Architecture \| SAO.md \| Write SAO.md \| — \| Y/N \| As-designed constraints \| \| As-built \| Code, templates, tests \| BPE-02–07 \| — \| Y/N \| What shipped \| ## Steps ### 1. Reconcile Requirements (read-only first) - Read inputs per ## Inputs below - Read the change request thoroughly; identify affected Screen IDs and scope - Read user journey — locate affected screen sections - Read feature files — locate affected scenarios - Read mockups — locate affected prototype screens - Read screen flow and IA guidelines when UI or navigation is in scope - Read SAO.md for architectural constraints touched by the change - Inspect as-built code and tests for the same Screen IDs - Flag conflicts between layers; do NOT assume — list open questions - If you don't understand user intent or UX details — ASK Fast path: If the matrix shows zero drift across all present spec layers and the change is purely cosmetic with no scenario impact, document the empty matrix and note minimal delta in the reconciliation document. ### 2. Propose Target State - Draft in-place revisions to canonical consumed artifacts (only layers where drift was found). Do not create parallel "Updated " copies — revise the same User Journey, Feature Files, Mockups, Screen Flow, and IA documents the project already uses. - Create Change Reconciliation Document* at `docs/plans/{FEAT}_CHANGE_RECONCILIATION.md` containing: - Trigger (change request summary) - Reconciliation matrix (from Step 1) - Proposed spec diffs (summary + file paths) - Open questions for user - Fast-path justification (if applicable) - Present target state to user; ask clarification questions - Gate: User must approve target state before closing this activity ### 3. Close Reconciliation - Ensure approved Change Reconciliation Document is saved at `docs/plans/{FEAT}_CHANGE_RECONCILIATION.md` - Ensure in-place revisions to canonical spec artifacts are complete (Step 2) - Technical replanning and implementation are out of scope for this activity — the human may invoke Plan Feature (Activity 96) separately when ready ## Rules to Follow ### I. Do Not Assume If unclear — ASK. Never guess user intent or implementation details. ### II. Spec Before Code Reconcile and update requirements artifacts before any implementation planning or coding. ### III. No Execution in Process Change Request Implementation, testing, and commits happen in later Build Feature activities after separate planning approval. ### IV. Traceability Every affected Screen ID must appear consistently across all present spec layers. Verify with project grep conventions. ## Success Criteria - Change request captured and understood - Reconciliation matrix complete for all affected Screen IDs - Canonical spec artifacts revised in place or explicitly marked unchanged (fast path) - Change Reconciliation Document created, saved, and user-approved - No code changes made during this activity ## Inputs Read these before starting this activity. They are produced earlier in the playbook and are authoritative — raise a drift event instead of deviating. - User Journey (Document, Required) — produced by Define User Journey (#36). - Screen Flow / Dialogue Map (Diagram, Required) — produced by Create Dialogue Maps (#38). - Feature Files (Document, Required) — produced by Write Feature Files (#39). - HTML Mockups (Code, Optional) — produced by Create Mockups (#40). - IA Guidelines (Document, Optional) — produced by Define Information Architecture (#37). - System Architecture Overview Template (Document, Required) — produced by Write SAO.md (#59). - Implementation Plan Template (Template, Required) — produced by Plan Feature (#96). - Definition of Done Checklist Template (Template, Required) — produced by Check Definition of Done (#101).
#9	Fix Bug	Read the 'BPE-9 Fix Bug' activity from playbook 'Junior Edda' (workflow 'Build Feature'). Follow its guidance when assisting me. ## Purpose Fix a reported defect using a test-first workflow: prove the bug with a failing test, implement the fix, prove green, run full regression, then gate integration/PR/hotfix decisions with the user. This activity implements code. It does not reconcile requirements — use Process Change Request (#103) for scope or spec changes. ## Prerequisites - A Bug Report (GitHub Issue) filed via `report_bug` MCP tool or the Feedback UI - Bug may have been filed during Check Definition of Done (#101), Finalize Feature (#102), or Acceptance, Bug Reports & Deploy Fixes (#183) ## Steps ### 1. Analyze Why Tests Missed the Bug - Read the Bug Report thoroughly (Description, Reproduction, Environment, severity if present) - Identify affected feature files, scenarios, Screen IDs, and code paths - Determine why existing tests did not catch the defect — missing scenario, weak assertion, wrong test layer, integration gap, flaky test, etc. - Document the gap briefly (issue comment or working notes) ### 2. Add or Extend Tests (Red) - Write or extend tests that reproduce the bug — they must fail before the fix - Prefer the lowest appropriate layer per SAO Test Strategy: unit → integration → AT (`docs/features/`) → E2E (`tests/e2e/`) - Follow `do-test-first` and `do-not-mock-in-integration-tests` - Run the new or extended test(s) and confirm they fail for the expected reason ### 3. Implement the Fix - Implement the minimal correct fix - Keep scope focused — no unrelated refactors ### 4. Prove the Fix (Green) - Run the new or extended test(s) — must pass - If UI or journey behavior changed, re-run relevant BPE-04 / BPE-05 checkpoints for the affected scenario ### 5. Full Regression - Run the full test suite: `pytest tests/` - Run E2E suite when UI or journey paths changed: `pytest tests/e2e/` - 100% pass rate required before closing — same standard as Finalize Feature (#102) ### 6. Commit and User Gate - Commit with Angular convention: ```bash git add -A git commit -m "fix({scope}): {bug title}" ``` - Ask the user: 1. Integrate or submit PR? — merge to main locally, or open / submit a pull request? 2. Hotfix deployment needed? — if production is affected, coordinate a patch release per MIN-06 / deployment playbook - Do not push until the user approves ## Rules to Follow Before fixing, read each Rule below in this playbook (by slug), then apply it: - `do-test-first` - `do-not-mock-in-integration-tests` - `do-follow-commit-convention` - `do-small-increments` - `do-informative-logging` (when the fix touches decision points) ## Success Criteria - Root cause of the test gap identified and documented - Failing test proves the bug before the fix - Fix implemented; proving tests pass - Full regression green (100% pass rate) - Commit made; user decision recorded on integrate / PR / hotfix ## Inputs Read these before starting this activity. - Bug Report (Document, Required) — produced by Check Definition of Done (#101), Finalize Feature (#102), or Acceptance, Bug Reports & Deploy Fixes (#183) via `report_bug` MCP tool or Feedback UI.
Plan Iteration
#1	Readiness Check	Read the 'PIN-1 Readiness Check' activity from playbook 'Junior Edda' (workflow 'Plan Iteration'). Follow its guidance when assisting me. ## Purpose Validate all prerequisites before planning begins. User provides a goal sentence (e.g. "we aim to complete act-11"). Each gate is checked in order — if any fails, stop with specific remediation instructions. ## Input The user's stated goal sentence. Example: "we aim to complete act-11" or "implement team management features". ## Gates ### Gate 1 — BSP (Bootstrap) Check that the project has been bootstrapped: ```bash test -f Makefile && echo "PASS" \|\| echo "FAIL: Run BSP workflow — project not bootstrapped" test -f requirements.txt && echo "PASS" \|\| echo "FAIL: Run BSP workflow — requirements.txt missing" ``` If either fails: STOP — "Project not bootstrapped. Run the BSP (Bootstrap Project) workflow first." ### Gate 2 — User Journey Coverage Identify the target act from the user's goal (e.g. "act-11", "team management"). Check `docs/features/user_journey.md` for a matching section: ```bash grep -i "act-{N}" docs/features/user_journey.md && echo "PASS" \|\| echo "FAIL" ``` If missing: STOP — "No user journey section found for {target}. Add the act section to `docs/features/user_journey.md` before planning." ### Gate 3 — Feature Specs (BDD) Check that `docs/features/act-{N}/` exists and contains `.feature` files: ```bash ls docs/features/act-{N}/.feature 2>/dev/null && echo "PASS" \|\| echo "FAIL" ``` If missing: STOP* — "No feature specs found for act-{N}. Run the ESM (Envision the System) workflow to produce `.feature` files." ### Gate 4 — Architecture (SAO.md) ```bash test -f docs/architecture/SAO.md && echo "PASS" \|\| echo "FAIL" ``` If missing: STOP — "No `docs/architecture/SAO.md` found. Run the DTA (Define Architecture) workflow first." ### Gate 5 — AI IDE Config (DSP) ```bash test -f CLAUDE.md && echo "PASS" \|\| echo "FAIL" ``` If missing: STOP — "No `CLAUDE.md` found. Run the DSP (Deploy Software Process) workflow to configure the AI IDE." ### Gate 6 — Agent proof harness (optional; when SAO §17 applies) When `docs/architecture/SAO.md` §17 documents in-app LLM agents (not "Not applicable"): ```bash test -f tests/support/agent_story.py && echo "PASS" \|\| echo "FAIL: TFK-02 must bootstrap tests/support/agent_story.py before agent-feature BPE" ``` If SAO §17 is N/A: SKIP — record `agent_harness: n/a`. If SAO §17 applies and the file is missing: STOP — "Agent proof helper missing. Run TFK-02 (Bootstrap Test Harness) to create `tests/support/agent_story.py` and ScriptedLLM fixtures before planning agent features." ## Mockup Check (advisory, after all gates pass) ```bash ls docs/ux/mockups/act-{N}/ 2>/dev/null && echo "Mockups found" \|\| echo "No mockups" ``` If no mockups: ask — "No mockups found for {target}. Create them per ESM-04 before planning? (yes/no)" - Yes → run ESM-04, then return to PIN-01 after mockups are created - No → proceed with feature specs only ## Output When all gates pass: - `target_act`: extracted act identifier (e.g. "act-11") - `feature_files[]`: list of `.feature` files in `docs/features/act-{N}/` - `has_mockups`: boolean - `agent_harness`: `ready` \| `n/a` (from Gate 6) Proceed to PIN-02 Orient & Validate Scope. ## Success Criteria - All required gates pass (Gates 1–5; Gate 6 pass or skipped when SAO §17 N/A) - User has confirmed mockup status - `target_act` and `feature_files[]` identified - `agent_harness` recorded when SAO §17 applies
#2	Orient & Validate Scope	Read the 'PIN-2 Orient & Validate Scope' activity from playbook 'Junior Edda' (workflow 'Plan Iteration'). Follow its guidance when assisting me. ## Purpose Build the iteration orient summary from the feature files identified in PIN-01, using historical lesson data where available. Establishes what the iteration will deliver and any risk flags. ## Session Resume Check If `docs/plans/iterations/pin-orient-{today's date}.md` already exists → skip this activity, load it as the orient summary, proceed to PIN-03. ## Prerequisites - `target_act` and `feature_files[]` from PIN-01 - `docs/lessons_learned/ITER-.md` files (may not exist on first iteration) ## Steps ### Step 1: Read Feature Specs For each file in `feature_files[]`, read the BDD scenarios. Extract: - Scenario titles - Given/When/Then outlines (to understand scope boundaries) - Tags if present (e.g. `@wip`, `@smoke`) — optional; scenarios with no tags are treated as fully eligible ### Step 2: Read Learning Log ```bash ls -t docs/lessons_learned/ITER-.md 2>/dev/null \| head -5 ``` If no files found → first iteration. Create the directory and continue with first-iteration defaults: ```bash mkdir -p docs/lessons_learned ``` - `velocity_ratio`: unknown - `dominant_drift`: none - `footprint_accuracy`: unknown If files found → read the 5 most recent. Each file has YAML frontmatter. Extract: ```bash for f in $(ls -t docs/lessons_learned/ITER-.md \| head -5); do awk '/^---/{p++} p==1{print} p==2{exit}' "$f" done ``` Compute across the 5 most recent entries: - velocity_ratio trend: improving (trend up) / stable / degrading (trend down) - dominant_drift: most frequent non-none value (or "none" if all none) - footprint_accuracy trend: improving / stable / degrading Also scan the body of the most recent file for flagged SAO.md decisions. If any are present: > "Prior iteration flagged SAO.md decisions not yet applied: {list}. Verify they were committed before this iteration begins." Note:* `docs/lessons_learned/log.yaml` is deprecated. If it exists, ignore it. ### Step 3: Validate Scope - If `velocity_ratio` degrading AND >2 scenarios: flag scope risk - If `dominant_drift` was `footprint_violation` in 2+ of last 5: flag complexity risk ### Step 3b: Scope Bounding Count total scenarios across all `feature_files[]`. Assess whether the full scope fits your effective context window: - Estimate complexity: scan each scenario's step count and whether it touches new models, new pages, or new infrastructure. - If the full scope feels manageable (scenarios are simple CRUD / UI checks with predictable footprints), proceed with all scenarios. - If the scope is large and scenarios are complex (new models, cross-cutting concerns, unresolved system dependencies), ask: > "This iteration has {N} scenarios. Some appear complex — would you like to: > A) Proceed with all {N} > B) Take a subset (tell me which scenarios or feature files) > C) Split across multiple iterations — first file's scenarios now, remainder via a second PIN run" Handle user response: - A: Proceed with all scenarios; record `scope_override: confirmed` in orient summary - B: Filter to the user-selected subset; record deferred scenarios in `scenarios_deferred[]` - C: Take only the first feature file's scenarios (or user-specified first batch); record `scope_deferred_files[]` If the full scope is simple and clearly manageable, do not ask — just proceed and note `scope_override: not_needed`. ### Step 4: Write Orient Summary Create `docs/plans/iterations/pin-orient-{YYYYMMDD}.md`: ```markdown ## Orient Summary — {date} ### Target Act: {target_act} Feature files: {list} ### Scenarios in Scope {list of scenario titles — after scope bounding} ### Scenarios Deferred {list or "None"} ### Velocity Trend {improving/stable/degrading or "First iteration"} ### Dominant Drift {signal_type or "None — first iteration"} ### Scope Validation {per-scenario risk flags or "No risks detected"} ``` ## Success Criteria - Orient summary written to `docs/plans/iterations/pin-orient-{date}.md` - Scenarios in scope identified (after scope bounding — may be a subset of all feature file scenarios) - Deferred scenarios recorded if scope was reduced - Risk flags computed or noted as first iteration - Ready for PIN-03
#3	Contract	Read the 'PIN-3 Contract' activity from playbook 'Junior Edda' (workflow 'Plan Iteration'). Follow its guidance when assisting me. ## Purpose For each BDD scenario in scope: run BPE-01 planning, create code skeletons (public + private helpers), build the context map and do-not-do list, verify checkpoint commands (behavior and log-story when applicable). After all scenarios: derive the file-level conflict map and write the execution manifest YAML. ## Prerequisites - Orient summary from PIN-02 - `docs/architecture/SAO.md` - BDD feature specs from `docs/features/act-{N}/` ## Steps ### Step 1: Create Iteration Branch (once) ```bash # Safe for retries — checks before creating git show-ref --verify --quiet refs/heads/iteration/YYYYMMDD-{goal-slug} \ && git checkout iteration/YYYYMMDD-{goal-slug} \ \|\| git checkout -b iteration/YYYYMMDD-{goal-slug} ``` ### Steps 2–8 repeat for each scenario ### Step 2: Read Specification and Record Feature File Path Read the BDD feature spec for this scenario. Note the exact `.feature` file path — you will record it in the manifest. Record: `feature_file_path: docs/features/act-{N}/{filename}.feature` Read `docs/architecture/SAO.md`. Note governing sections. ### Step 3: Identify System Dependencies Before running BPE-01, scan the scenario's steps for capabilities that require external infrastructure (not part of this feature's codebase footprint): Common patterns to check for: - "receives a notification" → `notification_service` - "receives an email" → `email_backend` - "background task" → `task_queue` - "search index" → `search_backend` If any are found: record them as `system_dependencies` for this scenario. If the infrastructure doesn't exist in the codebase, choose a resolution before proceeding: \| Option \| When to use \| Action \| \|--------\|-------------\|--------\| \| A — Prerequisite scenario \| Infrastructure is reusable across multiple scenarios \| Create a separate scenario S0 covering the infrastructure; add it as a dependency to all affected scenarios. Ask user before doing this. \| \| B — Stub & proceed \| Infrastructure is minor and can be stubbed \| Note stub in BPE-01 plan; create a stub file named exactly after the dependency identifier (e.g. `notification_service` → `methodology/services/notification_service.py`) with a no-op implementation; include it in the skeleton commit; annotate the manifest entry as `notification_service [STUBBED]`. MIN-04 checks for this file by name — the naming must match or MIN-04 will falsely escalate. Decide autonomously, note in manifest. \| \| C — Skip this iteration \| Infrastructure is substantial and out of scope \| Remove scenario from `scenarios_in_scope[]`; add to `scenarios_deferred[]` in orient summary. Ask user before doing this. \| ### Step 4: Run BPE-01 Follow BPE/BPE-01 completely. Output must include: a) Context Map — 3–5 `file:line_range` references with one-line note each. b) Do-Not-Do List — derived from SAO.md sections: ``` - Do NOT create a new Django app - Do NOT add a manager/repository layer ``` c) Applicable SAO.md Sections — list by heading name. d) Checkpoint Command — pytest command that proves behavior done. e) Log Story Script — from BPE-01 Section E (Where / Beat / Trigger / Must include). f) Log-story tests + command — when Section E is non-empty: `log_tests[]` names/paths and `checkpoint.log_story_command`. Important: When running BPE-01 inside PIN-03, skip BPE-01 Step 9 (Submit for Approval) and BPE-01 Step 10 (GitHub/Issue Management). The implementation plan is captured in the manifest; GitHub/GitLab issues are created in PIN-04. Running those steps here would produce duplicates. ### Step 5: Create Thorough Skeleton Write skeletons for every class and method — public AND private helpers. The skeleton is the complete architecture; an implementor must not need to add new methods. Production skeleton example: ```python class FeatureService: def execute(self, id: int, user) -> dict: """ :param id: Feature ID. Example: 42 :param user: Requesting user. :return: dict. Example: {"id": 42, "name": "..."} :raises PermissionError: If user cannot access :raises ValueError: If not found """ raise NotImplementedError() def _validate(self, id, user) -> None: raise NotImplementedError() def _serialize(self, obj) -> dict: raise NotImplementedError() ``` ### Step 6: Verify Checkpoint Commands ```bash {checkpoint.command} 2>&1 \| tail -5 # When log_tests present: {checkpoint.log_story_command} 2>&1 \| tail -5 ``` Acceptable: `NotImplementedError` or `collected 0 items`. Not acceptable: `ImportError`, `SyntaxError`. ### Step 7: Commit Skeleton ```bash git add -A git commit -m "chore(pin): skeleton S{N} — {scenario title}" ``` ### Step 8: Record Footprint ```bash git diff HEAD~1 --name-only ``` Record as `codebase_footprint[]`. Record `skeleton_commit` hash. ### After All Scenarios: ### Step 9: Build Conflict Map For each scenario pair, check footprint intersection: ``` S1 ∩ S2 = {tools.py} → CONFLICT → serialize S1 ∩ S3 = {} → no conflict → parallel ``` ### Step 10: Assign Parallel Groups Non-conflicting → same group. Conflicting → different groups with dependency. ### Step 11: Write Execution Manifest Create `docs/plans/iterations/ITER-YYYYMMDD-HHmm-{goal-slug}.yaml`: ```yaml iteration: goal: "{iteration_goal}" doctrine_version: "2.0" created_at: "{ISO8601}" conflict_map: path/to/file.py: [S1, S2] parallel_groups: A: [S1] B: [S3] C: [S2] drift_thresholds: absorbed: checkpoint_fail_retry: 1 escalated: footprint_violation: true method_explosion: true sao_violation: true checkpoint_fail_after_retry: true scenarios: S1: title: "{title}" parallel_group: A skeleton_commit: "{hash}" codebase_footprint: [...] feature_file_paths: - docs/features/act-{N}/{filename}.feature system_dependencies: - notification_service # or empty list [] if none; append [STUBBED] if stubbed log_story_script: - {beat: entry, where: "LoginView.post", trigger: method_called, must_include: ["email="]} - {beat: branch, where: "LoginView.post", trigger: bad_credentials, must_include: ["authentication failed"]} log_tests: - {name: test_login_log_story_reject, path: src/yggdrasil/auth/tests/test_login_view.py} checkpoint: command: "pytest ...::test_login_post_failure_rerenders_with_error -x" expected_exit_code: 0 log_story_command: "pytest ...::test_login_log_story_reject -x" # required when log_tests non-empty dependencies: [] context_map: [...] do_not_do: [...] sao_sections: [...] ``` Rules: - If `log_tests` is non-empty, `checkpoint.log_story_command` is required - Omit `log_story_script` / `log_tests` / `log_story_command` only when BPE-01 Section E is empty (document why) - PIN-04 consumers may embed both commands in the issue SCENARIO block ## Rules Before running BPE-01 inside this activity and writing the manifest, read each Rule below in this playbook (by slug), then apply it. Do not rely on memory of the rule text. Required: - `do-plan-before-doing` - `do-skeletons-first` - `do-test-first` - `do-informative-logging` - `do-assert-log-story` - `do-follow-commit-convention` - `do-small-increments` - `pytest` Activity-specific (not a substitute for the rules above): - When BPE-01 Section E (Log Story Script) is non-empty, the manifest must include `log_story_script[]`, `log_tests[]`, and `checkpoint.log_story_command`. ## Success Criteria - Skeleton committed for every scenario - `feature_file_paths[]` recorded per scenario (paths to .feature files) - `system_dependencies[]` recorded per scenario (empty list if none) - All system_dependencies resolved (stubbed, prerequisite scenario, or deferred) before proceeding - Stub files named to match their dependency identifier so MIN-04's codebase check succeeds - Execution manifest YAML written to `docs/plans/iterations/` - Conflict map derived from actual skeleton commits - All checkpoint commands verified (NotImplementedError or 0 items), including `log_story_command` when declared
#4	Publish	Read the 'PIN-4 Publish' activity from playbook 'Junior Edda' (workflow 'Plan Iteration'). Follow its guidance when assisting me. ## Purpose Create the Milestone and Issues from the execution manifest, on GitHub or GitLab. Ask user before creating. The Milestone becomes MIN's operational ground truth — it must contain the full YAML manifest inline. ## Prerequisites - Execution manifest YAML from PIN-03 (`ITER-.yaml`) with `milestone_number: null` placeholder - All skeleton commits on iteration branch - GitHub CLI (`gh`) or GitLab CLI (`glab`) authenticated ## Step 0 — Platform Detection ```bash REMOTE=$(git remote get-url origin 2>/dev/null \|\| echo "") if echo "$REMOTE" \| grep -q "github.com"; then PLATFORM=github gh auth status \|\| { echo "Run: gh auth login"; exit 1; } elif echo "$REMOTE" \| grep -q "gitlab"; then PLATFORM=gitlab glab auth status \|\| { echo "Run: glab auth login"; exit 1; } else echo "Cannot detect platform from git remote. Set PLATFORM=github or PLATFORM=gitlab manually." exit 1 fi echo "Platform: $PLATFORM" ``` ## Step 1 — User Gate Ask: "Platform detected: {github\|gitlab}. Create milestone and issues from the manifest? (yes/no)" - No* → write manifest to `docs/plans/iterations/` only. Stop. Inform user: "Manifest saved. Invoke MIN manually when ready." - Yes → continue. ## Step 2 — Bootstrap Labels GitHub: See GitHub Issue Operations skill — Bootstrap Labels section. GitLab: See GitLab Issue Operations skill — Bootstrap Labels section. ## Step 3 — Create Milestone GitHub: ```bash MILESTONE_NUMBER=$(gh api repos/{owner}/{repo}/milestones \ --method POST \ --field title="ITER-YYYYMMDD \| {iteration_goal}" \ --field description="<!-- MANIFEST --> $(cat {manifest_path}) <!-- /MANIFEST --> ## Iteration Goal {iteration_goal} ## Scenarios {list}" \ --jq '.number') echo "Milestone: #$MILESTONE_NUMBER" ``` GitLab: ```bash PROJECT=$(glab api projects/:fullpath --jq '.id') MILESTONE_NUMBER=$(glab api "projects/$PROJECT/milestones" \ --method POST \ --field title="ITER-YYYYMMDD \| {iteration_goal}" \ --field description="<!-- MANIFEST -->$(cat {manifest_path})<!-- /MANIFEST -->" \ --jq '.iid') echo "Milestone: #$MILESTONE_NUMBER" ``` ## Step 4 — Create Issues (one per scenario) For each scenario in manifest order: GitHub: See GitHub Issue Operations skill — Pattern 2. GitLab: See GitLab Issue Operations skill — Pattern 2. Issue body must include inline: - `<!-- SCENARIO -->` YAML block - `<!-- FEATURE_EXECUTION_GRAPH -->` YAML block (full graph for this scenario) - Context Map table - Do Not Do list - SAO.md Sections - Implementation Plan (full text from BPE-01) - Acceptance Criteria checklist — one checkbox per graph node plus scenario rollup checkpoint Labels: `status-queued, parallel-group-{X}` Dependency issues reference each other in body. ## Step 5 — Update Manifest with Milestone Number and Issue Numbers After the milestone is created (Step 3) and all issues are created (Step 4), update the manifest: ```bash # Write milestone_number into the iteration block python3 -c " import re content = open('{manifest_path}').read() content = re.sub(r'milestone_number: null', 'milestone_number: {milestone_number}', content, count=1) open('{manifest_path}', 'w').write(content) " # Also add github_issue or gitlab_issue per scenario (as before) # Edit the manifest YAML to add the issue number under each scenario key ``` Commit both updates together: ```bash git add docs/plans/iterations/ git commit -m "chore(pin): add milestone #{milestone_number} and issue numbers to manifest ITER-YYYYMMDD" ``` Verification: `milestone_number` in the manifest must now be an integer, not `null`. MIN-01 uses this field to locate the correct manifest. Do not proceed to PIN-05 if it is still `null`. ## Step 6 — Verify GitHub: ```bash gh milestone view {milestone_number} gh issue list --milestone {milestone_number} ``` GitLab: ```bash glab api "projects/:fullpath/milestones" --jq '.[] \| select(.title \| contains("ITER"))' glab issue list --milestone "{milestone_title}" --label "status-queued" ``` ## Success Criteria - Milestone created with full `<!-- MANIFEST -->` YAML block - One Issue per scenario with inline `<!-- SCENARIO -->` + `<!-- FEATURE_EXECUTION_GRAPH -->` + context map + do-not-do + implementation plan - All issues labelled `status-queued` + `parallel-group-{X}` - `milestone_number` written back to manifest (not null) - Local manifest updated with milestone number + issue numbers and committed ## Skills - GitHub Issue Operations (gh CLI) - GitLab Issue Operations (glab CLI)
#5	Human Acceptance	Read the 'PIN-5 Human Acceptance' activity from playbook 'Junior Edda' (workflow 'Plan Iteration'). Follow its guidance when assisting me. ## Purpose Verify CLAUDE.md iteration protocol sections are current, obtain explicit human approval, and deliver the single activation instruction for MIN. This is the final gate — do not proceed without explicit human go. STOP — mandatory human gate. ## Prerequisites - All skeleton commits on iteration branch - Execution manifest finalized (`ITER-.yaml`) with `milestone_number` populated - GitHub/GitLab Milestone + Issues published (PIN-04 complete) ## Steps ### Step 1: Verify CLAUDE.md Iteration Protocol Sections Read the `## Iteration Protocol`, `## Drift Handling`, `## Session Resume Protocol`, and `## Authority Model` sections of CLAUDE.md. Required sections must exist and contain: - Iteration Protocol* — references `<!-- MANIFEST -->` block, `status-queued`, fresh subagent per `feature_execution_graph` node with pytest gates - Drift Handling — absorbed and escalated thresholds (checkpoint_fail retry once; escalate on footprint_violation, method_explosion, sao_violation) - Session Resume Protocol — find `status-in-progress` → re-run checkpoint → PASS: close and continue / FAIL: retry - Authority Model — can-decide / must-escalate / cannot-do bounds for the iteration runner If any section is missing or substantially absent: update CLAUDE.md and commit: ```bash git commit -m "docs(claude): sync iteration protocol to current MIN workflow" ``` Do NOT compare against a `doctrine_version` field in the manifest — protocol version is managed by the Edda playbook release, not the manifest. ### Step 2: Present Review Summary ``` === PIN COMPLETE — REVIEW REQUIRED === Iteration: {iteration_goal} Milestone: #{milestone_number} — {milestone_title} Scenarios: {N} \| Groups: {A,B,...} {for each scenario:} S{N} [{group}] {title} Checkpoint: {command} Graph nodes: {N} (BPE-02...06) Footprint: {N} files Depends on: {deps or "none"} Sample: review one scenario's feature_execution_graph in Plan mode Conflict map: {file} -> [{S_N}, {S_M}] (serialized) {or "No conflicts — all scenarios parallel"} Platform: {github\|gitlab} Milestone: #{milestone_number} CLAUDE.md protocol: {current / updated at {timestamp}} Activate MIN with: "Work on Milestone #{milestone_number}" ====================================== ``` ### Step 3: Await Human Decision GO — "Approved" or "Work on Milestone #{N}" ```bash # GitHub: gh issue comment {first_issue} --body "PIN approved at {timestamp}. MIN authorized." # GitLab: glab issue note {first_issue} --message "PIN approved at {timestamp}. MIN authorized." ``` NO-GO options: - Skeleton issue on S{N} → return to PIN-03 for that scenario only - Re-sequencing needed → return to PIN-03 Step 8-9 to revise conflict map - Drop S{N} → close issue (won't-fix), remove from manifest, update Milestone description ## Success Criteria - CLAUDE.md iteration protocol sections are current (no doctrine_version comparison) - Explicit human go/no-go decision obtained - If go: approval comment posted on GitHub/GitLab - Human has the activation instruction with the concrete milestone number ## Rules - Never skip this gate regardless of time pressure - If human is unavailable: pause PIN, do not proceed to MIN - A no-go with a fix takes minutes; a bad MIN run wastes an hour
Test Automation Framework
#7	Add Step and/or Fixture to Library	Read the 'TFK-7 Add Step and/or Fixture to Library' activity from playbook 'Junior Edda' (workflow 'Test Automation Framework'). Follow its guidance when assisting me. # Add Step and/or Fixture to Library ## Objective On-demand activity invoked when ESM-05 "Write Feature Files" or BPE cannot assemble a scenario from existing steps/fixtures. Creates new reusable step definitions and/or fixtures following library conventions, then updates the catalogs. --- ## Trigger Called by other workflows when: 1. ESM-05 is writing a .feature and cannot find a matching step in the Step Library 2. BPE-04 needs a fixture preset that doesn't exist 3. A new domain area requires new step patterns 4. BPE-02 / MIN-04 declares a PRF-* row in SAO §17 but `tests/fixtures/llm_scripts/<prf-id>/` has no happy/reject script — add ScriptedLLM queue fixture(s) and catalog row (lane 1; pytest, not Gherkin) 5. DTA-19 / BPE adds a TASK-* golden row but `tests/fixtures/agent_tasks/<task-id>/` is missing input corpus, `expected.yaml`, or oracle `meta.yaml` — add frozen task fixture and catalog row (lane 4) 6. An existing PRF script needs a new reject_* variant (429, parse fail, HITL block, exhausted queue) per artifact 56 adverse sketch ## Process ### 1. Assess What's Needed - Identify the gap: missing step pattern, missing fixture, or both - Check if an existing step can be generalized/parameterized instead of creating a new one - For PRF/TASK triggers: confirm the gap is fixture/oracle catalog — not a missing Gherkin step (see TFK-03 footnote) ### 2. Create Step Definition (if needed) Follow library conventions: - Place in appropriate domain file (navigation, forms, tables, auth, assertions) - Use `data-testid` selectors - Parameterize generically - Log actions at INFO level Skip step creation when trigger is PRF script or TASK golden only. ### 3. Create Fixture (if needed) Follow library conventions: - JSON for static reference data → `tests/fixtures/presets/` - FactoryBoy for dynamic data → `tests/fixtures/factories/` - PRF ScriptedLLM queue → `tests/fixtures/llm_scripts/<prf-id>/` (`happy.json`, `reject_.json`) - TASK golden task* → `tests/fixtures/agent_tasks/<task-id>/` (`input/`, `expected.yaml`, `meta.yaml` per TFK-04 oracle schema) - Document scope and dependencies ### 4. Update Catalogs - Add new step to Step Library Catalog (GUI/AT only) - Add new fixture to Fixture Library Catalog - For agent fixtures: include `prf_id` / `task_id`, `oracle_type`, `pass_band`, `linked_test`, `sao_ref` ### 5. Verify - Write a small test scenario using the new step/fixture - Run it to verify it works - Ensure existing tests are not broken - PRF: run `pytest -m agent_proof` on the linked test; TASK: run `pytest -m quality` on the linked test (local or `agent-eval.yml` — not default PR until lane rules say so)
#8	Validate Test Health	Read the 'TFK-8 Validate Test Health' activity from playbook 'Junior Edda' (workflow 'Test Automation Framework'). Follow its guidance when assisting me. # Validate Test Health ## Objective Periodic health check of the test suite: analyze coverage quality (not just quantity), flag anti-patterns, identify tests that need strengthening, and verify all test levels pass. --- ## Process ### 1. Run Coverage Analysis ```bash make test-coverage ``` Review coverage report, but remember: coverage is meaningless without quality. ### 2. Quality Audit — Flag Anti-Patterns Check for typical mistakes: Useless Tests: - Testing default Model methods (`model.save()`, `model.delete()`, `model.__str__()`) - Testing Django auto-generated views without custom logic - Tests that only check return codes without verifying side effects Testing Implementation, Not Behavior: - Test checks `save_changes()` returns OK but never verifies changes actually persisted - Test is tightly coupled to implementation details Fat Controller Signal: - Logic in views/controllers instead of services → tests are hard to write - Guideline: Services do the thinking → Views package and provide formatted data → Templates render it Mocking Violations: - Any mock/patch in integration tests → violation of no-mock rule ### 3. Verify All Test Levels Pass ```bash make test # unit + integration make test-at # acceptance make test-e2e # E2E (if staging available) ``` When SAO §17 applies: ```bash make test-agent-proof # lane 1 (when AGENTS_ENABLED=true) make test-agent-live # lane 3 (nightly / manual) make test-agent-quality # lane 4 (promotion band) ``` ### 4. Produce Health Summary Categorize findings as CRITICAL / WARNING / INFO. --- ## Agent eval health audit (when SAO §17 applies) Skip this subsection when SAO §17 is N/A. ### FakeLLM / ScriptedLLM script hygiene \| Check \| Severity \| Action \| \|-------\|----------\|--------\| \| FakeLLM or test double returns `[]` / empty string on queue exhaustion \| CRITICAL \| Replace with explicit raise; add `reject_exhausted` script variant \| \| Script turn count < PRF test's expected agent loop iterations \| WARNING \| Extend `llm_scripts/<prf-id>/happy.json` \| \| Live LLM used in `@agent_proof` test \| CRITICAL \| Move to `@live_llm` or switch to ScriptedLLM \| \| `assert_agent_story` beats missing for declared PRF row \| CRITICAL \| Add happy/reject test per rule `do-assert-agent-story` \| ### Tool / domain mocks in `@agent_proof` \| Check \| Severity \| Action \| \|-------\|----------\|--------\| \| `@patch` on ToolExecutor, broker, or domain service in `@agent_proof` \| CRITICAL \| Remove patch — lane 1 requires real DB + real tool registry \| \| Asserting assistant chat prose instead of trace beats \| WARNING \| Refactor to `assert_agent_story` control-plane keys \| \| Lane 2 deterministic test uses ScriptedLLM \| INFO \| Use plain pytest — no LLM for D0/parse/validate \| Carve-out: only CAP-004 ScriptedLLM may stand in for the LLM in integration tests (rule `do-not-mock-in-integration-tests` ALTER). ### PRF coverage vs SAO §17 Build a matrix from SAO §17 PRF checkbox table vs repo: \| SAO §17 PRF row \| `llm_scripts/` present \| `@agent_proof` test \| happy + reject \| Status \| \|-----------------\|------------------------\|---------------------\|----------------\|--------\| \| e.g. PRF-SC01-04 \| yes/no \| yes/no \| yes/no \| PASS / GAP \| CRITICAL gaps: in-scope PRF row with no linked test or missing reject variant when artifact 56 marks adverse. WARNING gaps: test exists but catalog row in Fixture Library missing `sao_ref` / `linked_test`. ### TASK coverage vs SAO §17 (lane 4) \| SAO §17 TASK row \| `agent_tasks/` oracle \| `@quality` test \| pass_band documented \| Status \| \|------------------\|----------------------\|-----------------\|----------------------\|--------\| \| e.g. TASK-SC02-01 \| yes/no \| yes/no \| yes/no \| PASS / GAP \| Lane 4 gaps are WARNING until promotion window — do not fail PR CI for missing TASK tests. Include agent audit findings in the TAF-08 health summary with lane tags (L1–L4) for TFK-09 scoreboard input.
#9	Prepare Test Report	Read the 'TFK-9 Prepare Test Report' activity from playbook 'Junior Edda' (workflow 'Test Automation Framework'). Follow its guidance when assisting me. # Prepare Test Report ## Objective Analyze test coverage, last run results, and TAF-08 health findings to produce an actionable test report. Outcomes are turned into GitHub Issues assigned for implementation — closing the feedback loop between test health and development work. --- ## Process ### 1. Gather Inputs - TAF-08 "Validate Test Health" findings - Last CI/CD run results (pass/fail per stage, duration, flaky test history) - E2E screenshot archive - `tests.log` analysis When SAO §17 applies, also gather: - Last `agent-eval.yml` run (lanes 3–4): pass/fail, provider, model pin, duration - Lane 1 PR `test-agent-proof` history when `AGENTS_ENABLED=true` - TFK-08 PRF/TASK coverage matrix vs SAO §17 ### 2. Analyze Coverage Gaps For each area with insufficient coverage: - Identify which scenarios from .feature files lack corresponding tests - Identify services/methods with zero or low coverage - Assess risk and prioritize For agent projects: cross-check SAO §17 PRF/TASK rows against green tests (TFK-08 matrix). ### 3. Analyze Quality Issues \| Category \| Examples \| Issue Type \| \|----------\|----------\|------------\| \| Missing tests \| Service method with no test \| Scenario issue \| \| Weak tests \| Tests check return code but not side effects \| Enhancement issue \| \| Fat controllers \| Logic in views needing extraction \| Refactoring issue \| \| Mocking violations \| Mocks in integration tests \| Bug issue \| \| Flaky tests \| Non-deterministic pass/fail \| Bug issue \| \| Agent proof gaps \| PRF row unchecked vs repo \| Scenario issue (lane 1) \| \| FakeLLM hygiene \| Silent queue exhaustion \| Bug issue (lane 1) \| \| Task band regression \| recall below certified pass_band \| Enhancement issue (lane 4) \| ### 4. Produce Report and Create Issues For each finding, create a GitHub Issue with: - What: specific test/file/scenario that needs work - Why: what risk the gap exposes - How: concrete action - Effort: easy/medium/hard - Priority: critical/important/nice-to-have --- ## Agent eval scoreboard (lanes 3 & 4) When SAO §17 applies, append a Lane 3/4 scoreboard section to the test report. Omit when agents N/A. ### Lane 3 — Live contract eval \| Run id \| Date \| Provider / model \| Tests run \| Pass \| Fail \| Flake \| Notes \| \|--------\|------\|------------------\|-----------\|------\|------\|-------\|-------\| \| … \| … \| e.g. anthropic/claude-* \| `@live_llm` count \| \| \| \| temp=0; contract asserts only \| Track: tool-call envelope shape, rate-limit handling, timeout bounds. Not a merge gate — trend for infra/prompt regressions before promotion. ### Lane 4 — Task performance quality \| Task id \| Oracle type \| Pass band \| Last N runs \| Recall / metric \| Extras \| Certified \| vs prior cert \| \|---------\|-------------\|-----------\|-------------\|-----------------\|--------\|-----------\|---------------\| \| TASK-SC02-01 \| set_recall \| ≥0.95, extras≤1 \| 3 \| 0.97 \| 0 \| ✅ \| +0.02 \| Certification rule: promote only when all in-scope TASK rows meet `pass_band` for `n_runs` consecutive evals at temp=0. Regression vs last certified band → open GitHub Issue (priority: important) with task id and metric delta. ### Combined agent summary (for PIN / release notes) \| Lane \| CI home \| Last green \| Open gaps (from TFK-08) \| Release impact \| \|------\|---------\|------------\|---------------------------\|----------------\| \| 1 PRF \| optional PR step \| … \| … \| blocks merge if enabled + red \| \| 2 shell \| `ci.yml` integration \| … \| … \| blocks merge \| \| 3 contract \| `agent-eval.yml` \| … \| … \| advisory / optional CD \| \| 4 quality \| promotion band \| … \| … \| blocks promote if cert required \| Production signals (outside pytest): optionally note HITL reject rate, edit-before-approve ratio, learning-rule growth — feeds PIN-02; reference skill Agent Task Eval § Production quality. Feed scoreboard CRITICAL/WARNING rows into §4 issue creation with labels `agent-lane-3`, `agent-lane-4`, or `agent-prf-gap` as appropriate.