Sign in to create and edit playbooks. Sign In Register

Define AI Agent Architecture

DTA-19 Order: #19 Inception Has Dependencies

Updated 4 weeks, 2 days ago

Guidance

Define AI Agent Architecture

Condition: Run only if the system has in-app LLM agents (agentic loops, plan/worker execution, or LLM calls inside the application). Skip and write "Not applicable" in SAO §17 if none of: in-app agentic loop, LLM calls initiated by the app, plan/worker execution.

Reference: Playbook artifact AI Agent Reference Architecture (latest released Edda version).

Objective

Confirm components (Part 1), pick design patterns and scenarios (Part 2), choose serving mode (Part 3), select capabilities (CAP-xxx), choose an assembly template, and record decisions in SAO §17.

Process

  1. Fetch the latest AI Agent Reference Architecture from the playbook.
  2. Read How to use this document and Part 1 — Components; complete infra, model, vector (if needed), framework, memory, and observability checklist in SAO §17.
  3. Pick Part 2 — Design patterns and primary SC-xx scenario(s); use Part 5 — Quick reference for secondary CAP additions.
  4. Choose Part 3 — Serving mode (real-time, batch/queue, or both).
  5. Copy required + chosen optional CAP-IDs into SAO §17 from Part 4.1 Capability table.
  6. Confirm dependencies in Part 4.3 Module wiring.
  7. Note starting assembly template from Part 4.4 (T-01, T-02, T-03, or T-00 custom).
  8. Name integration proof test IDs (PRF-SCxx-xx, PRF-OBS-01) from Part 4.5 as DoD gate.
  9. If both SC-02 and SC-05 are selected, read Pattern 5 · SC-02 × SC-05 · Full rescan invariants and record checklist in SAO §17.
  10. If Pattern 4 / SC-01 with large domain state, decide dual execution path (CAP-037, CAP-038, CAP-039) and memory tiers (Component 5 / CAP-091–094).
  11. Record Pattern 4 anti-pattern checklist (mechanical step execution; worker failure bridge) when CAP-060 selected.
  12. Record decisions for SAO §17 via Write SAO.md (Activity 59).

Decisions to Make

1. Infra layer (Component 1)

Decision Choice Rationale
LLM provider anthropic / openai / ollama / …
Ollama readiness probe at startup Yes / No / N/A
Broker + result backend (CAP-060) Configured / N/A
Secrets server-side only Yes

2. Vector tier (Component 3)

Decision Choice Rationale
Vector store needed Yes / No
Backend pgvector / dedicated / in-process
CAP-090 / CAP-094 selected Yes / No

3. Design pattern & scenario selection

Read Part 2 pattern sections before ticking. Record primary scenario(s) and rationale.

SC-ID Pattern Name Selected? Rationale
SC-01 4 Conversational planner
SC-02 2 Field extractor / batch ingest
SC-03 7 Compiled pipeline
SC-04 3 Event-driven nudge
SC-05 5 Governed mutations

4. Serving mode (Part 3)

Mode Selected? CAP-IDs
Real-time (3.1) CAP-040, CAP-003, CAP-100
Batch / queue (3.2) CAP-060–066
Observability for serving (3.3) CAP-101, CAP-100

5. Dual execution path (Pattern 4 / SC-01 with large state)

Decision Choice Rationale
Context snapshot in chat (CAP-037) Yes / No
Snapshot invalidation on writes (CAP-038) Yes / No
Mutation-only tools in conversation (CAP-039) Yes / No
Full tool registry in plan worker only Yes / No

6. Memory tiers (Component 5)

Tier CAP-IDs Selected? Rationale
1 Hot snapshot 037, 038, 039
2 Semantic search 091
3 AI profile 092
4 Reference library 094, 090
5 Entity notes 093

7. Capability checklist

From Part 4.1 — tick every CAP your project implements:

CAP-ID Name Required for SC? Implement? Module path in project
CAP-001 LLM Port protocol
CAP-037 Context snapshot hydration
CAP-038 Snapshot invalidation hooks
CAP-039 Dual tool exposure
CAP-091 Semantic context search
CAP-092 AI-managed profile memory
CAP-093 Entity notes graph
CAP-094 Reference knowledge library
CAP-101 Agent interaction trace
(copy remaining rows for each selected CAP)

Minimum: all required CAP-IDs from your selected SC-xx row in Part 5 — Scenario index.

8. Assembly template

  • [ ] T-01 Planner (SC-01 / Pattern 4)
  • [ ] T-02 Field (SC-02 / Pattern 2)
  • [ ] T-03 Pipeline (SC-03)
  • [ ] T-00 Custom — list CAP-IDs: ___

9. Agent identities (CAP-121, CAP-122)

Identity Role Model tier Allowed tools
planning / execution / field

10. Agent Blackboard (if CAP-070 selected)

  • Durability tier: A — in-process / B — run-persistent
  • Schema keys: phase, hypothesis, current_plan, last_actions, next_intent
  • Max board size (chars): ______

11. Worker anti-patterns (if CAP-060 selected)

  • [ ] Worker uses LLM per step (CAP-054) — not mechanical tool script (Pattern 4 anti-patterns)
  • [ ] Worker failure bridged to UI (CAP-100 plan_failed and/or chat context injection)
  • [ ] PRF-SC01-06 mapped if hybrid worker is new

12. Observability (Component 6 / CAP-101)

  • correlation_id header/name: ______
  • Cost rate table location: ______
  • PRF-OBS-01 test file: ______

13. Integration proof (DoD gate)

PRF ID Scenario Test file Selected?
PRF-SC02-01 SC-02 thinking JSON
PRF-SC02-02 SC-02 parse fail loud
PRF-SC02-03 SC-02 domain D0 pre-filter
PRF-SC02-04 SC-02 D1 parse fail loud
PRF-SC01-01 SC-01 plan handoff
PRF-SC01-02 SC-01 429 retry
PRF-SC01-03 SC-01 blackboard retain
PRF-SC01-04 SC-01 dual tool exposure
PRF-SC01-05 SC-01 snapshot invalidation
PRF-SC01-06 SC-01 no mechanical worker
PRF-SC01-07 SC-01 worker failure bridge
PRF-SC05-01 SC-05 HITL
PRF-SC05-02 SC-02+05 full rescan invariants
PRF-OBS-01 all correlation + usage trace

14. SC-02 × SC-05 full rescan (if both selected)

From Pattern 5 · SC-02 × SC-05 · Full rescan invariants:

  • [ ] Rescan delete ops meet auto-apply confidence threshold (or rescan disables partial auto-apply)
  • [ ] ChangeSet apply ordering: deletes → updates → adds when rescan flag is set
  • [ ] PRF-SC05-02 mapped to integration test file

15. Scan Skills

Query Skills where capability_domain in: AI_AGENT, LLM_INTEGRATION, ASYNC_TASK, OBSERVABILITY. Report gaps.

16. Task performance quality (TASK-*)

From artifact 56 Part 4.6 — Task performance quality. Mirror the Part 4.5 Integration proof table style. These are lane 4 evals — measure model + prompt + tools against structured ground truth, not orchestration wiring (lane 1) or deterministic parse/validate (lane 2).

TASK ID Scenario Oracle type Pass band Selected? Test file
TASK-SC02-01 SC-02 set_recall recall ≥ 0.95, extras ≤ 1
TASK-SC02-02 SC-02 graph_diff zero unexpected deletes on rescan corpus
TASK-SC01-01 SC-01 plan_steps expected tool sequence + final state
TASK-SC05-01 SC-05 hitl_rate destructive ops blocked until approval
(copy remaining Part 4.6 rows for selected SC-xx)

Eval vs merge DoD:

Gate Lanes When Blocks merge?
Merge DoD 1–2 PR when AGENTS_ENABLED=true Yes — PRF-* under CAP-004 ScriptedLLM; lane 2 deterministic shell in normal integration
Contract eval 3 Nightly / optional CD cert No — temp=0 schema/contract bounds only
Promotion DoD 4 make test-agent-quality / scheduled eval No — regression vs last certified TASK-* band; human promotion decision

Record chosen TASK rows, fixture paths (tests/fixtures/agent_tasks/<task-id>/), and certified pass bands in SAO §17. Full catalog fields (fixture_ref, n_runs, oracle asserts) live in artifact 56 Part 4.6 — link there; do not duplicate prose in SAO.

Deliverables

  • Infra + vector tier decisions documented
  • Primary pattern(s) and SC-xx scenario(s) chosen with rationale
  • Serving mode (Part 3) selected
  • Dual execution path + memory tier decisions (when Pattern 4 / SC-01)
  • CAP-ID checklist complete for selected scenarios
  • Assembly template (T-01 / T-02 / T-03 / T-00) named
  • Agent identities + model tiers documented
  • Worker anti-pattern checklist completed (when CAP-060)
  • Observability trace documented (CAP-101)
  • PRF test IDs mapped to project test files
  • TASK-* rows mapped to golden task fixtures (when lane 4 in scope)
  • SC-02 × SC-05 rescan checklist completed (when both scenarios selected)
  • Skill coverage assessed
  • Decision recorded for SAO §17 via Write SAO.md (Activity 59)
Details
Order:
#19
Phase:
Predecessor:
DTA-17 Define Documentation Strategy
Successor:
DTA-18 Define MCP Architecture
Created:
Jul 17, 2026
Last Updated:
Aug 21, 2026
Workflow
Define Architecture

Analyze ESM artifacts, make architectural decisions across 16 domains (application structure through documentation strategy), scan available Skills for coverage, and …

View Workflow
Assigned Agent

No agent assigned

Rules

No rules linked.

Input Artifacts

No input artifacts

Output Artifacts 1