LEVRG / 06 AI Tactics

RAG brain. Structured prompts. Agent chains.

All agent tasks are issued as JSON-structured prompts — enforcing context, output format, and constraints. Reduces hallucination and enables pipeline chaining across agents.

⌘

Graph / RAG

Knowledge base · graphified across notes, ADRs, and code. Vector index refreshed nightly.

12,418 docs · 3.2M tokens

▲

Claude Code

Primary dev assistant — feature scaffolds, refactors, code review, and architecture sketches.

1,847 tasks · last 30d

◆

Codex · MCP

Codegen via MCP delegation. Handles boilerplate, tests, and well-scoped diffs in parallel.

624 tasks · last 30d

JSON-structured prompt · agent task schema

pattern · enforced via Pydantic at agent boundary

// agents/housing_etl.json
{
  "task": "housing_etl_ingest",
  "context": {
    "sources": ["zillow_api", "fred_api"],
    "target": "postgresql.housing_timeseries",
    "schedule": "celery-beat nightly 02:00 UTC"
  },
  "output_format": {
    "type": "pandas_dataframe → db_upsert",
    "validate": ["no_nulls", "date_range_check"]
  },
  "constraints": ["idempotent", "redis_cache_1h"],
  "budget": { "tokens": 8000, "retries": 2 }
}

Every agent task is wrapped in this structure instead of free-text instructions. 'context' tells the agent exactly what it's working with. 'output_format' tells it what to produce and how to validate it. 'budget' hard-caps token and retry spend — without it, a misbehaving agent can run indefinitely and rack up cost.

Agent task ledger · last 24h

agent-log topic · last 24h

98.4% success

Time	Agent	Task	Tokens	Status
14:42	claude	refactor.candle_renderer	4,218	ok
14:21	codex	test.gen.orderbook_ws	2,104	ok
13:58	claude	adr.draft.018	6,884	ok
13:32	codex	migration.diff.user_acl	1,948	ok
12:14	claude	review.pr.842	8,012	ok
11:48	codex	boilerplate.celery_task	1,204	ok
11:01	claude	refactor.lstm_pipeline	9,640	retry
10:34	codex	test.gen.fred_client	1,812	ok

Token count is cost — 8,000 tokens for a code review is normal; watch for tasks that balloon past 10K without producing output. The retry on the LSTM refactor means the agent hit a validation error on its first attempt and re-ran with the error injected as context — a feature, not a bug.

Operating tactics

how the agents are deployed in practice

01 — Context discipline

RAG-first, not chat-first

Every prompt is hydrated from the knowledge graph before it reaches a model. No "remember earlier?" — the context is rebuilt deterministically each call.

02 — Schema enforcement

Output as data, not prose

Models emit JSON matching a Pydantic schema. Anything unparseable is auto-retried with the validation error injected as next-turn context.

03 — Pipeline chaining

Agents are Celery tasks

Each agent invocation is a Celery task with idempotency keys. Failures retry with exponential backoff; outputs are checkpointed in Postgres.

04 — Budget & guardrails

Token caps per task

Every agent task declares a token budget and a retry budget. Overruns hard-fail and bubble to the alerting channel rather than silently degrading.

05 — Eval as first-class

Golden set + nightly diff

A frozen set of prompts re-runs nightly. Output diffs against last week's flag regression before any model upgrade ships to PROD.

06 — Human-in-the-loop

PRs, not commits

Agents never push to main. They open PRs with provenance metadata; CI runs the same gates as human PRs before reviewer sign-off.

The six principles that prevent the agent pipeline from going off the rails. RAG-first prevents guessing. Schema enforcement means outputs are always machine-parseable. Celery tasks means agents are resumable and observable. Budget caps prevent runaway spend. Eval sets catch regressions before model upgrades ship. PRs not commits keeps a human in the loop.