Demo harnesses
Each folder is a complete harness: a harness.yaml, the code it declares, its
data, and a README that states what it proves, which capabilities it
exercises, how to run it, what evidence to look for, and what to try next. They
were built — and are changed — only through the CLI (init, add, set,
memory), never by editing harness.yaml by hand.
Offline demos run on a scripted provider shipped in their extensions/:
no API key, and the same journal every time. The scripts never carry the
answer — they react only to what they are actually shown (their prompt, their
tool results, the signal map), so the evidence is real. Live demos need an
API key for their provider (any provider works via --provider/--model).
| demo | proves | runs |
|---|---|---|
| quickstart | a harness with no tools still adds a journal, a hashed spec, spending and turn ceilings, and a safety layer that keeps a credential out of the request, the trace and the answer | live |
| example-summarizer | a model held to a contract — shape, content and a house style it loads as a skill — and retried with feedback until the output earns a success | live |
| article-extractor | code parses, the model judges, and the answer is checked against the live page so invented headings fail | live |
| log-forensics | an allowlisted, confined shell command produces more than the context holds and the harness still answers from its last line | live |
| ranked-retrieval | structure beats model size: enforced phases, verify-first search and grounded ids make a 3B model reliable, measured by a local eval | live |
| ticket-triage | an MCP server as the only data source, reads fanned out in parallel, one validated report with no invented ids | live |
| routing-lab | playbooks change the model and the tools mid-run on a pinned plan; forks re-enter a run at any call; an aimed evolution is confirmed by measurement | offline |
| memory-lab | a small executor works over more data than its context holds — narrowing it in place, keeping notes, exporting by handle — and learns only through review | offline |
| signal-lab | the harness finds where its failures come from, drafts an aimed fix on its own, and keeps a change only when its eval confirms it | offline |
| delegation-lab | a harness hands a task to a peer only once the peer has earned it in measured runs, verifies the answer itself, and otherwise names the peer | offline |
By capability
| capability | where to see it |
|---|---|
journal, version hash, stats |
every demo; quickstart is the minimal case |
| guardrails: cost, wall clock | quickstart, example-summarizer, article-extractor |
guardrails: regex_output_filter, max_turns_hard_cap |
quickstart |
guardrails: tool_allowlist |
example-summarizer, article-extractor |
guardrails: no_network_write |
article-extractor |
redaction (logging.redact) |
quickstart, example-summarizer |
| provider egress screening | quickstart, log-forensics |
| confinement of spawned processes | log-forensics |
verification: output_schema, retry with feedback |
example-summarizer, routing-lab, memory-lab |
| verification: code validators | example-summarizer, article-extractor |
verification: grounded_references |
ranked-retrieval |
verification: regex_match |
ticket-triage, signal-lab, delegation-lab |
verification: file_exists, command_succeeds |
memory-lab |
custom @tool |
article-extractor, ranked-retrieval, signal-lab |
output hooks (strip_json_fence) |
article-extractor, log-forensics, ranked-retrieval |
skills and load_skill |
example-summarizer |
| MCP servers as tools | ticket-triage |
serving a harness (package, serve, mcp serve) |
quickstart, delegation-lab |
loop: plan_then_act |
routing-lab |
loop: sequential_steps |
ranked-retrieval, log-forensics |
| loop: parallel tool execution | ticket-triage |
| playbooks, per-playbook models | routing-lab |
forking and --resume |
routing-lab, memory-lab |
spill, handles, transform_result |
memory-lab, log-forensics |
notes |
memory-lab |
recall_runs |
log-forensics |
durable memory: memory.entries, propose_memory |
memory-lab |
durable memory: relevance selection, search_memory |
signal-lab |
reflection (evolution.reflect) |
signal-lab |
hiveloom signal |
signal-lab, routing-lab |
aimed proposals, evolve --propose, proposals apply |
routing-lab, memory-lab, signal-lab |
| auto-propose with trace excerpts | signal-lab |
evolve --experiment, hiveloom assess |
signal-lab (offline), ranked-retrieval (live) |
| evals, datasets, scorers, metric objectives | ranked-retrieval, signal-lab |
| delegation between harnesses, referrals, lineage | delegation-lab |
best_of_n is experimental and not in a demo: its plurality vote needs
answers that can be compared verbatim, which none of these tasks produce.
http_get with pre-declared hosts is documented in
spec.md.
Verified
scripts/package_e2e.py builds the wheel, installs it in a clean
environment, and drives every offline demo end to end, validating and
dry-running the live ones (a CI step). With --live and an OpenRouter key it
also runs the live demos on small models.