↓ Skip to main content
tomrochette.com

Agents

Author
Tom Rochette
History / Edit / PDF / EPUB / MD
Created: / Updated: / Status: in progress / Readability: 3 / 9 minutes read (~1756 words)
agents 49 ai 66 experiments 1 llm 68
Table of Contents

Every article in this section is written and maintained by LLM agents, with no human review before publication.

This is an experiment in delegating a slice of this blog to its own subject matter. A scheduled agent run refreshes the section daily: it builds and re-verifies a research index of short tool profiles, works a queue of articles I define, and verifies its own links before pushing.

The rules it operates under are public: the operating instructions. The how is public too: the methodology. The audit trail of every change it makes is public: the log. If the section turns into slop, the logs will show exactly where it went wrong, which is half the experiment.

Essays and trackers
#

Essays appear here as the daily agent runs publish them. The queue it works from is the work queue.

Comparison matrices
#

Every research category’s members compared on shared rows, plus the model providers compared as bundles and the model benchmarks compared as instruments.

  • Assistant Runtimes Feature Matrix - OpenClaw, Hermes, the shrinking variants, the Python core, the three Cowork desktops (OpenWork, Eigent, and the governed OpenWorker), the local-first platforms (Open WebUI, AnythingLLM, PrivateGPT), AstrBot the IM-platform veteran, and Octop the multi-user IM-first Tencent entrant the newest column, the trust ladder in one table, verified 2026-10-13.
  • Automated Research Feature Matrix - the thirty-one lab, product, and harness research loops divided on who runs the loop and who judges the output, from Lean kernels and official graders to critic agents, grader-isolated agent organizations, and a bank account, Karpathy Autoresearch the keep-or-revert founding artifact and Sakana AI Scientist the idea-to-paper ancestor, Arbor the held-out merge gate, EvoMap AutoResearch with its forged-commit provenance incident on record, Mole the budget-ledger researcher, AutoContext the promotion-gated self-improvement loop, and data-to-paper the data-chained manuscript pipeline the newest column, verified 2026-10-15.
  • Code Review Feature Matrix - the twelve AI reviewers divided on where your code runs, OpenQodex and OpenReview the newest, with both Kudelski exploit records named and the vendor-run-benchmark paragraph, verified 2026-10-11.
  • Context Engines Feature Matrix - the twenty-one context vendors and tools against delivery, deployment, and scale rows, code-review-graph the Tree-sitter context graph, Vanna the archived text-to-SQL ancestor, Entroly the receipt-backed compression layer, SoL-Pi the NV Labs Pi-harness efficiency extension, and Repowise the AGPL-3.0 local code-intelligence index the newest, verified 2026-10-13.
  • Control Planes Feature Matrix - governance, budgets, approvals, trust, settlement, and audit rows across eighteen members, from Paperclip’s agent company past its 100k stars to AgentField’s DID-identified agent backend the newest, verified 2026-10-13.
  • Evaluation and Review Feature Matrix - the seventeen quality-control columns divided on who judges, the agent, the metric suite, the deterministic gate, the platforms, the humans, the workbench, the benchmarks, or the workflow around the diff, Kitaru’s replay-based eval platform and MLR-Bench’s research-agent judge the newest, verified 2026-10-14.
  • Executions Feature Matrix - subscription features versus self-hostable infrastructure across trigger and execution rows, the official-Actions trio (Claude Code Action, Codex Action, Run Gemini CLI), Airut, and Rivet Agents the durable process-per-agent runtime the newest, verified 2026-10-14.
  • Harness Feature Matrix - the forty-two harnesses against twelve capability rows, Mini-SWE-agent, Oh My Pi, and Atomic Agent joined by Avante.nvim in the editor, OmO the 70k-star senpi fork, and AutoDev, gptme, and LongHorizon-Harness the newest, verified 2026-10-13.
  • Hybrid Execution Feature Matrix - constrained decoding versus validate-and-retry versus models born at the decision layer, now 30 columns, the two independent boards (JevBench v1.7.46 and the Decision Index 0.3.2) in the maintenance rows, Marvin the AI-functions toolkit the newest entrant, the guarantee mechanism as the deciding row, verified 2026-10-15.
  • Memory Feature Matrix - the file convention, Cabinet’s knowledge base over it, the portable format, the capture plugin, the meta-learning layer, and the services against memory-model and lock-in rows, twenty-eight members with PLUR the newest column, verified 2026-10-14.
  • Model Access Feature Matrix - the twenty-six model access providers (gateways, vendor plans including the Claude, ChatGPT, Google, and Grok subscriptions, flat subscriptions, the local inference engine Magnitude, and the self-hosted layer now spanning LiteLLM, Bifrost, Ollama, LocalAI, llamafile, and Experiential) against billing, entry price, quota form, and price-trajectory rows, LocalAI the newest, verified 2026-10-11.
  • Model Benchmark Matrix - the thirty-three model benchmarks grouped by the decision they inform, each with a one-sentence summary of what it evaluates, Lean eval the newest, verified 2026-10-11.
  • Model Provider Feature Matrix - the seven model providers compared as bundles on price tiers, cache and batch policy, context flatness, weights, and subscription transfer, verified 2026-10-13.
  • Orchestration Feature Matrix - the sixty-nine members: worktree managers, dashboards, mission controls, boards, CLI workflow runners, mobile clients, the multi-agent frameworks (CAMEL, AG2, CrewAI, Agno, Dify, Mastra, Sim, Squad, AutoGPT, AutoGen, LobeHub, MetaGPT, LangGraph) plus the self-evolving EvoAgentX, herdr the terminal-runtime daemon, AgentBox’s five-cloud box teleporter, Symphony the 10-10 entrant, the Coolify team’s local-plus-server dev environment Jean, Gas City the orchestration-builder SDK extracted from the dead Gas Town, Orkas the commander-and-specialist desktop, Flowise’s post-Workday archive the genre’s death record, Open Swarm pivoted to a proprietary company product, 1Code archived by its owner, and one agent town shut down, one deprecated and one orphaned among them, verified 2026-10-12.
  • People and Publications Feature Matrix - the thirty-three voices compared on focus, cadence, and reader slot, Cole Medin the build-your-own-agent video educator the newest, verified 2026-10-15.
  • Protocols Feature Matrix - the thirteen protocols stack rather than compete, PIC Standard the newest, and adoption falls with every step up the stack, verified 2026-10-13.
  • Retrieval Feature Matrix - the hosted parsing pipeline, the chunking library, the document parser, the two frameworks, the self-hosted code index, the RAG engines RAGFlow and R2R, the two graph-RAG carriers, the vectorless argument, and the code-structure bracket, R2R the newest, verified 2026-10-13.
  • Sandboxing Feature Matrix - the twenty-two isolation layers divided into boundaries, two orchestrators, a framework, and provisioning, Container Use, Brood Box, and aicontainer the newest, verified 2026-10-12.
  • Session Analytics Feature Matrix - the archive, the attribution CLI, the semantic-search resumer, the replay compiler, the live dashboard, the hosted OpenClaw tracer, the agent-readable analytics layer, the two spend auditors, the token-ledger trio, Claude Code Agent Monitor the control room, Coucou the notch-and-iPhone approver, and git-ai the git-layer provenance recorder now under OpenAI the newest, sixteen columns verified 2026-10-13.
  • Skills Feature Matrix - the spec, the curated packs (Agent-Native, Headcount), the vendor format, the harness mechanism, the optimizer, the security scanner, the registries, the packaging standard, and the de-AI writing skill, the GEPA reflective optimizer, and EvoSkill the skill-discovery entrant the newest, verified 2026-10-14.
  • Software Factory Feature Matrix - Fabro’s graph-authored factory, the stamped Python loop, Fluent’s learning loop, Fusion’s board-first loop, HAR’s fleet harness, Machinist’s controlled entrypoint, Ouroboros’ hidden grading, SuperPlane’s platform factory, Loki Mode’s receipt-gated CLI, and Zeroshot’s independent-review loop on the who-owns-the-loop axis, verified 2026-10-13.
  • Spec Driven Development Feature Matrix - the thirteen spec-first tools across the ownership and ceremony-sizing axes, Arc Kit the enterprise-architecture governance catalog the newest, Spec Kit at v1.1.3 and Superpowers at 7.0, verified 2026-10-13.
  • Surface Feature Matrix - the twenty-three surfaces (two of them death records) against eleven capability rows, CodeNomad and px0 the newest, verified 2026-10-11.
  • Task Management Feature Matrix - files versus database as the deciding row, AgentRQ the realtime human-in-the-loop board the newest column beside PlanDB’s compound graph and Veritas Kanban’s phase-gated authority, with the PRD pipeline and its license cost, verified 2026-10-13.
  • Trackers and Leaderboards Feature Matrix - eleven members compared on what their number measures, Vectara Hallucination Leaderboard the newest, from release timelines to gateway spend, BenchLM and Margin Lab among the established records, with a verification-strength row that inverts the popularity order, verified 2026-10-13.

Research index
#

One structured profile per tool or topic: what it is, status, strengths, cautions, pricing, and when to choose it over its rivals. All categories are refreshed in parallel every run; dead tools keep their entries, marked. Each category keeps its own index page below, listing its entries alphabetically with one-line summaries and the date each was added.

  • Assistant runtimes - personal assistant runtimes outside the editor, and the local-first platforms they run on.
  • Automated research - where the research loop runs autonomously, from the labs’ science programs to productized research agents, formal-proof engines, and open harnesses.
  • Code review - the machines that judge pull requests.
  • Context engines - the engines, packers, and filters deciding what enters the context window.
  • Control planes - governance, budgets, approvals, trust, settlement, and audit above the harness.
  • Evaluation and review - the gates, dashboards, and studies judging agent output and the harnesses themselves.
  • Executions - event-driven execution: hooks, schedules, and automation canvases.
  • Harnesses - the terminal and CLI agents that carry the model into your repo.
  • Hybrid execution - small fast models for typed decisions, and the benchmarks that measure them.
  • Memory - persistent memory, from file conventions to graph and temporal stores.
  • Model access - the gateways, coding plans, and flat subscriptions that sell access to models.
  • Orchestration - worktree managers, kanbans, dashboards, boards, CLI workflow runners, and multi-agent frameworks for running many agents at once.
  • People and publications - the voices steering the domain, and the lens each brings.
  • Protocols - the open standards stacking agents, editors, tools, and frontends together.
  • Retrieval - chunking, parsing, and the frameworks feeding agents the right slices.
  • Sandboxing - isolation layers from the workstation to the cluster.
  • Session analytics - turning agent session logs into searchable history, cost and latency audits, and live views.
  • Skills - the reusable capability format, from spec to registries.
  • Software factory - end-to-end factories owning the loop from spec to verified code.
  • Spec-driven development - specification-first workflows, from brownfield toolkits to platform bets.
  • Surfaces - the editors and IDEs where agents meet your code.
  • Task management - where agent work gets planned and tracked.
  • Trackers and leaderboards - the release trackers, leaderboards, and open datasets that watch the AI field itself.

Notes appear in their category’s index, alphabetically, as the daily agent runs publish them.