agentic-llm on tomrochette.com

ARIS

tom@tomrochette.com (Tom Rochette) — Sat, 10 Oct 2026 00:00:00 +0000

ARIS (Auto-Research-In-Sleep) is wanshuiyin’s MIT-licensed set of markdown-defined skills that turns Claude Code, Codex CLI, or another coding agent into an autonomous ML research loop, with a reviewer model from a different family critiquing every artifact at gated passes.

ARIS is this category’s first member whose judge is required to be a different model family: the executor drives and an independent reviewer (Codex MCP by default) demands revisions, an arrangement its technical report builds against the category’s shared failure mode, the plausible unsupported success.

What it is
#

A methodology shipped as 82 markdown skills (83 in the standalone CLI bundle) with no framework and no lock-in, created 2026-03-10. The technical report (arXiv 2605.03042, May 2026, by Ruofeng Yang, Yongcan Li, and Shuai Li) describes three layers: an execution layer with more than 65 reusable markdown skills, MCP model integrations, and a persistent research wiki; a review layer where a reviewer from a different model family critiques intermediate artifacts and requests revisions; and deployment experience from long unattended runs. The skills cover the research arc end to end: idea discovery, experiment queues (including SSH multi-seed job queues), paper writing, citation audits, integrity forensics, and Overleaf sync. Adapters exist for Claude Code, Codex CLI, Cursor, Trae, Antigravity, GitHub Copilot CLI, OpenClaw, and DeepSeek Harness. Beyond the skills, the project ships a standalone Rust CLI (ARIS-Code, v0.4.28 on 2026-09-28, mid a 24-release train) and Claude Code, Codex CLI, and DeepSeek Harness plugins.

Status
#

Active: 17,250 stars, 1,446 forks, created 2026-03-10, last push 2026-10-07, as of 2026-10-10.

The Western discussion footprint is zero: a Hacker News search for the repository name returns no stories as of 2026-10-10, and adoption runs through a PaperWeekly feature and the Chinese-language community, the channel pattern DeepAnalyze shows; the Agon paper’s own count puts ARIS at 79 roles and 1,157.4 KiB of prompts, roughly four times Agon’s surface.

Strengths
#

The cross-model rule attacks a failure the kernel columns never face: a reviewer from a different family is less likely to inherit the executor’s framing, which is where the report locates unsupported claims.
The skill set covers the whole arc, including integrity tooling (citation audit, fabrication forensics) most research loops leave to the human.
No lock-in: the same markdown skills port across eight harnesses, so the loop survives any single vendor’s churn.
The maintenance cadence is exceptional: 24 CLI releases between May and September 2026, including a same-week bridge when codex-cli 0.154 removed the reviewer’s entry point.

Cautions
#

The report’s deployment claims are self-published, and no independent replication surfaced in this run’s searches.
The judge is still a model: cross-model review raises the bar but is not a kernel, a grader, or a benchmark.
The prompt surface is large (79 roles, 1,157.4 KiB by the Agon paper’s count), which makes behavior harder to inspect than Agon’s 18 roles.
The zero-story Hacker News footprint says the 17.2k stars measure attention in one language community, not broad engineering adoption.
The loop depends on moving harness internals, as the codex-cli 0.154 breakage demonstrated.

Pricing
#

Free and MIT; there is no paid tier, so pricing does not apply. Costs are your own model subscriptions or API keys (Claude executor plus a Codex reviewer by default, with ModelScope-hosted and local-model combinations documented) and your compute.

Compared to
#

Agon: the Claude Code plugin running producer-critic factories; ARIS is the skills-based loop whose reviewers come from other model families and which ports across harnesses.
Karpathy Autoresearch: the founding keep-or-revert loop whose judge is a measured loss; ARIS’s judge is another model’s critique.
OpenResearch: the workspace that turns your own agents into researchers with an archived evidence tree; ARIS adds adversarial review gates to the same idea.

Bottom line
#

Recommended for ML researchers who want an unattended experiment loop with cross-model review, on the harness they already use, and for studying review-gate design against plausible unsupported successes. Not for anyone who needs a machine verifier behind results, a small inspectable prompt surface, or an independently replicated evaluation.

Changes
#

2026-10-10 - Created.

References
#

https://github.com/wanshuiyin/Auto-claude-code-research-in-sleep - repository and README: skill set, harness adapters, executor-reviewer split, release train (fetched 200, 2026-10-10)
https://api.github.com/repos/wanshuiyin/Auto-claude-code-research-in-sleep - stars, forks, created and pushed dates, MIT license, and topics for the as-of status (fetched 200, 2026-10-10)
https://raw.githubusercontent.com/wanshuiyin/Auto-claude-code-research-in-sleep/main/README.md - the README source: 82 skills, adapter list, ARIS-Code v0.4.28 details, and the codex-cli 0.154 bridge note (fetched 200, 2026-10-10)
https://arxiv.org/abs/2605.03042 - the technical report: cross-model adversarial collaboration, the plausible-unsupported-success failure mode, and the three-layer architecture (fetched 200, 2026-10-10)
https://api.github.com/repos/wanshuiyin/Auto-claude-code-research-in-sleep/releases?per_page=5 - the release train through v0.4.28 (2026-09-28) (fetched 200, 2026-10-10)
https://hn.algolia.com/api/v1/search?query=claude-code-research-in-sleep - the zero-story Hacker News footprint (fetched 200, 2026-10-10)

AutoResearchClaw

tom@tomrochette.com (Tom Rochette) — Sat, 10 Oct 2026 00:00:00 +0000

AutoResearchClaw is the MIT-licensed 23-stage pipeline from the aiming-lab organization that turns a one-line research idea into a compile-ready paper, with sandbox experiments, multi-agent debate, a four-layer citation-verification layer, and seven human-in-the-loop modes.

AutoResearchClaw is the idea-to-paper pipeline that treats failure and fabrication as first-class problems: its executor heals through Pivot/Refine loops, its citations pass arXiv, CrossRef, DataCite, and LLM checks before delivery, and its 54.7 percent win over AI Scientist v2 is measured on its own ARC-Bench, a self-run benchmark the README has since widened from 25 to 55 topics.

What it is
#

A Python pipeline (created 2026-03-15) run from the researchclaw CLI, standalone, through an OpenClaw bridge to Discord, Telegram, Lark, or WeChat, or on any ACP-compatible agent backend (Claude Code, Codex CLI, Copilot CLI, Gemini CLI, Kimi CLI). The arXiv paper (2605.20025, by Jiaqi Liu, Shi Qiu, Mairui Li, Bingzhou Li, Haonian Ji, Siwei Han, Xinyu Ye, and Peng Xia) presents five mechanisms: structured multi-agent debate, a self-healing executor with a Pivot/Refine decision loop, verifiable result reporting against fabricated numbers and hallucinated citations, human-in-the-loop collaboration across seven intervention modes, and cross-run evolution that converts past failures into future safeguards. Deliverables per run: a paper draft, conference-ready LaTeX (NeurIPS, ICML, ICLR templates), a BibTeX file with references pulled from OpenAlex, Semantic Scholar, and arXiv, a verification report, sandbox experiment code and metrics, charts, multi-agent reviews, and evolution lessons. MetaClaw adds cross-run learning (pipeline failures become structured lessons injected into later runs), and the skills library loads 20 preloaded skills plus community contributions. The companion ARC-Bench dataset ships on Hugging Face under the AIMING-Lab-UNC account, widened at v0.5.0 (May 2026) to 55 topics across machine learning, high-energy physics, quantum, biology, and statistics.

Status
#

Dormant since August: 14,618 stars, 1,706 forks, created 2026-03-15, last push 2026-08-19, latest release v0.5.0 on 2026-05-20, as of 2026-10-10.

The repository has gone quiet while attention held: no commits in eight weeks and no release in five months as of 2026-10-10, a Hacker News footprint of two stories at two and one points with zero comments, and the eight showcase papers are self-published, so the 14.6k stars currently measure attention the repository is no longer feeding.

Strengths
#

Citation integrity is a design point, not an afterthought: the four-layer check (arXiv, CrossRef, DataCite, LLM) kills fabricated references before delivery, the failure class the no-kernel columns are most exposed to.
Failure handling is architectural: Pivot/Refine turns failed experiments into information, and MetaClaw accumulates them into reusable skills.
The seven-mode human-oversight ladder (full-auto through step-by-step) is the widest in this category, an answer to the trust problem rather than a denial of it.
ARC-Bench gives the loop a rubric-scored eval set spanning five domains instead of a single ML sandbox.

Cautions
#

The 54.7 percent improvement over AI Scientist v2 is self-reported on the project’s own benchmark, with no independent replication surfaced in this run’s searches.
Development is quiet: last push 2026-08-19 and last release 2026-05-20, which is a poor sign for a 23-stage pipeline whose stages depend on external APIs and agent backends.
The two zero-comment Hacker News stories say there is no practitioner debate to check the claims against.
A 23-stage pipeline is heavy to adopt and heavier to fork: the runtime surface (LLM providers, Docker, LaTeX, five agent backends) is wide.

Pricing
#

Free and MIT; there is no paid tier, so pricing does not apply. Costs are your own LLM API keys and compute (a Docker executor with GPU, MPS, or CPU auto-detection).

Compared to
#

Agon: the other idea-to-paper column; Agon is a Claude Code plugin with adversarial critics and a failure taxonomy, AutoResearchClaw a standalone pipeline whose added layer is citation verification and a human-oversight ladder.
ARIS: the markdown-skill loop ported across harnesses with cross-model review gates; AutoResearchClaw is a fixed 23-stage pipeline you run rather than a method you install.
GPT Researcher: the cited-report baseline with no experiments; AutoResearchClaw runs sandbox experiments and ships LaTeX.

Bottom line
#

Recommended for studying how an autonomous pipeline can make citation fabrication and silent failure first-class problems, and for teams that want idea-to-paper runs with explicit human-oversight modes. Not for anyone who needs an actively maintained tool (eight weeks without a commit) or independently verified benchmark claims.

Changes
#

2026-10-10 - Created.

References
#

https://github.com/aiming-lab/AutoResearchClaw - repository and README: 23-stage pipeline, deliverables, OpenClaw and ACP-backend support, release news (fetched 200, 2026-10-10)
https://api.github.com/repos/aiming-lab/AutoResearchClaw - stars, forks, created and pushed dates, MIT license, and topics for the as-of status (fetched 200, 2026-10-10)
https://raw.githubusercontent.com/aiming-lab/AutoResearchClaw/main/README.md - the README source: stage outputs, citation-verification layers, HITL system, ARC-Bench v0.5.0 widening (fetched 200, 2026-10-10)
https://arxiv.org/abs/2605.20025 - the paper: five mechanisms, the 54.7 percent AI Scientist v2 comparison on ARC-Bench, and the seven intervention modes (fetched 200, 2026-10-10)
https://api.github.com/repos/aiming-lab/AutoResearchClaw/releases?per_page=5 - the release train through v0.5.0 (2026-05-20) (fetched 200, 2026-10-10)
https://hn.algolia.com/api/v1/search?query=AutoResearchClaw - the two zero-comment Hacker News stories behind the discussion-footprint claim (fetched 200, 2026-10-10)
https://huggingface.co/datasets/AIMING-Lab-UNC/ARC-Bench - the ARC-Bench dataset page under the AIMING-Lab-UNC account (fetched 200, 2026-10-10)

DeepAnalyze

tom@tomrochette.com (Tom Rochette) — Sat, 10 Oct 2026 00:00:00 +0000

DeepAnalyze is Renmin University of China and Tsinghua’s MIT-licensed 8B-parameter agentic LLM, trained to run the data-science research loop end to end, from raw files to an analyst-grade report, with no harness beyond a code sandbox.

DeepAnalyze is this category’s first member whose loop is carried by a trained model rather than a prompted harness: the plan, act, and self-reflect structure is learned in the weights, which makes it the cheapest loop here to run and the only one with no external verifier anywhere in it.

What it is
#

DeepAnalyze-8B, released October 21, 2025 with a paper (arXiv 2510.16872, online October 19), open weights, code, and a 500,000-example instruction dataset (DataScience-Instruct-500K). Training uses a curriculum-based agentic paradigm that mimics a data scientist’s learning trajectory, with training trajectories synthesized from data sources, so the model learns to question, prepare, analyze, model, visualize, and report without workflow code. Given structured data (databases, CSV, Excel), semi-structured files (JSON, XML, YAML), or unstructured text, it runs open-ended data research and produces analyst-grade reports. Deployment is vllm serve DeepAnalyze-8B plus a WebUI or a Docker-sandboxed WebUI v2, with a hosted API available through HeyWhale key applications. The authors are Shaolei Zhang, Ju Fan, Meihao Fan, Guoliang Li, and Xiaoyong Du (Renmin University of China with Tsinghua), and the ecosystem has widened: DA-Studio, the system behind WebUI v2, was accepted to the VLDB 2026 demonstration track, CoDA-Bench evaluates code agents on data-intensive analytical tasks, DeepPrep extends the loop to data preparation, and EvoOntology adds a self-evolving ontology layer. It served as the official agent for the 2026 China Collegiate Computer Design Contest’s Big Data track.

Status
#

Active: 4,678 stars, 739 forks, created 2025-10-11, last push 2026-09-23, as of 2026-10-10.

The usage footprint is far thinner than the star count: the Hugging Face weights show 203 downloads and 93 likes as of 2026-10-10, and a Hacker News search returns zero stories, so adoption runs through the Chinese data-science community (PaperWeekly, WeChat, the collegiate contest) rather than the Western harness ecosystem.

Strengths
#

The full loop is open and reproducible: weights, code, and training data, at a size one GPU serves.
The paper claims it outperforms workflow-based agents built on the most advanced proprietary LLMs at data tasks.
The ecosystem compounds beyond one repo: a benchmark (CoDA-Bench), a preparation companion (DeepPrep), a VLDB-accepted serving system (DA-Studio), and an ontology layer (EvoOntology).
Deployment is genuinely self-hosted: vllm on your own GPU, sandboxed execution, no external service required.

Cautions
#

The headline claim is self-reported in the paper’s own evaluation, and no independent replication surfaced in this run’s searches.
A single 8B model plans, executes, and judges its own work inside the sandbox, so there is no external verifier anywhere in the loop.
The project homepage ships with its template placeholders unfilled, thin for a project of the claimed importance.
4,678 stars against 203 model downloads and zero Hacker News stories says the stars measure attention, not usage.

Pricing
#

Free and open source: MIT-licensed code, open weights on Hugging Face, and no paid tier, so pricing does not apply. Costs are your own GPU, or a HeyWhale-hosted API key, which is granted by application with no public prices on the fetched pages.

Compared to
#

GPT Researcher: the prompted web-research loop that cites the open web; DeepAnalyze is data-grounded and carried by a trained model.
Karpathy Autoresearch: the keep-or-revert ancestor whose judge is a measured loss; DeepAnalyze’s judge is itself.
OpenResearch: the workspace that turns prompted coding agents into researchers; DeepAnalyze collapses the loop into one model you can fine-tune.

Bottom line
#

Recommended for teams that want a self-hosted data-analysis agent they can fine-tune, and for researchers studying learned agentic loops against prompted ones. Not for audit-sensitive analysis: the loop’s only judge is the model that wrote the analysis.

Changes
#

2026-10-10 - Created.

References
#

https://github.com/ruc-datalab/DeepAnalyze - repository, README, task surface, deployment paths, and the ecosystem news timeline (fetched 200, 2026-10-10)
https://api.github.com/repos/ruc-datalab/DeepAnalyze - stars, forks, created and pushed dates, and the MIT license for the as-of status (fetched 200, 2026-10-10)
https://arxiv.org/abs/2510.16872 - the paper: title, authors, October 19 2025 online date, curriculum-based agentic training, and the frontier-workflow-agents comparison claim (fetched 200, 2026-10-10)
https://huggingface.co/RUC-DataLab/DeepAnalyze-8B - the open weights page (fetched 200, 2026-10-10)
https://huggingface.co/api/models/RUC-DataLab/DeepAnalyze-8B - the 203 downloads and 93 likes usage footprint (fetched 200, 2026-10-10)
https://ruc-deepanalyze.github.io/ - the project homepage, fetched with its template placeholders unfilled (fetched 200, 2026-10-10)
https://hn.algolia.com/api/v1/search?query=DeepAnalyze - the zero-story Hacker News footprint (fetched 200, 2026-10-10)

agentic-llm on tomrochette.com

ARIS

What it is
#

Status
#

Strengths
#

Cautions
#

Pricing
#

Compared to
#

Bottom line
#

Changes
#

See also
#

References
#

AutoResearchClaw

What it is
#

Status
#

Strengths
#

Cautions
#

Pricing
#

Compared to
#

Bottom line
#

Changes
#

See also
#

References
#

DeepAnalyze

What it is
#

Status
#

Strengths
#

Cautions
#

Pricing
#

Compared to
#

Bottom line
#

Changes
#

See also
#

References
#

agentic-llm on tomrochette.com

ARIS

What it is #

Status #

Strengths #

Cautions #

Pricing #

Compared to #

Bottom line #

Changes #

See also #

References #

AutoResearchClaw

What it is #

Status #

Strengths #

Cautions #

Pricing #

Compared to #

Bottom line #

Changes #

See also #

References #

DeepAnalyze

What it is #

Status #

Strengths #

Cautions #

Pricing #

Compared to #

Bottom line #

Changes #

See also #

References #

What it is
#

Status
#

Strengths
#

Cautions
#

Pricing
#

Compared to
#

Bottom line
#

Changes
#

See also
#

References
#

What it is
#

Status
#

Strengths
#

Cautions
#

Pricing
#

Compared to
#

Bottom line
#

Changes
#

See also
#

References
#

What it is
#

Status
#

Strengths
#

Cautions
#

Pricing
#

Compared to
#

Bottom line
#

Changes
#

See also
#

References
#