Agents in the Wild

Sakana's AI Scientist-v2 wrote a paper that passed peer review at an ICLR workshop

Sakana AI's agent generated a full research paper, from hypothesis to experiments to write-up. It was accepted at an ICLR 2025 workshop with scores of 6, 7, and 6, then withdrawn as planned.

Organization
Sakana AI
Industry
AI Research
When
Mar 2025
Evidence
Research paper
Agents and tools
The AI Scientist-v2

Reported outcome

One of three fully AI-generated submissions cleared the acceptance bar at the ICBINB workshop (scores 6, 7, 6).

  • 6 / 7 / 6
    Reviewer scores
  • 3 (1 accepted)
    AI papers submitted

Caveats

Described in a paper with a stated method. Peer review status varies.

  • Humans chose the topic and picked which papers to submit.
  • Sakana said none of the papers would meet its bar for the main conference.
  • Outside reviewers reported hallucinations and overstated novelty in the accepted paper.
  • The experiment ran with the organizers' cooperation, and the paper was withdrawn after acceptance.

What the agent does

The agent proposes a hypothesis, writes and runs experiment code, analyzes the results, makes figures, and writes the paper. It uses a tree search over experiments to explore options.

Why it matters

It shows an agent can complete the full research loop at workshop level. It also shows the risk: reviewers caught real errors. Treat agent research as a strong first draft that needs expert checking.

What to copy

  • Use agents to run many small experiments, then have humans pick and check the best.
  • Ask the agent to log every run, so claims can be traced to data.
  • Always check citations and numbers by hand.

Starter prompt

Written by us.

Research question: {question}

1. List 3 testable hypotheses. For each, the experiment and the metric.
2. For the best one, write the experiment code. Use a fixed random seed.
3. Run it 3 times. Report mean and spread in a table.
4. Write a 1-page summary. Every number must come from the log files.
   Mark any claim you cannot support with [UNSUPPORTED].

Sources

  1. The AI Scientist generates its first peer-reviewed scientific publication — Sakana AI, Mar 2025
  2. Sakana claims its AI paper passed peer review, but it's a bit more nuanced — TechCrunch, Mar 2025

Added Oct 16, 2026. Spotted an error? Send a correction.

Get new use cases weekly

One short email. Sources included.

Related use cases