Airbnb migrated 3,500 test files with an LLM pipeline in 6 weeks instead of 1.5 years

Airbnb moved nearly 3,500 React test files from Enzyme to React Testing Library using an automated, step-by-step LLM pipeline with retry loops. Engineers had estimated 1.5 years of manual work.

Organization: Airbnb
Industry: Travel
When: Mar 2025
Evidence: Company reported
Agents and tools: LLM pipeline (custom)

Reported outcome

~3,500 test files migrated in 6 weeks; about 97% fully automated and the rest finished by hand.

~3,500
Files migrated
6 weeks vs ~1.5 years est.
Time
~97%
Automated

Caveats

Stated by the company that uses the agent (blog, earnings call, interview). Not independently audited.

The 1.5-year manual estimate is Airbnb's own.
Going from about 75% to 97% automated took much more effort than the first 75%.

What the agent does

Each test file goes through fixed steps: refactor, fix Jest errors, fix lint, fix TypeScript. After each step, the pipeline runs the check. If it fails, the error goes back to the model and it tries again. Many files run at once.

Key lesson

The team found that many retries worked better than a perfect prompt. Their lead engineer called it "brute force". Feeding the real error back is the most useful context you can give.

What to copy

Break a migration into steps that each have a pass/fail check.
Put a retry loop with the error message on every step.
Expect a long tail. Plan human time for the last few percent.

Starter prompt

Written by us. Use it inside a retry loop.

Convert this test file from Enzyme to React Testing Library.

Rules:
- Test behavior the user can see, not component internals.
- Use screen queries (getByRole first). Avoid test IDs unless needed.
- Keep every existing test case. Do not delete assertions.

{file_contents}

{if retry}: The last attempt failed with this error. Fix only what
the error requires:
{error_output}