Agents in the Wild

This directory is no longer updated. Last update: November 16, 2026.

GitHub's Qubot lets staff query the data warehouse in plain language, with hundreds of users

GitHub built Qubot, an internal Copilot-powered analytics agent available in Slack, VS Code and the Copilot CLI. GitHub found that structured, curated context made it more accurate and three times faster at reaching the right answer.

Organization
GitHub
Industry
Developer Tools
When
Jun 2026
Evidence
Company reported

Reported outcome

Hundreds of users running thousands of queries. In GitHub's experiments, curated context made Qubot three times faster at returning the right answer.

  • hundreds of users, thousands of queries
    Adoption
  • three times faster to the right answer
    Effect of curated context (experiments)

Caveats

Stated by the company that uses the agent (blog, earnings call, interview). Not independently audited.

  • GitHub gives no exact user count, accuracy rate or time-saved figure.
  • 'Three times faster' compares context setups in GitHub's own experiments. It does not compare Qubot with a human analyst.
  • GitHub sells Copilot, the product this agent is built on.
  • Qubot is for exploratory questions. It does not replace dashboards, and complicated questions still go to the data team.

What the agent does

Qubot lets any GitHub employee ask about any data model in the data warehouse in plain language. It runs in Slack, VS Code and the Copilot CLI.

It defaults to Kusto for recent event data and switches to Trino for complex joins and historical analysis, so users do not need to pick an engine.

What happened

GitHub says Qubot has hundreds of users running thousands of queries, and that questions in its data and analytics Slack channels fell sharply. It gives no number for that drop.

The main lesson was about context. In GitHub's experiments, structured and curated context made Qubot more accurate and three times faster at returning the right answer.

What to copy

  • Treat business definitions and data context as a maintained product.
  • Let the teams that own the data write its context.
  • Send each context change through a pull request and an offline eval.
  • Say what the agent is not for. Qubot does not replace dashboards.

Starter prompt

Written by us, not by GitHub.

Data context (tables, definitions, business rules): {context}
Question: {question}

Use only tables and metrics listed in the context.
Show the query, the result, and the definitions you used.
If the context does not define a metric, say so and ask. Do not invent a definition.

Sources

  1. How we built an internal data analytics agent — GitHub Blog, Jun 19, 2026

Added Nov 9, 2026 · Sources last checked Nov 16, 2026.

Get new use cases weekly

One short email. Sources included.

The newsletter is paused. Past issues stay in the archive.

Related use cases