The problem
How it works
In Cognition’s 18-month review, customers wrote instructions for one kind of change (a framework upgrade, a language migration, or unit tests across a few hundred repositories) and launched many Devin sessions at once, with engineers reviewing the pull requests. Devin’s own documentation gives the same recipe: tasks with clear success criteria such as passing tests or green CI, enough context and examples, and large projects split into sessions that can run in parallel.
Results and limitations
Cognition reports a 67% pull request merge rate (up from 34% a year earlier), file migrations that took 3 to 4 hours instead of 30 to 40 for engineers at one bank, and test coverage rising from 50 to 60% to 80 to 90% at customers. These are vendor-reported figures without published methodology.
Before you try it
By Cognition’s own account Devin performs worse on ambiguous work and when requirements change mid-task, and humans still have to check test logic and review its code. The docs suggest tasks of about three hours or less per session. Results depend on your setup and were not independently reproduced by Top 200 Repos.
Tools and repositories
No ranked repository is involved. This use case still stands on its own source.
- Tools
- Devin · GitHub · CI
- Prerequisites
- A Devin plan, repository access for Devin and a test suite or CI check that defines success.
Try it yourself: Write a migration playbook for one repository
Fill the fields to adapt the prompt. Empty fields stay as visible [placeholders].
Migrate [Repository] from [Current stack] to [Target stack]. Follow this playbook: [Playbook steps]. Work in small commits, keep behaviour identical, and do not change public interfaces. The migration is done when [Done when]. If a file does not fit the playbook, stop and list it with the reason instead of improvising. Open one pull request with a summary of what changed, what you could not migrate, and the exact commands you ran to verify it.Try the playbook on one repository and review the result before launching it across many.
Sources
- Devin’s 2025 Performance Review: Learnings From 18 Months of Agents At Work (opens in a new tab)
Vendor post · cognition.ai · Published 14 Nov 2025 · Source checked 11 Nov 2026, 10:02 UTC
What this source supports
- 67% of Devin pull requests merged, up from 34%
- Bank file migrations in 3 to 4 hours versus 30 to 40 for engineers
- Test coverage rising from 50 to 60% to 80 to 90% at customers
- Weak on ambiguous requirements and mid-task scope changes
- When to Use Devin (opens in a new tab)
Project documentation · docs.devin.ai · Undated · Source checked 11 Nov 2026, 10:02 UTC
What this source supports
- Clear success criteria such as tests or CI give the best results
- Split large projects and run sessions in parallel
- Rule of thumb: tasks of three hours or less