Where the Loop Breaks

A reference Dossier for the staff engineer, engineering manager, or founder already running a coding-assistant stack: the six code loops the team runs every week, the tool that earns each one today, and where the authority line has to sit between an agent that writes and a human who owns.

TL;DR

Engineers now run six code loops with a mix of IDE assistants and autonomous agents: bounded inline edits, multi-file refactors, test generation, feature builds behind a PR gate, first-pass PR review, and long-running autonomous sessions. Every tool on the market can now technically span more of the pipeline than it could a year ago. Teams get hurt when they let the agent's authority stretch to whatever the credentials, the tool defaults, and reviewer fatigue happen to allow, rather than setting the line against what a failure would actually cost.

The authority line follows the failure class, not the capability curve. Where the agent's mistake is cheap to catch downstream, let the agent write, commit, and open the PR. Where the mistake is expensive downstream, keep the human upstream at write or merge. The merge to main is the load-bearing gate; it moves slowest because it carries the most failure cost.

A starting stack pairs one assist layer with one refactor agent, one PR-gated feature agent, and one review bot:

• GitHub Copilot Business at $19 per seat for autocomplete and inline chat
• Claude Code (Pro $20 per month, Team Standard from $20 per seat annual) for local refactors and test drafting
• Codex (ChatGPT Plus $20 per month, Business $25 per seat) for PR-gated feature work
• CodeRabbit Pro at $24 per seat annual for first-pass PR review
• Optional: Devin Teams from $80 per month plus $40 per developer for long-running sessions when review capacity can absorb the extra PRs

For a 10-engineer team, assist-only lands at $430 to $490 per month; assist plus agent at $600 to $1,650; agent-first at $880. The number that decides whether the stack earned its seat is the count of agent-authored PRs a human actually reviews carefully in a week.

AI Coding Agents Are Not One Authority Line

A back-end lead at a fintech startup asks Codex to work an overnight batch of maintenance tickets. She scopes it in the prompt: one migration fix, one bug in the retry logic, one dependency bump. She uses her normal shell token so Codex can run the tests. Morning brings a Slack message from the platform team. Codex has opened four PRs (not three), one of them touching an auth adapter the ticket never mentioned. It has also amended a commit her teammate had pushed the day before, and it fired a workflow that tried to promote to staging. The code inside each PR is competent. What went wrong is that nobody ever wrote down which surfaces Codex was allowed to touch, and the shell token said "anything my user can do." The scope was set by the credentials, not the ticket.

A 25-person consumer-app team turns Cursor Agent on for everyone and adds Background Agents for the more experienced engineers. Week one is a party. The team ships twelve PRs off Cursor-authored branches, up from three or four the previous sprint. Week three the platform lead notices approvers rubber-stamping. The reviewers are reading the agent's summary, glancing at the CI check, and hitting approve. Six weeks in, a customer files a bug that traces back to an "obviously safe" refactor Cursor had proposed on a shared response helper. The refactor had quietly shifted what null meant in the JSON payload. The tool worked as advertised. What broke was review capacity: the team had multiplied write throughput without multiplying anyone's time to actually read the diffs.

A solo founder building an internal-tools product spins up Devin on a Monday morning with a list of eight backlog items and a coffee. She grants it broad repo access because the alternative is fiddling with permissions all day. By Friday, Devin has closed the loop on six of the eight items and is asking her to review the results. One item is a genuine fix. Two solve the ticket she wrote, not the customer problem behind it, because she had drafted the tickets quickly and Devin took them literally. Two more introduce their own new pattern for handling errors that doesn't match the codebase. The last one erases a chunk of instrumentation Devin decided was dead code. Nothing Devin did was outside its granted authority. The gap was upstream: she had asked an agent to interpret intent that was never fully written down.

Three teams, three different failures, one shared category. Authority was set by what the tool could do, not by what a mistake would cost. In every case the agent's reach grew to fit the credentials it happened to have, the defaults nobody changed, and the review time nobody had. None of the teams had written the line down first. The right question isn't which agent is best; the leaderboard answers that with a different vendor every quarter, and the answer keeps changing. The right question is: for each code loop the team runs, where does the line sit between what the agent may do and what a person owns?

A December 2024 arxiv study on automated code review looked at bot comments across real repositories and found high comment-resolution rates alongside longer PR closure times and a real share of faulty or irrelevant comments. Automated review is doing work, and it's also generating triage load. A green bot comment carries no merge authority on its own.

The Six Loops and the Authority Line That Earns Each One

Each sub-case names the loop, the tool that earns the slot, what ships clean, where the ceiling sits, and the Friday action, with a short table of production examples.

Bounded Inline and In-IDE Edits

The work is the tight inner loop: autocomplete, next-edit suggestions, single-file chat rewrites, small scoped edits. The engineer sees every keystroke before it lands.

The slot belongs to GitHub Copilot. Copilot owns the tab layer because the failure is cheap to catch at each suggestion, editor coverage is broad, and the seat economics are the friendliest in the category. Business is $19 per user per month and Enterprise is $39; Business and Enterprise prompts aren't used to train foundation models. Strong adjacent fits: Cursor Tab for Cursor-standard teams, and Tabnine for regulated environments needing on-prem or air-gapped deployment.

What ships clean: small idiomatic completions, boilerplate, type annotations, simple function rewrites, and local cleanup the engineer reviews one edit at a time.

The authority line sits at each suggested edit. The ceiling appears at silent semantic drift: a suggestion that compiles and looks idiomatic but changes an invariant, widens access, mishandles money, or normalizes a bad abstraction. The named failure mode is the plausible drift.

If you start this week, pick one autocomplete provider, disable the others, and write a one-page policy: which files can accept suggestions edit-by-edit, which require manual readback, and which reject autocomplete entirely. Security boundaries, money paths, and schema migrations always sit in the manual-readback pool.

Multi-File Refactor With Tests

The work is changing a pattern across many files with a test suite the agent iterates against. Rename migrations, dependency-injection reshaping, API-signature updates, framework-version cleanups.

The slot belongs to Claude Code. It reads the codebase, edits across the working tree, runs commands, iterates against tests, and runs from terminal, IDE, desktop, or browser. Hooks, plan mode, cloud sessions with configurable network egress, scoped credentials, branch restrictions, and audit logging give the loop the governance surface the failure class requires. Included in Claude Pro at $20 per month, Max 5x at $100, Team Standard from $20 per seat annual. Strong adjacent fits: Aider for terminal-first engineers who want a bring-your-own-model workflow with a public benchmark, and Cursor Composer for teams that want the same loop inside the Cursor IDE.

What ships clean: a mechanical rename across many files with tests still passing, a dependency-injection refactor across every call site, a framework-version migration with imports and fixtures updated, and a deprecated-API replacement.

The authority line sits at the commit. The ceiling appears at architectural smuggling: the agent makes local tests pass by moving responsibility across layers, weakening an abstraction, or building a compatibility shim nobody asked for. The named failure mode is the wide diff with the weak invariant.

If you start this week, give Claude Code one bounded refactor with a clean test command, a branch it can push to, and a written "do not touch" list. Auth, billing, migrations, and permission logic stay off the first-week list. Ship one merged PR by Friday where the human authored the commit boundary and reviewed every public-interface change.

Test Generation and Test Intent Review

The work splits. The agent writes tests: fixture setup, golden-path assertions, regression tests, edge-case enumeration. The human reads every test and asks whether the assertion pins the requirement or the implementation the agent just read.

The slot belongs to Claude Code for the drafting; the human owns the intent review. Claude Code is strongest because it reads the codebase, follows existing test style, and iterates on coverage. Strong adjacent fits: Sourcegraph Cody or Amp for large legacy repos where context assembly is the hard part, and CodeRabbit Pro+ for PR-time unit-test suggestions attached to a diff.

What ships clean: fixture setup and mocks for known interfaces, golden-path tests, regression tests from known bug reports, and table-driven tests matching existing style.

The authority line sits at test intent. The ceiling appears at tautological coverage: the agent reads the implementation, writes tests that assert what the implementation currently does, and coverage climbs while risk stays flat. The named failure mode is the covered-but-not-tested regression.

If you start this week, pick one bug the team fixed last month. Ask the agent to write the regression test first. Then manually reintroduce the bug in the production code and run the test. Ship a Friday PR carrying the regression test plus a one-line note in product language explaining what the test asserts.

Feature Build With PR Gate

The work is a bounded engineering ticket: scoped feature, narrow acceptance criteria, existing test harness. The agent iterates on a feature branch, commits, runs tests, opens a PR. The human owns whether the PR merges and whether anything deploys.

The slot belongs to Codex. Codex earns the loop because the product surface matches: Codex web runs background cloud tasks, Codex CLI runs locally with sandboxing and approval policy, and Codex reviews PRs, fixes CI, and posts through GitHub-connected workflows. The CLI runs with network off and workspace-limited file access by default. Business plans don't train on customer data. Included in ChatGPT Plus at $20 per month, Business at $25 per user per month. Strong adjacent fits: GitHub Copilot Coding Agent for GitHub-native orgs, and Cursor Background Agents for teams that want the same loop inside Cursor.

What ships clean: a small-to-medium feature with a precise ticket, a bug fix with a reproduction test, and a dependency upgrade with migration commits ordered and CI green.

The authority line sits at merge. The ceiling appears at spec substitution: the agent builds a plausible feature instead of the intended one, and the tests derived from that misreading go green. The named failure mode is the completed ticket for the wrong job.

If you start this week, pick one ticket a staff engineer can review carefully in under 30 minutes. Put the acceptance criteria on the ticket itself. Force the agent to open a PR against a protected branch. Auth, billing, PII, secrets, migrations, and money movement stay out of the first trial. Ship one agent-authored PR merged by a human, with a note stating whether the change solved the job the ticket described.

AI as First-Pass PR Reviewer

The work is triaging pull requests before a human reviewer sees them. The bot summarizes the change, flags risky diffs, suggests missing tests, points at obvious bugs, and posts a verdict.

The slot belongs to CodeRabbit. CodeRabbit is built around the PR object rather than the editor session, and the plans reflect that: Pro at $24 per developer per month annual or $30 monthly, Pro+ at $48 annual, Enterprise custom. CodeRabbit never trains on customer code and offers self-hosted Enterprise with retention opt-out. Strong adjacent fits: GitHub Copilot code review for teams that want fewer vendors, and Greptile when its codebase-aware review is materially better on the specific stack.

What ships clean: PR summaries in the format reviewers want, risky-file callouts, diff-local security hints, and missing-test suggestions attached to the diff.

The authority line sits at the human reviewer. The ceiling appears at false authority: reviewers start treating the bot's approval as if it were a reviewer's, and the merge gate collapses into review theater. The named failure mode is the bot verdict as merge signal.

If you start this week, enable AI review on one repo but write into branch protection that "bot approved" is never a merge condition. Ship a Friday report comparing bot comments to human review on ten PRs, and mark every bot comment as useful, duplicate, wrong, or dangerous.

Long-Running Autonomous Session

The work is a hours-long autonomous cloud session in the agent's own isolated workspace. The agent explores, writes, runs tests, sometimes self-debugs, and hands back an artifact: a branch, a PR, or a written report. The human batch-reviews at handoff.

The slot belongs to Devin. Devin is built for long autonomy: clean Linux VM sessions from environment snapshots, repo connections, tests, PRs, visual QA, docs, migrations, scheduled chores, and multi-session work. Session changes don't persist back into the base snapshot. Pro $20 per month, Max $200, Teams from $80 per month plus $40 per developer seat. Strong adjacent fits: Cursor Background Agents for teams already in the Cursor ecosystem, and Sourcegraph Amp Orbs for power users who want remote machines and pass-through pricing. Amp doesn't ask for approval before running tools by default, so it belongs in an isolated environment.

What ships clean: a backlog chore in a non-production area, a documented-pattern migration, a dependency upgrade across a service, a documentation pass, and a feature spike producing a report and proof-of-concept branch.

The authority line sits at PR, merge, and deploy. The ceiling appears at unbounded interpretation: over hours, the agent accumulates small reasonable-looking assumptions, and the final artifact can be correct locally and wrong for the product. The named failure mode is the coherent artifact for the wrong job.

If you start this week, assign Devin one non-production backlog chore with no production credentials and no deploy path, against a target branch that requires human review. Production credentials, incident hotfixes, and security-sensitive code stay out of long-running autonomy. Ship a Friday PR with one note on spec adherence, one on code quality, and one metric for human rework time.

What the Engineer Owns Regardless of Tool

The engineer owns architectural intent. An agent can read patterns, infer boundaries, and copy style. What it can't do is explain why the boundary is where it is, or which tradeoff the team accepted when the module was drawn that way. Module boundaries, API shape, cross-service contracts, data-model changes, and permission architecture stay with a human who can defend the choice.

The engineer owns the security review of anything the agent writes. Auth, PII, payments, secrets, tenant isolation, data deletion, and audit paths don't skip a review just because CI came back green. Replit's shared-responsibility page names the same rule: business logic, secrets, access control, and review-before-ship sit with the user. The vendor tier changes what's promised about training and retention. It doesn't change who owns the threat model.

The engineer owns incident response, deployment approval, and code review as the merge gate. Agents can draft diffs during an incident, but the blast radius, the rollback call, and the customer comms belong to the incident commander. Deploy touches customers, live data, revenue, and the pager, so the deploy gate is the last one to move. Treating a bot's verdict as merge approval turns review automation into theater the same week the team notices.

The engineer owns intent verification against the ticket. Broken code is rarely the most expensive agent failure. Correct-looking code that solves the wrong job is.

A team that hands architecture, security review, incident response, deploy approval, merge accountability, and intent verification to a platform has handed over the engineering craft. Six months later the codebase looks like something the agent shipped, not something the team owns.

Cost Calculus and Coexistence

Five candidate stacks for a 10-engineer team:

• Assist-only: GitHub Copilot Business ($19 per seat) plus CodeRabbit Pro ($24 per seat annual). $430 per month. Better autocomplete and first-pass review, no autonomous throughput. Right for teams with legacy code, high review discipline, and no capacity to absorb agent PRs.
• Assist plus agent: Cursor Teams ($40 per seat) plus Claude Team Standard ($20 annual). $600 per month, or up to $1,650 with Claude Team Premium. Strong in-IDE help, serious local refactor loop, controlled agent sessions.
• Agent-first: Devin Teams ($80 base plus $40 per developer) plus Cursor Teams ($40 per seat). $880 per month before overages. Attractive only if the team can review the output.
• OpenAI-heavy: ChatGPT Business at $25 per seat (Codex included). $250 per month. Strongest when Codex cloud tasks and PR workflows are the desired autonomy layer.
• Power-user: Amp with pass-through model pricing ($5 minimum credit purchase) plus the existing IDE. Cost-efficient for disciplined engineers, dangerous for teams that don't isolate shells and credentials.

What the seat price hides: model credit overages, reviewer hours on agent PRs, rework hours on code that passed CI but missed intent, security review time on any PR touching auth or payments, and rollback hours from regressions the agent caused.

Two agent layers earn their seats when one owns local refactor and the other owns PR-gated feature work, and a written authority-line policy decides which loop each ticket goes into. Without the policy, a second agent just produces two competing versions of the same feature by Wednesday.

Pitfalls and Anti-Patterns

Letting the Agent Commit Directly to Main

Cursor, Claude Code, Codex CLI, Aider, Cline, Amp, and Devin can all reach Git workflows through file writes, shell, or PRs. Branch protection has to be enforced outside the agent, not in the prompt. The fix is a hard rule at the branch layer: main is protected, protected branches require human review, and no agent token has push authority to a protected branch.

Treating the Agent's PR Review as a Merge Signal

CodeRabbit, Copilot review, Codex review, Cursor Bugbot, and Greptile can reduce review load. None of them replace accountability. The fix is a branch-protection rule that never counts a bot approval, plus a written review policy naming who owns the merge.

Opening Ten PRs No Human Can Review

Agent throughput isn't team throughput. Reviewer capacity is the real bottleneck, and no second agent scales it. The fix is a WIP limit on agent-authored PRs sized to what a human can review carefully per week.

Giving an Autonomous Session Production Credentials

Long-running sessions don't need write access to production. What they need is read-only scopes, fake secrets, sandboxed services, and a PR handoff. The fix is a credential pool that looks structurally different from production, and a rule that any task requiring real production credentials never runs inside an autonomous agent loop.

Letting Authority Be Set by Capability

"The tool can merge" is not an argument. "The failure is cheap to catch after merge" is an argument, and usually it's false. The fix is a written authority-line policy per loop, updated when the tool ships new features, so the team makes the decision instead of the vendor's changelog.

What to Validate Before Expanding the Authority Line

Before day one. Pick one loop and one authority-line move. Multi-file refactor and feature build with PR gate are good first candidates. Baseline the current loop for one full week: cycle time, review time, review-round count, escaped defects, and rework minutes. Write down the pass criteria.

Week one: run the current line. Keep tool configuration unchanged so the baseline is trustworthy.

Week two: expand the line. Move the authority line for the one loop. Track agent-PR merge time, human rework minutes per agent PR, agent-caused regressions, and cycle time. Keep everything else steady.

Expand only if the loop wins. The trial passes when cycle time drops (with review time counted), rework stays within tolerance, zero severe regressions reach production, and a staff engineer will defend the expanded line at the next incident review. Roll back on any severe regression, on reviewers approving on the bot's word alone, or on a PR queue that stops clearing.