Playbook
Tasteful Skills
“Tasteful Skills” argues that the best agent skills are not documentation or best-practice lists.
Back to Podcast Digest
Spec-driven development: the AI engineering workflow at Notion | Ryan Nystrom
TL;DR
Notion turned standup prep into an agent job — Ryan Nystrom built a custom Notion AI agent that runs daily at 9:00 a.m., scans the last 24 hours of Slack, merged PRs, closed tasks, Honeycomb metrics, and yesterday’s meeting, then generates a detailed pre-read so the team can spend standup on decisions instead of dead-eyed status updates.
The payoff is small in minutes but huge in cognitive load — Ryan says the automation may only save ~20 minutes a day, but more importantly it removes the soul-sucking context switching of hunting through updates and formatting them for different audiences.
Background coding agents let him ship fixes from a Notion task comment — using an internal system called “Boxy,” Ryan can mention Codex on a task, send a few sentences plus a screenshot, and get back a PR, preview URL, test notes, and even UI verification screenshots in about 20 minutes.
Spec-driven development is the real workflow shift, not just AI autocomplete — for a major Notion AI rewrite, Ryan started with an empty markdown spec, dictated ideas into Whisper, had Codex turn that into a structured spec, then pointed Codex at the spec and effectively one-shotted the implementation.
The spec becomes the source of truth in version control — instead of treating docs as disposable pre-work, Ryan keeps detailed markdown specs in the repo, updates the spec when behavior changes, and relies on explicit verification steps so agents can build and test against plain-English requirements.
Fast CI is now an AI scaling constraint, not just an engineering nicety — Ryan argues that slow pipelines were already bad for humans, but with agent workflows they become a hard throughput bottleneck; he says teams like Stripe doing 1,300 agent PRs a week simply can’t operate if CI takes an hour.
The Breakdown
Ryan’s big claim: AI made work feel fresh again
Ryan opens by saying AI has completely upended how he works after 20-plus years in software. He’s swapped IDEs, terminals, and tools more in the last year than in the previous decade, but instead of burnout he describes “joy and freshness” — waking up excited to tinker, build, and move faster.
The Notion standup agent that kills meeting prep
Ryan manages a 6–7 person team working on “Afterburner,” a project to cut Notion’s CI time to a quarter of its current level. To avoid painful round-robin standups, he built a custom Notion AI agent that generates a daily meeting pre-read from Slack, closed tasks, merged PRs, Honeycomb metrics, and the prior meeting transcript, so the team jumps straight into problems, wins, and decisions.
Why this works better than the usual glazed-over status meeting
Claire points out that most standups get lighter and less useful over time because nobody wants to prep them and nobody wants to sit through them. Ryan says the new format surfaces real work — like a mock server fix that improved tests by up to 13% — and also democratizes visibility for quieter engineers who might not naturally self-promote in meetings.
The hidden benefit: less burnout, more real work
A recurring theme is that AI removes “paperwork” rather than the interesting parts of managing. Ryan says he used to hate spending half his time compiling updates and writing reports, and now feels like he’s in a sweet spot: still supporting people, still coding, but no longer buried in administrative sludge.
Inside the custom agent: simple instructions, smart fan-out
Ryan shows the actual Notion AI setup: a scheduled 9:00 a.m. run, read-only access to most project systems, edit access only to the meetings database, and instructions to use sub-agents in a map-reduce style. He gives it a strict output template — CI speed, decisions, progress, bugs, risks — and even had the agent help configure its own Honeycomb MCP from a screenshot because, in his words, “too lazy… just take it, figure it out.”
Boxy: assigning Codex a task from inside Notion
The next demo is a lightweight internal tool called “Boxy,” a fleet of little VMs with Codex or Claude Code installed. Ryan gets a text from a friend asking for “copy link to tab” in a Notion tab block, writes four short sentences plus a screenshot in a task, mentions Codex in the comments, and within roughly 20 minutes gets a PR, preview environment, implementation notes, and screenshots of the agent verifying its own UI work.
His prompting style: admit confusion and force the model to defend itself
Ryan says one of his most useful prompt lines lately is: “I literally don’t know what I’m doing here. Explain it like I’m a 5-year-old.” Especially on CI work, he pushes back hard and asks the model to defend its choices with evidence, because he doesn’t just want reassurance — he wants a cited argument he can trust.
Spec-driven development at scale: start with markdown, not code
For a Notion AI agent harness rewrite, Ryan didn’t start in the editor at all: he opened an empty markdown doc, dictated thoughts into Whisper, asked Codex to turn the “yapping” into a formal spec, revised it, then told Codex to build from that spec. Because the spec included code pointers and explicit verification steps, the implementation mostly one-shotted, and now that spec file lives in the repo as the plain-English source of truth that other teams can read, agents can execute against, and humans can update over time.
Fast CI is now the throughput ceiling for agentic engineering
Ryan ends with a blunt point: CI speed was always important, but in an agent world it’s everything. If a run takes an hour, your background agent just spins; if it takes three minutes, both humans and swarms of agents can iterate dramatically faster — which is why he and Claire treat developer experience and CI performance as a prerequisite for getting real value from AI.
Share
Keep Reading
Two Ways to Follow
The Weekly Echo. The inbox-shaped summary of what mattered.
New editorials announced here.
