Building a Real AI Jarvis (GPT Realtime 2 + Cursor)
TL;DR
Zero coding required: Riley emphasizes that even complete novices can build a voice agent using Cursor, rating the required experience as 0 out of 10.
Built in roughly 3 prompts: The initial Jarvis app was created with a single long prompt, then refined through iterative follow-up prompts for design changes and bug fixes.
GPT Realtime 2 is the core technology: OpenAI's new voice model enables natural, interruptible conversation with sub-second latency and tool-calling capabilities.
Multi-modal capabilities out of the box: The agent handles web search via Exa API, image generation and editing with GPT Image, mermaid diagram rendering, and local note-taking.
Computer use mode lets it control your desktop: Ricky can open applications, type text, and interact with UI elements, though safety confirmations can be toggled off for faster operation.
Riley's team offers enterprise agent implementation: The video includes a pitch for helping companies deploy shared AI agents across teams via Slack, iMessage, and other platforms.
Summary
Riley Brown builds a fully functional voice-controlled AI assistant named Ricky that can search the web, generate and edit images, create diagrams, and control his computer, all in one sitting using Cursor and GPT Realtime 2 with zero coding experience required.
Was This Useful?
Share
Keep Reading
Make Alcreon Yours
Tune your feedFive quick questions, and the feed ranks what matters to you first.Or just get notified
The weekly Echo. Signal worth keeping in your inbox.
Every new piece, announced on X.
Read Next
See all
Playbook
The Retirement Email Isn't a Warning
Model retirements now arrive every few weeks; the config-eval-rehearsal loop turns each deprecation email from a fire drill into an afternoon swap.
Playbook
The Cheapest Model That Passes
OpenRouter lists 400 models behind one API. The fix for choosing isn't a better leaderboard, it's a four-step protocol that ends in a real eval.
Playbook
Cheap Models, Hard Tasks
Most agent workflows route every step to the frontier model by default. The bill scales with how chatty the agent gets, even when most steps don't need that brain.