We Cut 94% of AI Coding Tokens With a Local Code Index - Rajkumar Sakthivel, Tesco

TL;DR

• 1

  90% of AI coding costs come from input, not output: Files, search results, and context sent to the model account for most expenses, while the AI's generated code represents only 10%.

• 2

  Their typical query sent 45,000 tokens when only 5,000 mattered: They paid for 40,000 tokens of irrelevant code every single query, like ordering one pizza but paying for ten.

• 3

  Cutting input by 94% saves 61% total cost: Output compression only saves about 8% total, but input reduction has massive impact because that's where the money goes.

• 4

  Dual search (meaning + keyword) catches what each misses alone: Meaning-based search finds related ideas but misses exact names; keyword search does the opposite. Together they reduce missed results from 1 in 4 to 1 in 10.

• 5

  Simple scoring formula beats complex AI models for relevance: A 50/30/20 formula (meaning/keyword/recency) runs in 0.4ms without extra AI calls, faster than asking AI to judge its own results.

• 6

  Real test on FastAPI: 83K tokens down to 4.9K per question: That's 94% reduction with 90% accuracy finding the right code, tested on 53 files with 20 real developer questions.