Cloudflare cut its inference bill by 68% by being obsessive about optimization
After the cost per run of its AI code reviewer doubled, Cloudflare analysed its token usage. Prompt caching and routing requests to the right model cut its inference spending by 68% and made reviews faster.