All work
Sole researcher / engineer
Semantic Caching R&D: LLM Cost Optimization (Wakil.ai)
2025 · Production R&D · 10,000 MAU platform
Semantic-caching R&D for Wakil.ai, a legal AI platform with 10,000 monthly active users: a semantic FAQ cache layer targeting about 10% of query traffic, built to cut LLM API costs without hurting answer quality.
Highlights
- Ran a semantic-caching study on 4,000 real production queries.
- Found the headline “55% overlap” figure misleading: most of it came from same-session follow-ups, not reusable FAQ traffic that’s safe to cache.
- Identified a safe scope (~10-15% of traffic) and designed a Layer 1 FAQ cache with strict query gates and query-only embeddings.
- Documented false-positive cases and recommended a staged rollout, turning “cache everything” into a defensible, data-grounded strategy.