Engineering · 8 min read
How we cut RAG latency 4× without changing the model
Caching, batching and a different retriever — the three changes that took p95 from 2.4s to 600ms in production.
AI engineering, from the field.
Notes from engineers shipping AI in production across ASEAN. No think-pieces. Just what worked and what didn’t.
Latest posts
What we’ve been writing.
Governance · 12 min read
The governance review we wrote for MAS
A walk-through of the actual evidence pack we submitted, mapped clause-by-clause to MAS FEAT principles.
Engineering · 6 min read
Why we stopped using vector DBs for small corpora
Under 100k chunks, BM25 plus a re-ranker beats half the vector pipelines we’ve inherited. Here’s the math.
Process · 10 min read
A month inside an AI Sprint — what we shipped, what we binned
Daily diary of a four-week sprint with an ASEAN insurer — including the use case we declined to take into production.
Security · 9 min read
Prompt injection: the patterns we actually see
Twelve months of red-team logs across client engagements. The five injection patterns that account for 80% of attempts.
Subscribe to the engineering notes.
Once a month. The post that did the most work, plus what we shipped. No sponsor slot, no newsletter swap.
Subscribe
One letter a month · one-click unsubscribe