Your LLM feature has no error signal when it gets worse. Drift, hidden hallucinations, and missing monitoring let decay hide inside plausible outputs.
Thắng Đoàn•LLM Engineering
Building and operating LLM features: RAG, evals, routing, cost and latency.
Thắng Đoàn•You patched your prompt for the tenth time and output still drifts. Fine-tuning helps, but only after evals and prompting stop moving the needle.
Thắng Đoàn•A clever prompt produces one good output. A product must survive users you never met, and the prompt cannot carry that weight alone.
Thắng Đoàn•You switched to smaller models and costs barely moved. The problem is not which model you picked, it is that no one decides which request deserves which model.
Thắng Đoàn•Your p95 dashboard says the platform is healthy, but users still complain. LLM latency has two phases, and your monitoring probably only sees one.
Thắng Đoàn•Token prices hit record lows, yet agentic workloads multiply token usage so fast that cost still decides which features survive production.
Thắng Đoàn•A great prompt still fails randomly. Evals are how you catch those failures before customers do, and they decide whether your AI ships or stalls.
Thắng Đoàn•AWS Bedrock now lets one setting pick retrieval-only, single-shot, or agentic search. Field reports say most RAG failures are engineering choices, so pick the cheapest mode that works.
Thắng Đoàn•