Back to blog

Why do evals matter more than prompts?

Thắng Đoàn
Thắng Đoàn

You spent weeks tuning the perfect prompt, and your agent still gave a customer a confidently wrong answer. The problem is not the prompt. You have no regression net: no repeatable test that tells you when a change breaks the system.

The same prompt, different outcomes

LLM applications fail in ways traditional software does not. The same prompt can produce different outputs, and a retrieval step can return the wrong document while every HTTP status reads 200, per marktechpost. An agent can loop through many tool calls, burn thousands of tokens, and still deliver a wrong answer. Prompting skill cannot protect you from this randomness. Only measurement can.

  • Standard APM misses semantic quality like retrieval relevance and reasoning traces.
  • Observability platforms record prompts, tool calls, tokens, costs, and score outputs.

Evals are just tests

An eval is simply a test for an AI system. You give the system an input, then apply grading logic to its output to measure success, as Anthropic's engineering team defines it. Without good evals, teams get stuck in reactive loops, catching issues only in production where fixing one failure creates others. Agents make this worse because they modify environments across many turns, so errors propagate and compound. A prompt is one guess; an eval suite is a safety net you keep.

  • Tools like Promptfoo and DeepEval flag regressions in CI before production.
  • Ragas scores RAG faithfulness and context relevance without large labeled datasets.

Tie evals to business outcomes

Forbes reports that one-off benchmarks rarely predict business outcomes. Effective evals mirror real journeys such as answering a billing question or drafting a proposal. Teams score outcomes the way a business would, not the way a leaderboard would. Forbes lists practical metrics that make this concrete.

  • Grounded-answer rate: answers that stick to verified sources.
  • Brand-tone adherence: output matches your voice.
  • Cost per successful task, and faster resolution.

The hidden cost of testing

CIO reports that nearly 80% of enterprises have deployed AI agents, yet most underestimate evaluation costs, which can exceed expectations by far. Using a second model to vet outputs, called LLM as a judge, can cost more than running the agent itself, CIO reports. Budget for evals like production infrastructure, not an afterthought.

  • Non-deterministic outputs make testing more expensive than traditional QA.
  • Microsoft's Copilot tool runs repeatable scenario tests across user profiles to cut manual effort.

What this does not solve

Evals do not make a bad product idea good. They measure the scenarios you thought to test, and novel failures still surface in production. Nor do they remove non-determinism; they only bound it. Per Anthropic, multiple trials are needed just to get consistent results.

Which three customer journeys would you turn into eval scenarios this quarter?
What is your current cost of testing per agent run, and who owns that budget?

Pick your agent's most common failure case and write one input plus grading logic for it today.

Sources

marktechpost: Top LLM Observability and Evaluation Platforms in 2026

Anthropic: Demystifying Evals for AI Agents

Analytics Insight: How to Test LLMs, AI Agents and Generative AI Applications

Microsoft Learn: Response quality evaluations for the Employee Self-Service

Forbes: How To Maximize LLM And Multi-Agent ROI With AI Evals

CIO: AI agent evaluations: The hidden cost of deployment

Share:

Recommended for you

Enjoyed this article?

Subscribe for new articles. No spam. Unsubscribe anytime.

By subscribing you agree to receive the newsletter. No spam, and you can unsubscribe anytime.