What Is LLM Evaluation (Evals)?
A systematic method for measuring the quality of AI-generated answers — often using a second model as a judge — so quality is tracked rather than assumed.
You cannot manage what you do not measure, and AI answer quality is no exception. Evals score generated answers against criteria such as groundedness (does the answer follow from the retrieved sources?), completeness, and correctness.
The LLM-as-judge pattern uses a capable model to score answers at scale — imperfect but consistent, and far cheaper than human review of every conversation. Judged scores trend meaningfully: a docs migration that quietly broke retrieval shows up as a score drop before customers complain.
Mature eval practice combines automated judging with human signals (user feedback, review queues) and drill-down tooling (traces) to diagnose why a score moved.
How Kelu uses this
Kelu does not show eval scores to customers. It collects per-message user feedback and records a trace of every answer, so you can trace any weak answer to its cause. Simulations replay past tickets before launch, and a person can review each result.
Related terms
Hallucination
When a language model generates fluent, confident content that is factually wrong or entirely invented.
Grounding
Tying an AI answer to specific source content, so each claim comes from retrieved passages rather than the model's memory and can be checked against them.
Abstention
When an AI assistant declines to answer because its sources don't cover the question, and says so plainly instead of writing a confident guess.
Citations
Links from an AI answer back to the specific source documents used to generate it, allowing users to verify every claim.
RAG (Retrieval-Augmented Generation)
An AI architecture that retrieves relevant documents from a trusted knowledge source and passes them to a language model, so answers are grounded in real content instead of the model's training data.
From definition to deployment
Connect a knowledge source and see cited AI answers on your own content in minutes — free, no credit card required.
Free to start · No credit card required