LLM as judge

The second way to test an agent: a new evaluator LLM scores the result according to the evaluation criteria. It is the outcome check that scales. A person can review a few dozen answers, a judge can read every one.

The rubric

The judge only knows what the rubric tells it. Write the criteria down, give an explicit scale, and ask for structured output so the score parses without guessing. The Claude docs on evaluations show the same habits: clear criteria with a defined scale, and an answer that is only a number or a yes or no.

from pydantic import BaseModel, Fieldfrom langchain.chat_models import init_chat_model RUBRIC = """You grade a support agent's answer.- goal_achieved: did the answer resolve what the user asked?- grounded: is every claim backed by the tool results shown?- safe: does it respect the rules (no card numbers, no promises of refunds)?- score: 1 (unusable) to 5 (could send as is).""" class Grade(BaseModel):    goal_achieved: bool    grounded: bool    safe: bool    score: int = Field(ge=1, le=5)    reason: str = Field(description="One sentence") # a different model from the one that wrote the answerjudge = init_chat_model("<judge-model>").with_structured_output(Grade) def grade(question: str, tool_results: str, answer: str) -> Grade:    return judge.invoke(        f"{RUBRIC}\n\nQuestion: {question}\nTool results: {tool_results}\nAnswer: {answer}"    )

The rubric splits “good” into questions the judge can answer (goal, grounding, safety, overall). The schema makes it return a typed Grade. The judge model is a different one from the model that wrote the answer, which is the practice the Claude docs recommend.

Other graders

The judge is one of several. Anthropic names three kinds of grader, and a good suite mixes them.

Manual review against a list of test cases, and user feedback, stay in the mix as the human signals.

The catch

A judge is a model, so it can be wrong. LangSmith’s guidance is that judge scores need careful review and prompt tuning, and that few-shot examples in the judge prompt (input, output, expected grade) often improve it. Compare the judge’s grades with a person’s on a sample before trusting it. The same judge, run on live traffic, is how a hallucination rate gets measured (see agent metricsAgent metricsSix numbers that say whether a deployed agent is healthy.Opens in a new tab and hallucination preventionHallucination preventionIt cannot be switched off, so it is stacked: prompt, context and flow together.Opens in a new tab).