How to Evaluate a Large Language Model: Scoring Dimensions and Prompt Templates

Why You Can't Rely Solely on Leaderboards

Public leaderboards (various Arenas and benchmark rankings) have their value—they offer a quick snapshot of general model tiers. However, relying directly on them for selection decisions has three critical flaws:

• Tasks are irrelevant to your use case: Benchmarks often test math, general reasoning, or English-language tasks, whereas your scenario might involve Chinese legal Q&A, extracting data in specific formats, or customer service scripts tailored to your product. The relevance may be low.
• Contamination and overfitting: Questions from popular benchmarks may have leaked into training data, or models may have been specifically optimized for these tests, inflating their scores artificially.
• Averages mask distribution: A model with a high average score might perform merely adequately on the specific type of task you care about most.

Use leaderboards as a reference to narrow down your candidate pool; however, final selection must be driven by your own evaluation dataset. This follows the same logic as why RAG tuning requires building an evaluation set—optimization detached from real-world scenarios is merely self-comfort.

Step 1: Build Your Own Evaluation Dataset

This is the foundation of your entire evaluation process—the most labor-intensive step, but also the one with the highest return.

• Sample from real-world needs: Draw questions directly from actual user queries, historical support tickets, and genuine documents; avoid fabricating "ideal" prompts out of thin air. Fifty authentic examples are worth more than five hundred imagined ones.
• Cover the distribution, not just common cases: Include both high-frequency typical tasks and deliberately selected hard cases, edge conditions, and inputs prone to errors. Model performance gaps often emerge in the long tail.
• Annotate expected answers or scoring criteria: Ideally, provide standard answers (e.g., for extraction or classification tasks); for open-ended tasks, specify which key points a good answer must include and what errors it must avoid.
• Scale: A starting set of 50–100 examples is sufficient to reveal significant differences; there's no need to wait until you've accumulated thousands. Begin small and expand later.

Once built, your evaluation dataset becomes a reusable asset: use it to retest whenever you switch models, tweak prompts, or upgrade the model itself, ensuring consistent horizontal comparisons.

Step 2: Define Scoring Dimensions

The vague notion of "good or bad" must be broken down into measurable dimensions, with weights varying by task type. Common dimensions include:

Select just 3–5 dimensions most relevant to your specific scenario; don't try to cover everything. For each dimension, define a simple scale (e.g., 1–5) and clearly articulate the criteria for each rating level—the more concrete the standards, the more consistent the scoring.

View full-size image

×

Radar chart comparing different scoring dimensions

Only by breaking down "good or bad" into measurable dimensions like correctness, instruction following, and completeness can you achieve objective comparison.

Step 3: How to Score—Three Approaches

1. Automated Metrics (When Ground Truth Exists): Calculate accuracy or F1 score for classification tasks, field match rate for extraction tasks, and hit rate for retrieval. These methods are objective and scalable but apply only to tasks with clear right-or-wrong answers.

2. Human Evaluation (Most Accurate, But Costly): Humans score each item against defined criteria. This yields the highest quality results, especially for open-ended or highly subjective tasks. The downsides are speed, cost, and the need to align scoring standards across evaluators. It is best suited for small-scale deep evaluation and establishing a calibration baseline for automated scoring.

3. LLM-as-a-Judge (The Mainstay of Scalable Evaluation): Use a strong model to score answers based on your specific criteria. This approach can rapidly process hundreds of items in an evaluation set, making it the current standard for large-scale assessment. However, several pitfalls must be understood:

• Position/Length Bias: The judge model may favor longer responses or options appearing earlier in a list. When comparing two answers, swap their positions and evaluate each pair twice to average out bias.
• Self-Bias: Models often prefer outputs that match their own style; avoid using the target model as its own judge.
• Hardcode Criteria into the Prompt: Do not simply ask for "a score." Provide clear scoring dimensions, specific criteria for each rating level, and require the model to state its reasoning before assigning a score (this improves consistency).
• Human Calibration Required: First have humans evaluate a small batch, then compare those scores with the LLM judge's results. Only deploy the judge at full scale once it is confirmed reliable.

Practical Combination: Use an LLM judge for initial screening across the entire dataset, and human evaluation on a small subset for calibration—balancing scale with credibility.

A Ready-to-Use Scoring Prompt Template

You are a strict, impartial reviewer. Score the "answer" against the criteria below.

[User question]
{question}

[Answer under review]
{answer}

[Reference points] (if any)
{reference_points}

[Scoring dimensions, 1-5 each]
- Correctness: are the facts accurate, is anything invented. 5 = fully accurate with no hallucination, 1 = contains clear errors
- Instruction following: does it meet the format and constraints in the question. 5 = fully met, 1 = seriously off
- Completeness: does it cover the reference points. 5 = all points present, 1 = large omissions
- Relevance: is it on topic without padding. 5 = tightly on topic, 1 = largely off topic

[Output format]
Give a short reason per dimension first, then the scores, then a JSON summary:
{"correctness": x, "instruction": x, "completeness": x, "relevance": x, "reason": "one-sentence verdict"}

Note: score only from the information given, do not fill in gaps from imagination; reasons must point at the specific problem.

Key Points: Reasoning before scoring (forcing the model to "think through" its decision improves consistency), hardcoded dimensions and scales, and structured output for easy batch aggregation. Embed this template into your scripts, feed it an evaluation set, and automatically generate dimension-specific score tables for each model.

View full-size image

×

Engineers comparing benchmark results across multiple models

A pragmatic combination: LLM judges handle the full volume while human experts calibrate a small batch—balancing scale with credibility.

Don't Forget Cost, Latency, and Stability

Model selection isn't about quality scores alone. When two models are equally viable, compare them across these dimensions:

• Cost: Calculate token expenses based on your actual request volume; when quality is comparable, the cheaper option wins.
• Latency: For consumer-facing and interactive scenarios, time-to-first-token (TTFT) and total duration directly impact user experience.
• Stability: Run the same batch of prompts multiple times to assess output variance, format consistency, and rate-limiting strictness.
• Context Window and Capability Boundaries: Evaluate whether the context window is sufficient, how reliable tool calling is, and the strength of Chinese language support.

Only by placing quality scores alongside these engineering metrics in a single table do you have a complete basis for selection. In production environments, you can also use an AI gateway to conduct A/B testing across multiple models with real traffic data.

Target Audience and Alternatives

This approach is designed for technical teams and product leads selecting models for products or seeking to validate whether "switching models yields better results." If you are an individual user occasionally experimenting, such formality isn't necessary—simply test a few providers manually using your specific real-world questions; intuition will suffice. However, whenever selection involves production deployment, financial investment, and accountability for outcomes, even an evaluation set of just 50 items can elevate your decision-making from "hearing that Model X is strong" to confirming "in my scenario, Model X outperforms Model Y by exactly 12%."

Common Questions

Q: How often should the evaluation set be updated? A: Add new cases whenever scenarios change or fresh failure examples emerge. Continuously collecting real-world edge cases exposed in production ensures your evaluation set stays aligned with actual needs, evolving into your most valuable benchmarking asset.

Q: When a new model is released, do I need to retest everything from scratch? A: With a fixed evaluation set, the cost of retesting is minimal—just run the script once again. This is precisely why building an evaluation set pays off: models iterate rapidly, but you retain a consistent yardstick for ongoing comparison.

Q: Won't LLM judges be less accurate than humans? A: On dimensions with clear criteria, well-calibrated LLM judges can achieve high agreement with human evaluators while being significantly faster and cheaper. The key lies in defining standards clearly, mitigating positional bias through shuffling or masking, and using human feedback for calibration. For tasks involving purely subjective aesthetic judgment, human evaluation remains recommended.

Summary

The right approach to evaluating large language models: use leaderboards only to narrow down options; rely on your own evaluation set for final selection. Follow a three-step process—build an initial dataset of 50–100 items grounded in real-world needs, break down "good or bad" into 3–5 measurable scoring dimensions, then score using LLM judges across the full set supplemented by human calibration on small batches, while also factoring in cost, latency, and stability. With this methodology and the provided scoring prompt template, you can shift model selection from hearsay about who's strongest to data-driven proof of which model best fits your needs.