AI Benchmark

A benchmark is a fixed collection of tasks with a defined scoring rule, used to measure and compare AI models. Examples cover multiple-choice knowledge tests, code-generation problems, math word problems, and image recognition.

Why it matters

Benchmarks give a shared yardstick for progress and help shortlist models. However, scores may not reflect your own use case, and published results can be inflated if test items leaked into training data. Treat a benchmark as one signal among several, not a verdict.

Example

A coding benchmark might run model-written functions against hidden unit tests:

164 problems, 1 attempt each
solved: 123 -> pass@1 = 75%

The figure is only meaningful alongside the exact prompt format, settings, and benchmark version used.

Common mistakes

• Choosing a model from a leaderboard rank without testing on your own examples.
• Comparing scores produced with different prompts or settings.
• Ignoring cost, speed, and reliability, which benchmarks rarely capture.

Related

• Large language model
• Overfitting
• Inference
• Dataset

यह पेज सामान्य जानकारी है, जिसे हमारी संपादकीय टीम ने लिखा है और दस्तावेज़ीकरण बदलने पर अपडेट किया जाता है। सर्वर और प्रदाता अलग-अलग होते हैं, इसलिए बदलाव पहले सुरक्षित तरीके से जाँचें। देखें: संपादकीय नीति और अस्वीकरण.