Deep dive · AI & automation

Multi-LLM scoring: why one model shouldn't grade a human (LLM-as-a-judge in production)

LLM as a judge in production: position, verbosity and self-preference bias, how multi-model evaluation reduces them, and two real AI grading pipelines.

By Akash Bansal, founding engineerUpdated 4 November 20267 min read

I've built two systems where a model's score affects a real person. NextMantra scores candidates' résumés and live voice interviews against the role's rubric and produces a ranked shortlist for recruiters. Brahmo grades long-form, handwritten exam answers. Neither leaves the judgement to one model alone. NextMantra scores with multiple LLMs. Brahmo splits reading and judging across model families and gives a human expert the final say. This post explains why, using the published research on LLM judge bias, and shows how each pipeline is laid out.

What is LLM-as-a-judge?

LLM-as-a-judge is using a language model to evaluate something (an answer, an essay, a transcript, another model's output) against criteria and return a score or a preference. The idea went mainstream with Zheng et al. (2023), "Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena", which found that strong judges like GPT-4 reach over 80% agreement with human preferences, about the same level at which humans agree with each other.

That headline is why teams adopt it. The same paper's other finding is why you shouldn't trust it alone: it names position, verbosity and self-enhancement bias, plus limited reasoning on some tasks.

What biases do LLM judges have?

Four are well documented. Each has a cheap structural fix.

Self-preference matters more than it looks for hiring and education. Candidates and students now draft with AI assistants. If the judge shares a family with the assistant that polished the answer, you may be measuring stylistic familiarity rather than quality.

Why shouldn't one model grade a human?

Because a single judge turns a model's quirks into a person's score, and there is nothing in the system to notice. With one model, a bias is invisible: every score carries it consistently. With several models from different families, a bias shows up as disagreement, and disagreement is something you can route to a human. The ensemble is less about averaging towards a better number and more about building a detector for the cases where the number shouldn't be trusted.

This is the same reasoning as redundancy in safety-critical engineering: you don't make one sensor perfect, you use dissimilar sensors and act on the conflict.

How does the Brahmo exam-grading pipeline work?

Brahmo grades long-form exam answers. It's an event-driven pipeline with three model stages and a human at the end. Each stage does one job and hands a result to the next.

1. OCR with Gemini. Handwritten answers become text. This is a reading problem, not a judging problem, so it gets its own model and its own output. If a score looks wrong, the first question is "did we read it correctly?", and the transcription is there to check.
2. Semantic matching. The transcribed answer is matched against the expected points for the question. Matching on meaning rather than keywords is what lets a student who explains the right idea in their own words still get credit.
3. Rubric scoring with Claude. A different model family from the reader scores the answer on a weighted rubric, criterion by criterion, with the matched evidence in front of it. With a weighted rubric, the final mark is a weighted sum of criterion scores rather than a single number the model makes up.
4. Human-expert override. An expert can review and change any score. The pipeline proposes; the human decides.

Splitting the work also means the OCR model never judges and the judge never reads raw handwriting, so an error in one stage can't hide inside another. And because the pipeline is event-driven, a stage can be retried or swapped for a better model without touching the others.

How does NextMantra score candidates?

NextMantra scores two things: how well a résumé fits the role, and how the candidate performed in the live voice interview. Both are scored by multiple LLMs against the role's rubric, and the result is a ranked shortlist a recruiter reviews. The scoring runs after the interview, outside the real-time voice loop. I explain why that split matters in the voice interviewer architecture post.

If you're building candidate scoring, the bias table translates into four rules:

• The rubric comes from the role. Each role defines what's being assessed, and the models score against those criteria, not against a general sense of "good candidate".
• Score one candidate at a time. Don't put candidates side by side in a prompt. That removes position bias at the source. Rank afterwards, in code, from the scores.
• Several models, then a rule. Let multiple models score independently. Agreement means the score can stand. Disagreement is the signal that a human should look.
• The recruiter decides. A shortlist is a recommendation for a person to review, not an automatic rejection.

How do you design a multi-LLM grading system?

These are the rules I'd give anyone building one, in the order I'd apply them.

1. Score per criterion, never holistically. "Rate this answer 1–10" invites vague, length-biased judgements. "Does the answer identify the cause? 0, 1 or 2, quote the sentence" is checkable. This is close to the form-filling approach of G-Eval, which asks the model to reason through evaluation steps before scoring.
2. Evidence before score. Require the model to quote the part of the answer that justifies each criterion score, then give the score. Wang et al. found that generating evidence first reduces bias. It also gives the human reviewer something to check in seconds.
3. Structured output. JSON with a fixed schema per criterion. Validate it, and retry or flag on failure. Free text can't be aggregated.
4. Different families, not different sizes. Two models from the same provider share training data and habits. The PoLL result works because the panel spans families.
5. Aggregate in code. Weighted criteria, then a disagreement check per criterion. A simple rule (for example, flag when judges differ by more than one rubric point) beats a clever one you can't explain to a candidate.
6. Calibrate against humans. Build a gold set: a few hundred items graded by experts, ideally twice by different experts. Measure agreement between the pipeline and humans with quadratic weighted kappa, the metric used in the well-known Hewlett automated essay scoring competition, and compare it with human-to-human agreement on the same set.
7. Re-run the gold set on every change. A new model version, a new prompt or a new rubric is a code change. Treat the gold set as its regression test.

What does multi-model evaluation cost?

More than one call, but less than people fear. Scoring happens once per item, not once per conversational turn, so it's a small share of the running cost of something like a voice interview. And the panel doesn't need frontier models: Verga et al. found their panel of smaller models was over seven times cheaper than one large judge while performing better. Spend the money on diversity, not on size.

What about fairness and the law?

If you score job candidates, you are in regulated territory. New York City's Local Law 144 requires bias audits and candidate notices for automated employment decision tools. The EU AI Act's Annex III lists AI used in employment, including evaluating candidates, as high-risk. The design rules above aren't only good engineering; they're what an auditor will ask for: documented criteria, per-score evidence, logs, and a human who can override.

When should you not use an LLM judge?

• When the answer is checkable by code. Multiple choice, numeric answers and unit tests don't need a model.
• When you have no gold set and no plan to build one. Without calibration you don't know whether the scores mean anything.
• When nobody will review the flagged cases. Disagreement detection only helps if a human acts on it.

If you're building an AI grading or candidate-scoring feature and want it designed so it can stand up to an audit, this is the kind of AI work I take on as a founding engineer: here's how I work, and the engineering page has more depth. For the product side of Brahmo-style grading in an exam-prep app, see the exam-prep app build story.