Who judges the AI? Fine-tuning Mistral 7B as a specialized evaluator
By Dorian Rousseau - Project engineer at Euranova
Who judges the AI? Fine-tuning Mistral 7B as a specialized evaluator
Generative AI systems can now produce answers, summaries, recommendations, and analyses in seconds. Evaluating whether those outputs are accurate, relevant, and grounded is much more difficult.
A response can sound convincing while overlooking an important source, contradicting the available evidence or relying on outdated information. As organizations move from isolated prototypes to production AI systems, this creates a fundamental challenge: generating content is becoming increasingly easy, but verifying its quality remains expensive.
Human experts are still the most reliable judges for complex or ambiguous cases. However, asking them to review every output is difficult to scale. Manual evaluation is slow, costly and sometimes inconsistent, particularly when an AI application produces thousands of responses every day.
This raises a broader question:
If AI systems are expected to operate at scale, can part of their quality control also be automated?
The emerging role of AI evaluators
One possible answer is the LLM-as-a-Judge approach: using a language model to evaluate the output of another AI system.
Instead of producing the final answer, the judge examines it according to a defined set of criteria. Depending on the use case, it might determine whether an answer is complete, whether it is supported by the supplied documents, or whether a retrieved source contains enough information to answer a question.
This approach helps organizations continuously monitor AI quality, compare models and prompt strategies, detect regressions before deployment, and reserve human review for high-risk cases.
But using a large proprietary model as the judge introduces another difficulty. Every evaluation becomes an external API call, bringing recurring costs, potential confidentiality constraints, and dependency on an external provider.
At scale, the system responsible for quality control can become almost as expensive and difficult to govern as the application it evaluates.
Does an AI judge need to be a giant model?
This leads to the central hypothesis of our research:
Could a smaller open model be trained to evaluate a specific AI system reliably?
A specialized evaluator does not need to answer every possible question or reproduce the broad capabilities of a frontier model (the state of the art, highly capable generalist LLMs at the technological edge, like GPT 5.6). It only needs to apply a clearly defined evaluation framework consistently.
To test this hypothesis, we fine-tuned Mistral 7B as an automated evaluator for Retrieval-Augmented Generation systems.
In a RAG architecture, the model generates its answer using documents retrieved from an external knowledge base. Evaluating such a system therefore requires more than checking whether the final text sounds plausible. We also need to determine whether the retrieved information was relevant and whether the answer was properly supported by that information.
Our Mistral 7B judge was trained and evaluated across four scenarios:
- Binary classification: classifying an example as insufficient or sufficient
- Three-class classification: assigning a numerical class of 0, 1 or 2
- Semantic three-class classification: selecting insufficient, partial or complete
- Regression: assigning a continuous score between 0 and 1
The objective was not to create a universal judge. It was to determine whether a resource-efficient model could become a dependable specialist inside a controlled evaluation pipeline.
When acceptable accuracy hides a failing model
The first training iterations produced disappointing results. Depending on the task, performance remained close to the level of simple statistical baselines.
More importantly, the model was not truly learning the evaluation criteria. It was largely reproducing the most frequent answer in the dataset. This is known as the majority-class trap.
The synthetic dataset generated for the experiment was severely imbalanced, with the dominant class accounting for approximately 70% of the examples. A model could therefore obtain an apparently acceptable score simply by predicting that class repeatedly.
Consider, for example, a customer support system classifying incoming emails as "Urgent" or "Routine." If 90% of past tickets were labeled "Routine," a model might achieve 90% accuracy simply by flagging everything as "Routine", effectively ignoring every urgent request. While the accuracy score looks impressive, the model has failed entirely: it didn't learn to identify urgency; it only learned to play the odds and identify which prediction is statistically safest.
This illustrates an important limitation of accuracy as a standalone metric:
A model can produce the right answer frequently without learning the right reasoning.
The first fine-tuned version reached approximately 69% to 71% accuracy on the classification tasks. However, its behaviour and its similarity to the majority class baseline showed that these numbers did not yet represent a reliable evaluation capability.
The real problem was hidden in the training objective
The failure was not caused by a lack of model capacity. It originated in the way the training examples and the loss function (a mathematical way of measuring how far the AI's prediction is from the correct answer, which the model tries to minimize during training to improve its performance) had been structured. Each example combined the original question, the generated answer, the retrieved context and the expected evaluation. Together, these elements represented approximately 3,500 tokens. The actual score the model needed to learn represented only around four tokens at the end of this long sequence.
In the original Language Modeling format, the model calculated its error across the entire example. It was therefore trained to reproduce thousands of contextual tokens, while the small evaluation output contributed almost nothing to the overall learning signal. In practical terms, we were asking the model to find a four token whisper in a 3,500-token room.
A completion only loss mechanism was intended to solve this problem by masking the context and calculating the error only on the expected answer. However, the initial dataset stored the prompt and the answer as one continuous block of text. Without a clear programmatic boundary between them, the masking mechanism could not operate correctly.
The model was consequently penalized for failing to reproduce the phrasing of the prompt itself rather than being trained exclusively to predict the final evaluation.
The architectural pivot: teaching only what matters
To correct this, we restructured the dataset into a Prompt + Completion structure. The question, generated answer, and context were placed in the prompt, while only the expected evaluation label was placed in the completion.
By enabling the parameter completion_only_loss=True, prompt tokens received a masking value, excluding them from loss calculation. The model could still read the full context, but its gradient updates focused entirely on the target evaluation tokens. A diagnostic check on 50 examples proved the concept, with loss dropping to 105 within ten epochs. At scale across a 1,202 example regression dataset, the original Language Modeling format stagnated at a minimum validation loss of 0.1799, whereas the Prompt + Completion format reduced it to 0.0664, a nearly threefold error reduction.
Before collecting more data or scaling compute, teams must first verify that their training objective accurately isolates the targeted behavior.
Data quality beats raw volume
Correcting the loss function allowed the model to learn, but it did not solve the imbalance in the training data.
We therefore compared three strategies.
1. Majority class truncation
The first approach removed examples from the dominant class until its size was comparable to the combined minority classes.
This prevented the model from optimizing its predictions around a single default answer. However, it also discarded valid examples, reducing both the size of the training dataset and the number of samples available for evaluation.
Balancing the classes improved the distribution, but at the cost of useful information.
2. Loss weighting
The second approach preserved the complete dataset and modified the loss function instead.
Errors on underrepresented classes received a higher penalty, calculated according to the inverse frequency of each class. In theory, this should force the model to pay more attention to rare examples without removing any data.
Although this strategy retained the full evaluation pool, its results remained inconsistent. Increasing the cost of minority-class errors did not completely prevent the model from exploiting the statistical structure of the dataset.
3. Strict bucket balancing
The third strategy restructured the dataset into balanced semantic buckets.
For classification, each bucket represented one target class. For regression, the continuous scoring range was divided into score intervals. Each bucket was then capped at 150 examples. Instead of generating new synthetic data to fill underrepresented categories, the context was manipulated algorithmically (e.g., by dynamically shuffling and injecting negative chunks to lower retrieval scores).
This produced a flat training distribution. The model could no longer improve its loss by predicting the most frequent answer. It had to learn the semantic differences between the examples.
This final strategy produced the strongest results:
| Evaluation task | Accuracy | Evaluation ratio | MAE |
|---|---|---|---|
| Three-class classification | 98.10% | 103/105 | - |
| Semantic three-class classification | 98.10% | 103/105 | - |
| Binary classification | 97.14% | 102/105 | - |
| Regression | 80.95% exact match | 85/105 | 3.05% |
The results support a recurring principle in model alignment: the structure, precision and distribution of the data can matter more than its raw volume. A smaller, balanced dataset teaches the intended distinctions more effectively than a larger dataset dominated by repetitive examples.
Making fine-tuning resource efficient
The project combined several components into a reproducible training pipeline. Ragas was used to orchestrate the generation of synthetic evaluation examples. A capable generator model produced the questions, answers and assessment scenarios required to train the judge. The fine-tuning itself was performed with Hugging Face's SFTTrainer, while QLoRA was used to adapt Mistral 7B efficiently.
Unlike full fine-tuning, QLoRA does not update every parameter in the original model. The base weights are frozen and quantized into a four-bit NF4 representation. Small, trainable low-rank adapters are then inserted into the model's attention layers. Only these adapters are updated during training. This substantially reduces the required GPU memory while preserving the knowledge contained in the base model. It makes specialization possible without retraining seven billion parameters or maintaining a completely separate copy of the original model for every evaluation task.
From efficient training to sovereign AI
The experiments were executed on MareNostrum, a European supercomputing cluster located in Barcelona. While specialized open models reduce provider dependency, genuine technological control also depends on training location and data governance.
European supercomputing initiatives give organizations access to high-performance infrastructure without requiring internal GPU clusters. For Belgian organizations considering a similar approach, our article Sovereign compute at scale: architecting for the Belgian AI Factory Antenna explains how BE-AIFA provides a local access, advisory and compliance layer for navigating the wider EuroHPC ecosystem.
However, training infrastructure differs from production hosting. Teams must architect the entire model lifecycle: validating data, fine-tuning on high performance compute, exporting adapter weights, deploying to low-latency inference environments, and continuously monitoring output against human reference cases.
What this means for AI teams
The experiment demonstrates how a specialized evaluator can serve as a practical core component within an AI quality pipeline. Throughout the development lifecycle, a fine-tuned judge can automatically evaluate large datasets, compare retrieval strategies, benchmark prompt variations, and catch regressions long before a RAG application reaches production. In ongoing operations, it enables continuous quality monitoring across model iterations, sharply reduces reliance on costly proprietary APIs, and intelligently routes high-risk or ambiguous cases to human reviewers.
Ultimately, the value of this approach lies not in removing human oversight, but in shifting where human attention is focused. Instead of manually auditing routine outputs, experts can focus on shaping evaluation criteria, validating representative samples, and investigating edge cases where the automated judge signals uncertainty. In this model, automation delivers scale and consistency, while human expertise retains command over judgment, governance, and exception handling.
The biases that remain
While these results validate the technical feasibility of our approach within this scope, they do not make the model an impartial or universal judge. LLM-based evaluators remain subject to several known biases that must be addressed in future work.
First, position bias can cause the model to favor information simply because it appears early in the prompt. Mitigating this will require randomly shuffling retrieved chunks across context windows to ensure scoring stays consistent. Second, knowledge bias occurs when the judge relies on its pre-trained memory instead of sticking strictly to the provided RAG context. A real issue when internal knowledge conflicts with the source documents. Addressing this will involve stricter prompt optimization through frameworks like DSPy, as well as reward-based penalties for facts absent from the context.
Finally, format bias can lead the model to give higher marks to longer or better-structured answers, even when their factual content is weaker. Robustness testing must therefore evaluate equivalent answers across different lengths and styles, ensuring the judge evaluates the underlying evidence rather than the writing style.
From feasibility to production readiness
While the experimental results are promising, production deployment demands a higher standard of evidence. The strongest classification outcomes were observed across 105 evaluation examples. Confirming that the model successfully learned the targeted tasks, yet highlighting the need for larger, more diverse, test sets to reliably gauge performance in unseen scenarios.
To bridge this gap, the next validation phase must thoroughly stress-test the model in real world conditions. This requires testing on larger datasets strictly separated from synthetic training generation, integrating real world production RAG examples, and expanding evaluation across diverse domains and document structures. Crucially, performance must be benchmarked directly against expert human judgments while undergoing adversarial testing to uncover potential position, knowledge, or format biases. From an operational standpoint, this rollout should be supported by confidence thresholds that automatically escalate uncertain cases to human reviewers, alongside continuous post deployment monitoring to detect model drift over time.
Ultimately, the takeaway is not that a 7B model can unilaterally replace every form of evaluation, but rather that a carefully aligned 7B model can perform exceptionally well when focused on a clearly defined evaluation task.
Conclusion: better alignment, not just bigger models
Our study demonstrates that fine-tuning Mistral 7B as a specialized LLM judge is both technically feasible and effective within its tested scope. Crucially, the decisive factor was not increasing model size, but fundamentally redesigning the learning problem.
The breakthrough came from isolating the true training signal: separating context from the expected evaluation, calculating loss strictly on relevant completions, stripping away majority class shortcuts, and curating a balanced, semantically precise dataset. By pairing these data centric refinements with QLoRA for resource efficient adaptation and running the pipeline on European supercomputing infrastructure, the model moved beyond reproducing statistical patterns to genuinely learning the intended evaluation criteria.
While this does not remove the need for human oversight, independent testing, or careful governance, it establishes a credible blueprint for scalable, controllable, and technologically sovereign evaluation systems. Ultimately, the broader lesson is clear: organizations do not always need a larger model. A carefully aligned architecture, trained on the right signal and supported by the right infrastructure, can readily outperform far more expensive approaches on a well defined task.