Small models don’t know EARS. So I’m testing before I train.
I tried open models up to 12B on a simple job: rewrite an existing requirement into EARS notation without inventing anything. None did it reliably. Here is the evaluation I built before training a 350M helper model.
EARS, the Easy Approach to Requirements Syntax, came out of Rolls-Royce engine control work and was published by Alistair Mavin and colleagues in 2009. It is a handful of sentence templates that make requirements less ambiguous. Every safety-critical engineer I know has read about it. Far fewer write in it consistently.
That makes it a good job for a small, local model: take a requirement someone already wrote, and fix its wording and structure without changing what it says. The model does not invent requirements. It is a helper, not an author.
The five patterns
| Pattern | Template | Example |
|---|---|---|
| Ubiquitous | The |
The parking brake shall hold the vehicle on a 30% gradient |
| Event-driven | When |
When the driver pulls the switch, the parking brake shall apply |
| State-driven | While |
While the vehicle is moving, the parking brake shall ignore the switch |
| Unwanted behaviour | If |
If wheel speed signals are lost, then the parking brake shall … |
| Optional feature | Where |
Where auto-hold is fitted, the parking brake shall … |
The examples are mine and illustrative.
What I found zero-shot
I gave the job to open models up to 12B parameters, from the LFM, Qwen and Gemma families, with a clear prompt and examples of each pattern.
| Model group | Picked the right pattern | Kept the original values | Verdict |
|---|---|---|---|
| Open models, up to 12B | Inconsistently | No: invented content that was not in the input | Not usable |
| Frontier model | Yes, first prompt | Yes | Knows EARS |
The second column is the one that matters. A requirement like “the brake should engage quickly” has no time in it. A model that fills one in has produced something worse than useless in a safety process, because it looks right.
The frontier result tells me EARS is in the training data. It is just too rare for small models to keep reliably. That is a fine-tuning problem, not a prompting problem.
Test first, then train
Before generating a single training example, I built the evaluation. The rule I set is simple: anything in the output that was not in the input is a failure.
Figure 1. The gate. Values in the input (numbers, units, signal names) are extracted and diffed against the output. A new value fails the item, whatever else it got right.
| Evaluation design | Choice |
|---|---|
| Suite size | 650 to 1,100 behavioural items, in five test classes |
| Primary gate | Fabrication detection by diffing input and output values. Deterministic, no model judging another model |
| Missing information | The model must emit a flag, for example [MISSING: timing], never a guess |
| Second task | Rewrite test cases into ISO/IEC/IEEE 29119-3 structure. Separate task, same model |
| Training data | Synthetic only. No company data |
| Domains | Automotive, aerospace, medical, industrial |
The second task is deliberately separate. The model corrects a requirement, or it corrects a test case. It does not derive tests from requirements. Keeping the scope that narrow is what makes “no fabrication” testable.
Why so small
| Candidate | Size | Why it is on the list |
|---|---|---|
| LFM2.5 | 230M | Smallest; runs anywhere, even on a laptop CPU |
| LFM2.5 | 350M | My starting point: room for two tasks |
| A 1.2B model | 1.2B | Fallback if 350M cannot hold both tasks |
The reason is privacy. Requirements are some of the most sensitive documents an engineering company owns. A model small enough to run on the engineer’s own machine means the text never leaves it.
What I would tell any team doing this
Write the evaluation before the training set. Make your primary metric something a script can check, not something another model has to judge. And for anything safety-relevant, measure what the model added, not just what it fixed. The most dangerous output is the one that is fluent, plausible and wrong.