A Sixth of the Size

There is a number in the National Library of Sweden's model card that deserves more attention than it has received, and it is not the one they put in the headline.

KBLab, the data lab at the National Library of Sweden, released a suite of Swedish speech recognition models in February 2025 under the name KB-Whisper. They are built on OpenAI's Whisper and fine-tuned on more than 50,000 hours of Swedish speech. The headline claim on their own model card is that the best performing model reduces the word error rate by an average of 47 percent compared to OpenAI's whisper-large-v3.

That is a good number. It is not the interesting one.

The interesting one is on the middle row of the same table.

KB-Whisper small has a lower error rate than OpenAI's largest Whisper model. On all three test sets. Small has 244 million parameters, large-v3 has 1.55 billion — a factor of 6.4. KBLab themselves write that their small model beats “a model six times its size”.

One sixth of the parameters, and it wins.

What the numbers actually are

WER, word error rate, is the share of words that come out wrong — insertions, deletions and substitutions counted against a reference and divided by the length of that reference. Lower is better. A 5.4 means that just over five words in a hundred are incorrect.

The three columns are three public Swedish test sets. FLEURS is Google's multilingual corpus, in which a handful of native speakers read the same sentences in 102 languages. Common Voice is Mozilla's crowd-sourced recordings, of which 54 hours are Swedish. NST is the set of speech corpora collected by the company Nordisk Språkteknologi and today maintained by the Language Bank at the National Library of Norway.

Let me take the 47 percent apart, because the average hides a spread worth seeing. Compared with whisper-large-v3, KB-Whisper large cuts errors by 57 percent on Common Voice, 54 percent on NST — and 31 percent on FLEURS. The average comes to 47. So the claim that the model more than halves the errors holds on two of the three test sets, not on all three. It is a small thing. It is also exactly the kind of small thing that separates a number you can trust from one you cannot.

The thin layer

whisper-large-v3 is not a bad model for Swedish. It is a model expected to handle about a hundred languages, and Swedish is one of them. That relationship is the whole point.

KBLab report in their paper exactly how thin the layer is. whisper-large-v3 was trained on 680,000 hours of speech. Of those, 2,119 hours were Swedish. That is three parts per thousand.

KBLab's own corpus looks different: 34,261 hours of subtitled material — 18,468 from SVT, 8,629 from the Swedish Media Database and 7,164 from YouTube — plus 21,949 hours of parliamentary debate, 250 hours of NST and 54 hours of dialect recordings from the Institute for Language and Folklore. A total of 56,514 hours in the first training stage.

Twenty-seven times more Swedish than the base model ever heard. And the figure of 50,000 is not arbitrary: KBLab point out that the Whisper authors themselves reported that improvement begins to level off somewhere around there, and built the corpus accordingly.

That is where the finding sits, and it is not a Swedish finding:

When a language or a domain sits thin in the base model's training data, adaptation buys more than size does.

A model one sixth the size with enough of the right data beats one six times larger that got three parts per thousand. This says nothing about Swedes being more capable than Americans. It says something about where the marginal return sits when the underlying data is skewed — and that conclusion transfers to any domain underrepresented in what the models were trained on. Technical language. Industry terminology. Clinical notes. Image domains outside the internet photograph.

It is also an argument that cuts directly against the easiest thing to sell, namely the next larger model.

The measurement on my own desk

Today I downloaded KB-Whisper large to a Mac mini and ran it.

What surprised me was that the planned conversion turned out to be unnecessary. mlx_audio 0.4.3 reads the Hugging Face format directly: ModelDimensions.from_dict maps the configuration's d_model and encoder_layers onto mlx's own names, and Model.sanitize renames the weights — encoder.layers. becomes encoder.blocks., self_attn.q_proj becomes attn.query. KBLab's official safetensors load as they are, without a single weight passing through a conversion tool. Third-party conversions of the same model exist on Hugging Face; they were deliberately passed over, because the uploaders are unknown and there is no reason to trust them when the original works.

The test: 40.48 seconds of Swedish news material against a known reference. The audio was synthetically read — it is one of the daily audio briefs this site publishes itself — and that matters for how the result should be read. More on that shortly.

Result: one error in 94 words. 1.1 percent. The error was Xeon, Intel's processor family, heard as Action.

Timings: the model loaded in 13.0 seconds cold and 1.5 seconds warm. The transcription of the 40.48 seconds took 13.3 seconds the first time and 4.4 seconds the second — that is 3.0 times real time cold and 9.2 times warm. A ten-second spoken command becomes text in about a second once the model is loaded.

KB-Whisper large weighs 3.22 GB on disk, small 0.56. The whole chain runs locally: the speech recognition, the language model and the speech synthesis sit on the same machine, and no step has a cloud fallback. That is not a principle I am asserting here, only a description of the setup.

What the measurement does not say

Here is the part the rest of the text stands or falls on, because 1.1 percent is a number that is easy to quote wrongly.

One error in 94 words is one error. The uncertainty around a single outcome is wide enough that the figure cannot meaningfully be distinguished from zero, or from three percent. It is a spot check, not a measurement series, and I am reporting it as a spot check.

Everything measured here is read speech. FLEURS is read sentences. Common Voice is read sentences. NST consists, by KBLab's own description, of scripted sentences and isolated words. My own test was not even a human reading, but a speech synthesizer. Nothing in this material says anything about conversation, about two people talking over each other, about telephone audio, about a workshop with machinery in the background, or about someone interrupting themselves mid-sentence. That is where speech recognition normally breaks, and it is precisely the case none of these numbers illuminate.

One of the three columns is home ground. KBLab state plainly that Common Voice and FLEURS are not part of the training data and therefore measure the model's out-of-domain performance. NST does not qualify: 250 hours of NST sit in the training corpus, and the test is a held-out portion of the same collection. This is correct method and KBLab report it openly — but it means the NST column measures something different from the other two. Worth noting: NST yields the second largest gain, 54 percent, while FLEURS — the cleanest comparison — yields the smallest, 31.

The audio file was truncated. The script ran longer than the roughly 40 seconds actually recorded, and the audio stopped mid-sentence. The model then hallucinated a tail: twelve words of pure invention after the audio fell silent. Counted across the whole passage up to the cut, that gives 14 errors in 95 reference words, or 14.7 percent instead of 1.1. I report both figures, because the difference between them is not measurement uncertainty but a known property: Whisper models invent when audio is cut abruptly. The practical lesson is concrete enough to be worth writing down — send whole utterances to the recognizer, never clipped segments, and trim the silence at the end.

A second test failed, and I report no figure from it. I wanted to know how the model handles my own domain vocabulary — place names, company names and shop-floor terms from the industry I work in. The test was made with synthetic speech, and the synthesizer turned out to have rewritten the text before reading it: the first line turned a single name into four entirely different words. The test therefore measures speech synthesis and speech recognition in series, and the errors cannot be separated. Did the synthesizer say the word correctly and the model mishear it, or did the synthesizer already say it wrong? The material cannot answer that, and a figure drawn from it would be invented precision. The test failed on method. The only thing that can be said is that the ordinary Swedish in the same audio file was transcribed correctly throughout. The question of the domain vocabulary remains open, and what answers it is a recording of a real voice.

And the measurer is also the measured. The table is KBLab's own, produced by the same lab that built the models, and no independent replication is reported. To this comes a detail KBLab themselves raise as a weakness: the training data was filtered using whisper-large-v3 — that is, using the very model the comparison is then made against. Three test sets are, moreover, three test sets. This is strong circumstantial evidence. It is not proof that the model is better everywhere.

Why it matters anyway

The model dates from February 2025. It has been open under Apache 2.0 for a little over eighteen months, the work was presented at Interspeech 2025, and the large version is downloaded on the order of seventy thousand times a month according to Hugging Face's counter, read on September 9, 2026. Nothing in this text concerns a new release, and that is deliberate. A measurement result does not age the way a release does.

What makes the sixth interesting is not national. It is that it moves a boundary for what can be done without the cloud. A model of 0.56 GB that beats one of 3.09 is the difference between speech recognition as a service you buy and speech recognition as a file you hold. For everything that cannot leave the building — patient audio, negotiations, recordings from a production floor, government material — that is not an optimization but a precondition.

And for everyone else sitting on material that lies thin in the large models' training, the measurement points in a definite direction: the cheap path is not to wait for the next larger model. It is to take a smaller one and give it the right data.

It is not a glamorous conclusion. It merely happens to be the one the numbers support.