Sovereignty Starts at the Tokenizer

The most expensive sovereignty decision in an Arabic AI program is usually made before anyone has picked a model, by people who do not speak Arabic. It is made in the tokenizer.

Sovereign AI is sold as geography. The data stays in the country, the GPUs sit in a national facility, the contract names a local entity, and a minister cuts a ribbon. All of that is real and all of it is the easy half. Everything running on that hardware arrived from somewhere else: the base weights, the vocabulary the model reads with, the training recipe, and the benchmark used to declare the result good. Each of those carries design choices made by teams optimizing for English, and those choices do not stop applying at the border.

Data residency decides where your text sleeps. The tokenizer decides what your language costs. Only one of those appears on every invoice, every latency budget, and every context window you will ever pay for.

What a tokenizer costs a language

A tokenizer is the lookup table that turns text into the numbered pieces a model consumes. It is trained on a corpus, and it learns to keep frequent sequences whole while splitting rare ones into fragments. Aleksandar Petrov and colleagues measured what happens when the same passage is translated into different languages and fed through the same tokenizer. The token counts diverged by as much as fifteen times. That gap opens before the model runs at all, and it persists in tokenizers that were built with multilingual support in mind.

Arabic sits on the wrong side of that gap for a structural reason. Words are built from a consonantal root shaped by a pattern, and prepositions, the definite article, conjunctions and possessive pronouns all attach directly to the word rather than standing apart. A single written Arabic word can carry what English spreads across five. A vocabulary learned mostly from English text has no reason to store that word as a unit, so it arrives at the model in fragments that line up with nothing meaningful.

That fragmentation is billed three times. It is billed in price, because commercial inference is metered per token. It is billed in latency, because output tokens are produced one after another and a longer sequence takes longer to finish. And it is billed in context, because a fixed window holds fewer Arabic words than English ones. The third bill is the one that damages systems rather than budgets. A retrieval pipeline sized for English documents quietly truncates Arabic ones, and nobody sees an error.

The layers you can actually own

The useful move is to stop arguing about whether a system is sovereign and start listing which of the six layers in Figure 1 you control. Data residency and serving are commercial questions with commercial answers. The layer that changed recently is the weights.

Arabic-first open models now exist and are documented in the literature. Jais, released in 2023 by Inception, MBZUAI and Cerebras, is a 13 billion parameter model pretrained on a mixture of Arabic, English and code, and it outperformed the open multilingual models of its size on Arabic. ALLaM came out of Saudi Arabia's SDAIA. Fanar was built end to end at Qatar Computing Research Institute under Hamad Bin Khalifa University, sponsored by Qatar's Ministry of Communications and Information Technology with the stated purpose of enabling sovereign AI development. Fanar Star is 7 billion parameters trained from scratch on roughly a trillion tokens. Fanar Prime is 9 billion, built by continuing training on top of Gemma-2 9B with the same corpus.

That second variant is the pattern worth copying. You do not have to pretrain from zero to own a model. You take open weights, extend the vocabulary so your language stops being fragmented, continue training on your own corpus, and the result is a model you can serve, audit, quantize and modify without asking anyone. Sovereignty at the weights layer is now an engineering project rather than a national program.

Small models are what make independence affordable

There is a persistent assumption that a sovereign model has to be a large one, and it is the assumption that kills these programs at the budget stage. When my co-authors and I assessed the full carbon bill of Noor, a very large Arabic language model, we accounted for data collection and storage, research and development, pretraining, and projected serving. Inference and the exogenous costs around the project moved the total more than people expected. Training is the headline. Serving is the mortgage.

Read that in money and the conclusion is the same. A 7 to 13 billion parameter Arabic-first model that fits on hardware you already have turns sovereignty into a line item. Frontier-scale sovereignty stays a slide. And because the smaller model reads Arabic with an Arabic vocabulary, it also spends fewer tokens per document than a much larger multilingual model, so it wins on latency and unit cost at the same time it wins on quality for the language you care about.

Evaluation is the layer nobody can sell you

The top of the stack is the layer most programs never budget for. ArabicMMLU, published at ACL 2024 by a team at MBZUAI, covers 40 tasks and 14,575 multiple choice questions in Modern Standard Arabic, drawn from school exams across North Africa, the Levant and the Gulf. It is the right place to start and it is not your benchmark. It does not contain your dialect, your regulations, your document formats, or the way your customers actually type when they are annoyed.

A few hundred items written and graded by your own staff, in the dialect your users write in and on the tasks your system is bought to perform, is the one artifact no vendor can supply. It is also the artifact that makes every layer beneath it swappable. When you own the evaluation, changing model providers becomes a Tuesday afternoon. When you do not, you are choosing between vendors on the strength of their own report cards.

What leaders should do

1. Measure tokens per word on your own corpus before you shortlist a model. Take a thousand real documents, run them through each candidate tokenizer, and multiply the result by your price per token and your context window. That number will shape your budget more than any benchmark score on the vendor's slide.
2. Write your sovereignty claim as the six layer table in Figure 1 and mark each layer owned, rented or imported, with a named owner. If the honest version and the publishable version differ, you have found your roadmap.
3. Fund the evaluation set before the model. Commission 300 to 500 items in your dialect and your domain, graded by your own people, versioned in a repository, and re-run against every candidate.
4. Default to the smallest Arabic-first open-weight model that passes that evaluation, extend its vocabulary on your corpus rather than pretraining from scratch, and repeat the comparison every quarter as new open weights appear.

Sovereignty is not a place. It is a list of dependencies you can name, priced and owned deliberately rather than by default. The tokenizer earns first mention because it is the most invisible dependency in the stack and the one that charges you on every single request, in a currency that never appears in the procurement document.

The encouraging part is that the layers have become cheaper to own in the past three years. Arabic-first open weights exist and are documented. Vocabulary extension is well understood. Continued pretraining on a national corpus is within reach of a competent team with a modest cluster. What remains scarce is the willingness to measure what your own language costs you, and to build the evaluation that only you can build.