The project in detail
From corpus building to publishing an open model: the roadmap, the scientific method, the needs, the funding, and the questions you are asking.
Where the project stands
Lingua Nostra is today a research project at the concept stage. No prototype exists yet, and that is the logical consequence of where we currently stand.
« Notre priorité du moment est la constitution du corpus et la recherche des premiers partenaires scientifiques et financiers. »
Before building a reliable artificial intelligence for a regional language, it is essential to assemble a quality corpus, obtain the necessary permissions, and bring together the scientific and technical resources needed. That preparatory work is precisely what Lingua Nostra is engaged in right now. The project stands at its beginning, and that phase is already under way.
Already identified and in the public domain:
Still to be surveyed and negotiated:
Assembling this corpus is the first phase of the project.
Roadmap
The project advances by milestones, not by dates. Each phase prepares the next. Here is the planned progression.
Gathering and digitising Provençal texts: Félibrige almanacs and press, reference dictionaries, Provençal literature. Optical character recognition on older editions, and re-running OCR on all the texts available at the BnF. This is one of the longest tasks of the project, and probably one of the most decisive.
Correcting recognition errors, removing duplicates, balancing, shuffling, building corpora with particular emphases, and formatting these sets into a standard suitable for training.
Starting from a large multilingual model and adapting it to Provençal in three steps: continued pre-training, where the model absorbs the written language; instruction fine-tuning, where it learns to follow instructions rather than merely continue text, using thousands of question-and-answer pairs drafted and corrected by native speakers; then alignment to their preferences, which anchors the register, accuracy and relevance of the answers.
A network of native speakers validates, corrects and decides through simple interfaces designed for a non-technical audience. Speakers, together with the quality of the corpus, are a major factor of success. Without them, there is probably no quality model: this is what we will try to demonstrate by comparing with a baseline model not adjusted by speakers.
Releasing the model and the code under an open licence, self-hostable, together with corpus documentation. The tool then belongs to the language community.
Transposing the method proven on Provençal to other languages of France. The second language targeted would be Breton, in particular its Vannetais variety. In the longer term, other languages of the Midi could each receive their own model. Lou Pichoun serves as a laboratory for the future.
Scientific methodology
Why a corpus is indispensable. In natural language processing (NLP), a large language model learns by observing vast quantities of text. For it to speak Provençal, it must be exposed to Provençal in sufficient quantity and quality. The value of the final artificial intelligence model would depend directly on that of the corpus.
Why adapting an existing large model rather than starting from scratch. Training a language model from nothing requires volumes of data and computing power far beyond the reach of a project like this one. A large multilingual model has already learned the general structure of language; it then only needs to be reoriented toward Provençal. Several low-resource languages, from Turkish to Irish, have been adapted this way with modest means, even without a closely related high-resource language to draw on.
The three stages of training. First, continued pre-training, where the model absorbs the written language and learns to extend its sentences naturally. Then instruction fine-tuning, a stage common to every language assistant: the model learns to follow instructions using thousands of question-and-answer pairs covering translation, grammar, culture and conversation. What is specific to Provençal is that these pairs are drafted and corrected one by one by native speakers, making this the most demanding human contribution to the project. Finally, preference alignment, where its style is refined by showing it, between two answers, which one a speaker prefers.
How validation works. The model is evaluated separately in each dialect. During fine-tuning, it systematically produces two answers; a Rhodanian or Maritime speaker selects the best one in their own dialect. Each dialect could thus be validated by its own speakers.
How performance is measured. Several instruments complement one another: perplexity, which measures the model's ease with unseen text; fill-in-the-blank tests, where a word is masked to check whether the model recovers it; a question-and-answer set on Provençal culture and literature; and above all an assessment by several independent experts judging grammaticality, linguistic accuracy and register naturalness.
The long-term ambition: a free Provençal voice. Natural language processing does not stop at text. In time, Lou Pichoun aims to add an open-source Provençal speech synthesis system (Text-to-Speech, TTS), trained on recordings of consenting native speakers. The challenge is not merely technical: generating a voice is today a solved problem. What matters is the governance of that voice: informed consent, open weights, a voice belonging to the community, quality on a par with proprietary solutions. It is this ethical and sovereign framework that will make the Provençal voice a lasting resource for Provence, rather than an export product. Automatic speech recognition (transcription) would complete the loop: understanding, responding, and reading aloud in Provençal.
Challenges and uncertainties
A research project has unknowns. Stating them clearly is part of what makes an approach serious.
Writing in two dialects. The model will need to produce equally well in Rhodanian and Maritime Provençal. This is a real challenge, doubling the corpus, validation and evaluation work.
The quality of the generated Provençal. Achieving an irreproachable standard of language remains the main unknown; only speaker validation will resolve it.
A challenge already under control: mobilising speakers. The network of associations would make it possible to bring together the speakers needed to correct and fine-tune the model.
Needs and funding
Funding Lingua Nostra means making specific resources possible. This is of course not a detailed budget, but an outline of prospects.
Surveying Provençal texts, identifying dormant holdings at universities and associations, locating rights holders.
Scanning works not yet digitised, applying optical character recognition to older editions, proofreading after machine processing.
Mobilising and supporting the speakers who will correct the model's outputs and choose between two versions.
Renting the computing power needed for training, on specialised servers, for the duration of the intensive phases.
Drawing on expertise in Provençal and sociolinguistics to ensure rigour of language and dialectal choices.
Building the training pipeline, the community validation interface and the self-hostable final tool.
Lingua Nostra's funding rests on several successive sources. A first crowdfunding model could launch the project; we will also apply for institutional, private and even academic funding.
The primary purpose of the funds is to pay for the working time needed to build the corpus and the various tools, then to train and publish the Provençal AI.
Institutional, private and academic funding. A complete application file has already been assembled to pursue these sources of support. The Lingua Nostra project takes the form of two entities: an association working for the digital flourishing of minoritised language communities in France, the Provençal-speaking community first and foremost, and a single-shareholder company (SASU) developing natural language processing tools and specialising in training sovereign language models and generative artificial intelligence.
Follow the project
The project is in its early stages. Leave your address to follow its progress and be notified of the launch of Lou Pichoun and of each milestone along the way.
Scientific references
Adapting a large model to a low-resource language is not a hypothesis: several teams have done it, sometimes with very little data and modest hardware. Here are the works that mark the way.
Models for minority languages
A bilingual Irish-English model and the project's primary methodological reference: continued pre-training followed by fine-tuning, validated by native speakers. Approximately 44 hours of computation on two graphics processors.
The Irish pioneer: continued pre-training with vocabulary extension. The reference for adaptation methods.
Adaptation to Basque, a language structurally more distant from French than Provençal. Evaluated on real language examinations.
Adaptation on near-consumer hardware, starting from roughly 274 million words. The most directly reproducible case.
Usable results from a tiny corpus, comparable to the available Provençal corpus. Proof that a small volume is enough to get started.
Ayla Rigouts Terryn and Justin Royer (Université de Montréal) are developing part-of-speech tagging and lemmatisation for Chuj, an under-documented Mayan language, from field data. A doctorate devoted to these tools began in 2026.
Minoritised languages and digital flourishing
Mélanie Jouitteau (IKER, CNRS) calls for public, scientific and industrial mobilisation to make minoritised languages compatible with artificial intelligence, without which linguistic diversity risks a massive reduction.
Megan Cotnam-Kappel, holder of the Research Chair on the digital flourishing of Franco-Ontarian communities (University of Ottawa), defines this field as supporting a dynamic online presence, individual and collective agency, equitable access to technology and digital skills. Article: Megan Cotnam-Kappel and Jean-Luc Ciocca, « Les inégalités d'accès, de compétences et de pouvoir d'agir numériques et linguistiques des élèves : perspectives du personnel enseignant en Ontario français », Minorités linguistiques et société, no. 23.
Training, adaptation and evaluation of models
Ikram Belmadani, Benoît Favre, Richard Dufour, Frédéric Béchet and Carlos Ramisch (TALN 2025) compare continued pre-training, supervised fine-tuning and their combination to adapt a general model to the French medical domain with limited data.
Elie Antoine, Frédéric Béchet, Géraldine Damnati and Philippe Langlais (EMNLP 2024) show, on a French benchmark, that some examples make models fail regardless of their size, because of their linguistic complexity.
Hichem Ammar Khodja, Frédéric Béchet, Quentin Brabant, Alexis Nasr and Gwénolé Lecorvé (LREC-COLING 2024) propose a dataset that tracks how facts change over time, to add, correct or remove knowledge in an already trained language model without retraining it.
Ahmed Hamdi, Alexis Nasr, Nizar Habash and Núria Gala (WANLP 2015) build a part-of-speech tagger for the Tunisian dialect, a low-resource language, by drawing on the resources of closely related Modern Standard Arabic.
Frequently asked questions
Because Provençal is seriously endangered: this project is first and foremost an act of preservation. Because Provençal, legible to French speakers from the region, is a lever for language reclamation. Because it is the language of the project's founder, who has access to the speaker network needed for validation. And because no model speaks Provençal today: this would be a first.
Large commercial models do not master Provençal: the language is almost entirely absent from their training data. When prompted, they produce an approximate Provençal mixed with French. They can neither speak it properly nor serve as a judge. Moreover, relying on them means outsourcing the preservation of intangible heritage to private companies, most of them outside France.
A dedicated model, trained on a Provençal corpus validated by native speakers, reaches a quality that no general-purpose model offers for Provençal. It can be self-hosted, published under an open licence, and remain under the community's control rather than depending on a proprietary service.
The realistic target is between 50 and 100 million words of Provençal. That may seem modest, but the Northern Sámi model produced usable results with four times less. The available Provençal corpus falls within this viable range.
For technological independence and data sovereignty. A self-hosted model depends on no company, cannot be withdrawn or modified by a third party, and guarantees the longevity of the tool and control over the data of the speakers who made it possible.
So that the tool belongs to the language community rather than to any institution. An open licence allows anyone to use, study, improve and redistribute the model. That is the condition for genuine digital sovereignty of the language.
Because the quality of a model depends first of all on the quality of its corpus. Before any training can begin, that corpus must be assembled, permissions obtained, and scientific and technical resources secured. That preparatory phase is where the project stands today: a premature prototype would tell us nothing reliable.