Mille-Pensées is one of the best 7b LLM that reasons in French that we have post-trained from a Qwen2.5. When you ask most top recent 7b LLMs to solve a maths question in French, most of them will correctly answer in French, but will reason in English. We have therefore built a post-training pipeline and data-mix that makes the LLM also reason in French, and applied this pipeline to produce the Mille-Pensées LLM. Mille-Pensées reasons in French but also get better results than Qwen2.5-Maths on French Maths benchmarks. Furthermore, Mille-Pensées also improves over Qwen2.5-maths on English Maths benchmarks, which suggests that our French Maths datamix is competitive with previous pipelines.
Mille-Pensées and its dataset are freely released and available on Huggingface Hub: https://huggingface.co/GLauzza/Mille-Pensees
Contributors: Gabriel Lauzzana (main), Imane Ouada, Christophe Cerisara
Other LLMs that we have contributed to in 2025 can be found at this news webpage.