I helped train the largest Swedish language model of its time
SWE-LLM is a large language model that writes proper Swedish. At the time it was the largest Swedish language model. I took part in developing and training it, in a project led by our CTO, Emil Wagman.
Featured in Breakit and Ny Teknik
70Bparameters, fine-tuned from Llama 3 on NVIDIA A100 GPUs
How we built it
We started from Llama 3 70B and fine-tuned it on a curated set of more than 12,700 Swedish texts, about 13.3 million tokens, covering more than 250 kinds of text: news-style articles, professional communication, technical writing and creative writing. Some of it was written as conversations, so it could chat too. We trained it on NVIDIA A100 GPUs.
What it is good at
Grammar, structure and style in standard Swedish, especially in articles, technical reports, explanations and long texts. We released it publicly for researchers, developers and companies, and Breakit wrote about it.
What I took from it
I learned how to collect and clean training data, fine-tune a model, run the training infrastructure and judge how a model behaves in our own language. A lot of that went straight into Incredible.
SWE-LLM gave us practical experience working with Swedish training data and evaluating model behaviour in our own language.
