Skip to content

Posts

  • 2 min read

The Imperative of Danish Foundation Models: Bridging the Linguistic AI Divide

In recent years, the field of machine learning has experienced a transformative shift, primarily driven by the advent of foundation models. These models, pre-trained on vast amounts of data, can be finetuned for various downstream tasks, making them invaluable across multiple domains. However, the dominance of the English language in the development of these models poses significant challenges for smaller language communities. The Danish Foundation Models project emerges as a crucial initiative to ensure that the Danish language does not lag behind in this AI revolution.

  • 5 min read

Datahåndtering

For at kunne træne sprogmodeller (Large Language Models, LLM) skal der store mængder af data til. Fra vi modtager rådata til at de kan bruges til at træne sprogmodeller på, gennemgår de en transformationsprocess.

Følgende er en overordnet beskrivelse af denne processen. Vi udvikler og forbedre løbende processen, for at sikre at vi bruger state-pf-the-art metoder og praksis.

  • 2 min read

Datakilder

De data som sprogmodeller trænes på er afgørende for hvad de kan bruges til. I Danish Foundation Models (DFM) er tilgangen at vi skal have sikkerhed for at vi må benytte de data vi træner på fra data ejere, samt at vi har fokus på værdiskabende use-cases. Dette gør vi blandt andet gennem samarbejdet med Dansk Sprogmodel Konsortium.

  • 5 min read

Tutorial: Finetuning Language Models

This notebook will allow you to try out finetuning of the munin-7b-alpha model or, indeed, any other generative model out there.

We'll be finetuning the model on a Danish translated instruction tuning dataset, using the QLoRA method.

  • 3 min read

Tutorial: Merging Language Models

Model merging is a relatively new method that allows one to combine the weights of different language models into a single model.

In this notebook you'll get to try this out, as well as try to interact with the merged model to see the results!

  • 3 min read

Releasing Munin 7B Alpha - A Danish LLM

We are excited to announce the release of the first model from the Danish Foundation Models project, nicknamed Munin 7B Alpha. This model represents the beginning of our research into Danish Large Language Models (LLMs), employing continual pre-training based on the already pre-trained Mistral-7b-v0.1 model. It has been pre-trained on the Danish Gigaword dataset, which has been instrumental in training various Danish BERT-style models.