NorGPT-369M: A Norwegian Generative Language Model

NorGPT-369M is a 369 million parameter generative pretrained transformer model, part of the NorGLM suite, specifically engineered for the Norwegian language. Built upon the GPT-2 architecture, this model is designed to facilitate text generation and understanding in Norwegian.

Key Capabilities

• Norwegian Language Generation: Optimized for producing coherent and contextually relevant text in Norwegian.
• Multilingual Training: Trained on a substantial 200GB dataset, encompassing approximately 25 billion tokens, including Norwegian, Danish, Swedish, German, and English, which enhances its understanding of related languages.
• GPT-2 Architecture: Leverages the robust and widely understood GPT-2 framework for its generative capabilities.

Good for

• Non-Commercial Norwegian NLP Applications: Ideal for research and development in natural language processing tasks focused on Norwegian.
• Text Generation in Norwegian: Suitable for applications requiring the creation of Norwegian text, such as content generation, chatbots, or creative writing tools.
• Exploring Generative Models for Scandinavian Languages: Provides a strong baseline for projects involving Norwegian and related Nordic languages due to its diverse training data.