CYFRAGOVPL/PLLuM-12B-nc-instruct
CYFRAGOVPL/PLLuM-12B-nc-instruct is a 12 billion parameter instruction-tuned large language model developed by a consortium of Polish scientific institutions, led by Politechnika Wrocławska. Built on Mistral-Nemo-Base-2407, it is specialized for Polish and other Slavic/Baltic languages, with a 32768 token context length. This model excels at generating contextually coherent text, question answering, and summarization in Polish, making it ideal for domain-specific applications, particularly within Polish public administration.
Loading preview...
PLLuM: A Family of Polish Large Language Models
PLLuM-12B-nc-instruct is part of the PLLuM family, a suite of large language models developed by a consortium of Polish scientific institutions, with Politechnika Wrocławska as the project leader. This 12 billion parameter instruction-tuned model is built upon the Mistral-Nemo-Base-2407 architecture and is specifically optimized for Polish and other Slavic/Baltic languages, while also incorporating English data for broader generalization. It features a substantial 32768 token context length.
Key Capabilities
- Deep Polish Language Specialization: Trained on up to 150 billion tokens of high-quality Polish text, along with additional Slavic, Baltic, and English data.
- Organic Instruction Tuning: Refined using a unique, manually curated dataset of approximately 40,000 Polish prompt-response pairs, including multi-turn dialogues, designed to capture subtle aspects of human-model interaction.
- Polish Preference Learning: Utilizes the first Polish-language preference corpus for alignment, enhancing correctness, balance, and safety, particularly for sensitive topics.
- Strong Performance: Achieves state-of-the-art results in general Polish-language tasks and top scores on custom benchmarks relevant to Polish public administration.
- Retrieval Augmented Generation (RAG) Support: Designed to perform well in RAG settings, making it suitable for complex information retrieval and question answering.
Good for
- General Polish Language Tasks: Text generation, summarization, and question answering in Polish.
- Domain-Specific Assistants: Particularly effective for applications in Polish public administration, legal, and bureaucratic contexts requiring domain-aware retrieval.
- Research & Development: Serving as a robust foundation for building downstream AI applications that demand strong command of the Polish language.
This specific model, PLLuM-12B-nc-instruct, is intended for non-commercial use under the CC-BY-NC-4.0 license.