CYFRAGOVPL/Llama-PLLuM-70B-chat-2412
The CYFRAGOVPL/Llama-PLLuM-70B-chat-2412 is a 70 billion parameter large language model from the PLLuM family, developed by a consortium of Polish scientific institutions led by Politechnika Wrocławska. Built on Llama 3.1, it is specialized in Polish and other Slavic/Baltic languages, with additional English data for generalization. This chat-tuned model excels in generating contextually coherent text, question answering, and summarization, achieving state-of-the-art results in Polish-language tasks and top scores in Polish public administration benchmarks.
Loading preview...
PLLuM: Polish Large Language Models
CYFRAGOVPL/Llama-PLLuM-70B-chat-2412 is a 70 billion parameter model from the PLLuM family, developed by a consortium of Polish scientific institutions. This model is built upon Llama 3.1 and is specifically designed for Polish and other Slavic/Baltic languages, incorporating English data for broader applicability. It has been extensively refined through instruction tuning, preference learning, and advanced alignment techniques.
Key Capabilities & Features
- Specialized Multilingualism: Optimized for Polish, Slavic, and Baltic languages, with strong performance in English.
- Extensive Polish Data: Pretrained on up to 150 billion tokens of high-quality Polish text, alongside other languages.
- Organic Instruction Tuning: Fine-tuned on a unique dataset of ~40k manually created Polish "organic instructions," including multi-turn dialogues, to capture subtle aspects of human-model interaction.
- Polish Preference Corpus: Utilizes the first Polish-language preference corpus for alignment, enhancing correctness, balance, and safety, particularly for sensitive topics.
- State-of-the-Art Performance: Achieves top scores in custom benchmarks for Polish public administration tasks and state-of-the-art results across broader Polish-language tasks.
- Chat Alignment: The
-chatsuffix indicates it has been aligned on human preferences, making it generally safer and more efficient for dialogue and general-purpose scenarios.
Intended Use Cases
- General Language Tasks: Ideal for text generation, summarization, and question answering in Polish and related languages.
- Domain-Specific Assistants: Particularly effective for applications in Polish public administration, legal, and bureaucratic contexts, especially when combined with Retrieval Augmented Generation (RAG).
- Research & Development: Serves as a robust foundation for building downstream AI applications requiring strong Polish language command.