CYFRAGOVPL/Llama-PLLuM-70B-chat-2512
The Llama-PLLuM-70B-chat-2512 is a 70 billion parameter large language model developed by HIVE AI, continuing the work of the PLLuM consortium. It is specialized in Polish, built upon the Llama 3.1-70B architecture, and refined with extensive Polish instruction tuning and preference learning. This model excels at generating contextually coherent text in Polish and English, making it highly effective for general language tasks and specialized applications, particularly within Polish public administration.
Loading preview...
PLLuM: Polish Large Language Models
CYFRAGOVPL/Llama-PLLuM-70B-chat-2512 is a 70 billion parameter model from the PLLuM family, developed by HIVE AI (formerly the PLLuM consortium). This model is built on the Llama 3.1-70B architecture and is specifically optimized for the Polish language, while also incorporating English data for broader generalization.
Key Capabilities and Development
- Specialized Polish Data: Trained on large-scale, high-quality Polish and English text corpora, with rigorous cleaning and deduplication.
- Organic Instruction Dataset: Features the largest Polish collection of manually created "organic instructions" (~70k), designed to cover subtle aspects of supervised fine-tuning and mitigate linguistic transfer from non-Polish data.
- Polish Preference Corpus: Utilizes the first Polish-language preference corpus (~60k manually annotated pairs) to teach correctness, balance, and safety, especially for sensitive topics.
- Alignment: Aligned to human preferences for safer and more efficient use in dialogue and general-purpose scenarios.
- Performance: Achieved top scores on custom benchmarks relevant to Polish public administration and state-of-the-art results in broader Polish-language tasks.
Good For
- General Language Tasks: Proficient in text generation, summarization, extraction, and question answering in Polish and English.
- Domain-Specific Assistants: Particularly effective for applications within Polish public administration, legal, and bureaucratic contexts requiring domain-aware retrieval.
- Research & Development: Serves as a robust foundation for AI applications where strong command of the Polish language is essential.
Limitations
Like other LLMs, it may produce hallucinations, and biases might emerge in controversial topics. Very long context tasks can also be challenging.