CYFRAGOVPL/Llama-PLLuM-70B-instruct-2512
The CYFRAGOVPL/Llama-PLLuM-70B-instruct-2512 is a 70 billion parameter instruction-tuned large language model, part of the PLLuM family, specialized in Polish with additional English data. Developed by the PLLuM consortium and continued by HIVE AI, it is built on the Llama-3.1-70B architecture. This model excels in generating contextually coherent text in Polish, offering assistance in tasks like question answering and summarization, and is particularly optimized for applications within Polish public administration.
Loading preview...
PLLuM: Polish Large Language Models
CYFRAGOVPL/Llama-PLLuM-70B-instruct-2512 is a 70 billion parameter instruction-tuned model from the PLLuM family, developed by the PLLuM consortium and later HIVE AI. This model is built upon the Llama-3.1-70B architecture and is specifically designed for high performance in the Polish language, augmented with English data for broader generalization.
Key Highlights & Capabilities
- Specialized Polish Data: Trained on extensive, high-quality Polish and English text corpora, rigorously cleaned and deduplicated.
- Organic Instruction Dataset: Features the largest Polish collection of manually created "organic instructions" for supervised fine-tuning, designed to mitigate negative linguistic transfer.
- Polish Preference Corpus: Utilizes the first Polish-language preference corpus, manually assessed for correctness, balance, and safety, especially for sensitive topics.
- State-of-the-Art Performance: Achieves top scores on custom benchmarks relevant to Polish public administration and state-of-the-art results in broader Polish-language tasks.
- Instruction Fine-Tuning: Fine-tuned with approximately 70k manually curated Polish instructions, 33k programmatic instructions, 15k RAG-style instructions, and 45k synthetic, context-aware instructions.
- Alignment and Preference Learning: Aligned using ~60k manually annotated preference pairs to produce safer, balanced, and contextually appropriate responses.
Intended Use Cases
- General Language Tasks: Ideal for text generation, summarization, extraction, and question answering in Polish.
- Domain-Specific Assistants: Particularly effective for applications within Polish public administration, legal, and bureaucratic topics, especially when combined with domain-aware retrieval.
- Research & Development: Serves as a robust foundation for AI applications requiring strong command of the Polish language.