CYFRAGOVPL/Llama-PLLuM-8B-instruct-2512
The Llama-PLLuM-8B-instruct-2512 model, developed by CYFRAGOVPL as part of the PLLuM consortium and later HIVE AI, is an 8 billion parameter instruction-tuned large language model based on Llama-3.1-8B. Specialized for Polish language tasks, it incorporates extensive high-quality Polish and English data, including a unique Polish preference corpus and organic instruction dataset. This model excels in generating contextually coherent Polish text and is particularly optimized for applications in Polish public administration, achieving state-of-the-art results in relevant benchmarks.
Loading preview...
PLLuM: Polish Large Language Models
CYFRAGOVPL's Llama-PLLuM-8B-instruct-2512 is an 8 billion parameter instruction-tuned model, part of the PLLuM family, built upon the Llama-3.1-8B architecture. Developed by the PLLuM consortium and continued by HIVE AI, this model is specifically designed for the Polish language, integrating additional English data for broader generalization. It leverages extensive, high-quality Polish and English text corpora, rigorously cleaned and deduplicated.
Key Capabilities
- Specialized Polish Language Processing: Optimized for generating contextually coherent text in Polish.
- Extensive Data Collection: Trained on large-scale, high-quality Polish and English text data.
- Organic Instruction Dataset: Utilizes a unique, manually created Polish instruction set (70k instructions) to mitigate negative linguistic transfer and refine supervised fine-tuning.
- Polish Preference Corpus: Features the first Polish-language preference corpus (~60k pairs) for enhanced correctness, balance, and safety, especially in sensitive topics.
- Instruction Fine-Tuning: Fine-tuned with a mix of manually curated, programmatically derived, RAG-style, and synthetic instructions.
- Alignment and Preference Learning: Uses manually annotated preference pairs to produce safer and contextually appropriate responses.
- Strong Performance: Achieves top scores in custom benchmarks relevant to Polish public administration and state-of-the-art results in broader Polish-language tasks.
Good For
- General Language Tasks: Ideal for text generation, summarization, extraction, and question answering in Polish.
- Domain-Specific Assistants: Particularly effective for applications in Polish public administration, legal, and bureaucratic contexts.
- Research & Development: Serves as a robust foundation for AI applications requiring strong command of the Polish language.