CYFRAGOVPL/PLLuM-12B-chat-2512
CYFRAGOVPL/PLLuM-12B-chat-2512 is a 12 billion parameter chat-optimized large language model developed by CYFRAGOVPL under the HIVE AI initiative, based on Mistral-Nemo-Base-2407. It is specialized in Polish, incorporating extensive Polish and English data, and fine-tuned with a large, manually curated Polish instruction dataset and preference corpus. This model excels in general language tasks and is particularly strong in applications for Polish public administration, achieving state-of-the-art results in Polish-language tasks.
Loading preview...
PLLuM-12B-chat-2512: Polish-Centric LLM for Public Administration
PLLuM-12B-chat-2512 is a 12 billion parameter chat model from the PLLuM family, developed by CYFRAGOVPL and HIVE AI. Built upon the Mistral-Nemo-Base-2407 architecture, this model is specifically designed for the Polish language, integrating additional English data for enhanced generalization. Its development involved rigorous data collection, cleaning, and deduplication of large-scale Polish and English text corpora.
Key Capabilities and Development Highlights
- Extensive Polish Data: Trained on a substantial corpus of high-quality Polish text, ensuring deep linguistic understanding.
- Organic Instruction Dataset: Features the largest Polish collection of manually created "organic instructions" (approximately 70k), designed to cover subtle aspects of supervised fine-tuning and mitigate negative linguistic transfer.
- Polish Preference Corpus: Utilizes the first Polish-language preference corpus (~60k manually annotated pairs) to enhance correctness, balance, and safety, particularly for sensitive topics.
- Specialized Benchmarks: Achieves top scores on custom benchmarks relevant to Polish public administration and state-of-the-art results in broader Polish-language tasks.
- Alignment: The
-chatvariant is aligned to human preferences, making it safer and more efficient for dialogue and general-purpose scenarios. - RAG Optimization: Post-training included optimization for Retrieval-Augmented Generation (RAG) scenarios, with a specific Jinja prompt format provided for effective document-based question answering.
Intended Use Cases
- General Language Tasks: Ideal for text generation, summarization, extraction, and question answering in Polish.
- Domain-Specific Assistants: Particularly effective for applications within Polish public administration, legal, and bureaucratic contexts.
- Research & Development: Serves as a robust foundation for AI applications requiring strong command of the Polish language.