CYFRAGOVPL/PLLuM-4B-base-2512

PLLuM: A Family of Polish Large Language Models

PLLuM models are a series of large language models (LLMs) developed by the PLLuM consortium and later by HIVE AI, with a strong focus on the Polish language, augmented by English data for broader generalization. The PLLuM-4B-base-2512 is a 4 billion parameter model based on gemma-3-4b-pt.

Key Capabilities

• Extensive Polish Data Collection: Trained on large-scale, high-quality Polish and English text corpora, rigorously cleaned and deduplicated.
• Organic Instruction Dataset: Features the largest Polish collection of manually created "organic instructions" for supervised fine-tuning, designed to mitigate negative linguistic transfer.
• Polish Preference Corpus: Includes the first Polish-language preference corpus, manually assessed for correctness, balance, and safety, especially for controversial topics.
• Specialized Evaluation: Achieves top scores on custom benchmarks relevant to Polish public administration and state-of-the-art results in broader Polish-language tasks.
• RAG Optimization: Post-trained to perform well in Retrieval-Augmented Generation (RAG) settings, with a specific prompt format for document-based question answering.

Good For

• General Language Tasks: Ideal for text generation, summarization, extraction, and question answering in Polish.
• Domain-Specific Assistants: Particularly effective for applications within Polish public administration, legal, or bureaucratic contexts requiring domain-aware retrieval.
• Research & Development: Serves as a robust building block for AI applications where a strong command of the Polish language is essential.