PLLuM: A Family of Polish Large Language Models

CYFRAGOVPL's Llama-PLLuM-70B-chat is a 70 billion parameter model from the PLLuM family, specifically designed for Polish and other Slavic/Baltic languages, with additional English data for broader generalization. Built on the Llama 3.1 architecture, it leverages an extensive collection of high-quality Polish text data (up to 150 billion tokens) for pretraining and continued pretraining.

Key Capabilities

• Specialized Polish Language Processing: Optimized for generating contextually coherent text in Polish, achieving state-of-the-art results in Polish-language tasks.
• Advanced Instruction Tuning: Refined using a unique, manually curated "organic instructions" dataset (~40k prompt-response pairs, including multi-turn dialogues) to mitigate negative linguistic transfer.
• Preference Learning for Safety and Balance: Incorporates the first Polish-language preference corpus, manually assessed for correctness, balance, and safety, especially for controversial topics.
• Domain-Specific Adaptations: Demonstrates top scores in custom benchmarks relevant to Polish public administration, with specialized RAG-based models for complex information retrieval.
• Robust Context Handling: Features a 32768 token context length, suitable for detailed interactions.

Good For

• General Language Tasks: Ideal for text generation, summarization, and question answering in Polish.
• Domain-Specific Assistants: Particularly effective for applications in Polish public administration, legal, and bureaucratic contexts.
• Research & Development: Serves as a strong foundation for AI applications requiring a deep command of the Polish language.