PLLuM: Polish Large Language Models

CYFRAGOVPL/Llama-PLLuM-8B-instruct is an 8 billion parameter instruction-tuned model from the PLLuM family, built upon Llama 3.1. Developed by a consortium led by Politechnika Wrocławska, PLLuM models are specifically designed for Polish and other Slavic/Baltic languages, incorporating English data for enhanced generalization. This model leverages a 32768 token context window.

Key Capabilities

• Specialized Polish Language Processing: Extensive training on up to 150 billion tokens of Polish text, including a unique 40k manually created "organic instructions" dataset and the first Polish-language preference corpus.
• Instruction Following: Refined through supervised fine-tuning (SFT) and alignment techniques to generate contextually coherent and helpful responses.
• High-Quality Alignment: Utilizes a demographically diverse team of annotators for preference learning, ensuring linguistic correctness, balance, and safety, especially for sensitive topics.
• Domain-Specific Performance: Achieves top scores on custom benchmarks relevant to Polish public administration tasks and state-of-the-art results in broader Polish-language tasks.
• Retrieval Augmented Generation (RAG): Trained to perform effectively in RAG settings, providing answers based on provided documents with citations.

Good For

• General Polish Language Tasks: Text generation, summarization, and question answering in Polish.
• Domain-Specific Applications: Particularly effective for intelligent assistants in Polish public administration, legal, or bureaucratic contexts.
• Research and Development: Serving as a foundational model for AI applications requiring strong command of the Polish language.