CYFRAGOVPL/Llama-PLLuM-8B-instruct
CYFRAGOVPL/Llama-PLLuM-8B-instruct is an 8 billion parameter instruction-tuned causal language model based on Llama 3.1, developed by a consortium led by Politechnika Wrocławska. This model is part of the PLLuM family, specialized in Polish and other Slavic/Baltic languages, with additional English data for broader generalization. It is refined through instruction tuning on a large Polish instruction dataset and preference learning, excelling in generating contextually coherent text and assisting with tasks like question answering and summarization, particularly for Polish public administration use cases. The model has a context length of 32768 tokens.
Loading preview...
PLLuM: Polish Large Language Models
CYFRAGOVPL/Llama-PLLuM-8B-instruct is an 8 billion parameter instruction-tuned model from the PLLuM family, built upon Llama 3.1. Developed by a consortium led by Politechnika Wrocławska, PLLuM models are specifically designed for Polish and other Slavic/Baltic languages, incorporating English data for enhanced generalization. This model leverages a 32768 token context window.
Key Capabilities
- Specialized Polish Language Processing: Extensive training on up to 150 billion tokens of Polish text, including a unique 40k manually created "organic instructions" dataset and the first Polish-language preference corpus.
- Instruction Following: Refined through supervised fine-tuning (SFT) and alignment techniques to generate contextually coherent and helpful responses.
- High-Quality Alignment: Utilizes a demographically diverse team of annotators for preference learning, ensuring linguistic correctness, balance, and safety, especially for sensitive topics.
- Domain-Specific Performance: Achieves top scores on custom benchmarks relevant to Polish public administration tasks and state-of-the-art results in broader Polish-language tasks.
- Retrieval Augmented Generation (RAG): Trained to perform effectively in RAG settings, providing answers based on provided documents with citations.
Good For
- General Polish Language Tasks: Text generation, summarization, and question answering in Polish.
- Domain-Specific Applications: Particularly effective for intelligent assistants in Polish public administration, legal, or bureaucratic contexts.
- Research and Development: Serving as a foundational model for AI applications requiring strong command of the Polish language.