CYFRAGOVPL/Llama-PLLuM-70B-chat
Llama-PLLuM-70B-chat is a 70 billion parameter large language model developed by CYFRAGOVPL, specialized in Polish and other Slavic/Baltic languages, built upon the Llama 3.1 architecture. It features a 32768 token context length and is refined through extensive instruction tuning and preference learning on high-quality Polish data, including a unique organic instruction dataset and the first Polish-language preference corpus. This model excels in generating contextually coherent Polish text, assisting with tasks like question answering and summarization, and is particularly effective for applications in Polish public administration.
Loading preview...
PLLuM: A Family of Polish Large Language Models
CYFRAGOVPL's Llama-PLLuM-70B-chat is a 70 billion parameter model from the PLLuM family, specifically designed for Polish and other Slavic/Baltic languages, with additional English data for broader generalization. Built on the Llama 3.1 architecture, it leverages an extensive collection of high-quality Polish text data (up to 150 billion tokens) for pretraining and continued pretraining.
Key Capabilities
- Specialized Polish Language Processing: Optimized for generating contextually coherent text in Polish, achieving state-of-the-art results in Polish-language tasks.
- Advanced Instruction Tuning: Refined using a unique, manually curated "organic instructions" dataset (~40k prompt-response pairs, including multi-turn dialogues) to mitigate negative linguistic transfer.
- Preference Learning for Safety and Balance: Incorporates the first Polish-language preference corpus, manually assessed for correctness, balance, and safety, especially for controversial topics.
- Domain-Specific Adaptations: Demonstrates top scores in custom benchmarks relevant to Polish public administration, with specialized RAG-based models for complex information retrieval.
- Robust Context Handling: Features a 32768 token context length, suitable for detailed interactions.
Good For
- General Language Tasks: Ideal for text generation, summarization, and question answering in Polish.
- Domain-Specific Assistants: Particularly effective for applications in Polish public administration, legal, and bureaucratic contexts.
- Research & Development: Serves as a strong foundation for AI applications requiring a deep command of the Polish language.