CYFRAGOVPL/Llama-PLLuM-70B-instruct-2508
The CYFRAGOVPL/Llama-PLLuM-70B-instruct-2508 is a 70 billion parameter instruction-tuned causal language model developed by the HIVE AI Consortium, based on Llama 3.1. Specialized for Polish and other Slavic/Baltic languages, it incorporates extensive high-quality Polish data and advanced alignment techniques. This model excels in general language tasks and is particularly effective for domain-specific applications within Polish public administration, offering a 32768 token context length.
Loading preview...
PLLuM: A Family of Polish Large Language Models
This model, Llama-PLLuM-70B-instruct-2508, is part of the PLLuM family, developed by the HIVE AI Consortium. It is a 70 billion parameter instruction-tuned model built upon Llama 3.1, specifically designed for Polish and other Slavic/Baltic languages, with additional English data for broader generalization. The model leverages a 32768 token context length.
Key Capabilities
- Specialized Language Focus: Optimized for Polish, Slavic, and Baltic languages, with extensive high-quality Polish text data (around 150B tokens) used in pre-training.
- Advanced Instruction Tuning: Fine-tuned with approximately 55k manually curated Polish "organic instructions," 30k programmatically derived instructions, and 55k synthetic, context-aware instructions.
- Preference Learning and Alignment: Utilizes the first Polish-language preference corpus for alignment, focusing on truthfulness, linguistic correctness, safety, fairness, conciseness, coherence, reasoning, and helpfulness.
- Strong Performance: Achieves state-of-the-art results in broader Polish-language tasks and top scores on custom benchmarks relevant to Polish public administration.
- RAG Capabilities: Additionally trained to perform well in Retrieval Augmented Generation (RAG) scenarios, with specific prompt formatting for document-based question answering.
Good For
- General Language Tasks: Text generation, summarization, and question answering in Polish.
- Domain-Specific Assistants: Particularly effective for applications in Polish public administration, legal, and bureaucratic contexts.
- Research & Development: Serving as a foundational model for downstream AI applications requiring strong Polish language command.
Limitations
Like other LLMs, it may exhibit potential hallucinations and biases, despite extensive alignment efforts. Very long context tasks might also pose challenges depending on hardware constraints.