CYFRAGOVPL/Llama-PLLuM-70B-instruct-250801
Llama-PLLuM-70B-instruct-250801 is a 70 billion parameter instruction-tuned large language model developed by HIVE AI, based on Llama 3.1. It is specialized in Polish and other Slavic/Baltic languages, trained on extensive Polish corpora (up to 150B tokens) and refined with a large, manually curated Polish instruction dataset and preference corpus. This model excels in general language tasks and is particularly effective for applications in Polish public administration and legal domains, supporting a 32768 token context length.
Loading preview...
PLLuM: Polish Large Language Model Family
CYFRAGOVPL/Llama-PLLuM-70B-instruct-250801 is a 70 billion parameter instruction-tuned model from the PLLuM family, developed by HIVE AI. This model is built upon Llama 3.1 and is specifically designed for Polish and other Slavic/Baltic languages, incorporating additional English data for broader generalization. It leverages an extensive, high-quality Polish text corpus (up to 150B tokens) for pretraining.
Key Capabilities
- Specialized Polish Language Understanding: Trained on a massive Polish corpus, ensuring high accuracy and contextual coherence in Polish.
- Organic Instruction Tuning: Fine-tuned with ~55k manually created Polish instruction-response pairs, including multi-turn dialogues, to cover nuanced human-model interactions.
- Polish Preference Learning: Utilizes the first Polish-language preference corpus for alignment, enhancing correctness, balance, and safety across seven criteria (truthfulness, linguistic correctness, safety, fairness, conciseness, coherence & reasoning, helpfulness & instruction-following).
- Retrieval Augmented Generation (RAG): Optimized for RAG scenarios, capable of answering questions based solely on provided documents and citing sources.
- High Context Length: Supports a context window of 32768 tokens.
Good For
- General Language Tasks: Text generation, summarization, and question answering in Polish.
- Domain-Specific Assistants: Particularly suited for applications in Polish public administration, legal, and bureaucratic sectors.
- Research & Development: A strong foundation for building downstream AI applications requiring robust Polish language capabilities.