PLLuM: Polish Large Language Model Family

CYFRAGOVPL/Llama-PLLuM-70B-instruct-250801 is a 70 billion parameter instruction-tuned model from the PLLuM family, developed by HIVE AI. This model is built upon Llama 3.1 and is specifically designed for Polish and other Slavic/Baltic languages, incorporating additional English data for broader generalization. It leverages an extensive, high-quality Polish text corpus (up to 150B tokens) for pretraining.

Key Capabilities

• Specialized Polish Language Understanding: Trained on a massive Polish corpus, ensuring high accuracy and contextual coherence in Polish.
• Organic Instruction Tuning: Fine-tuned with ~55k manually created Polish instruction-response pairs, including multi-turn dialogues, to cover nuanced human-model interactions.
• Polish Preference Learning: Utilizes the first Polish-language preference corpus for alignment, enhancing correctness, balance, and safety across seven criteria (truthfulness, linguistic correctness, safety, fairness, conciseness, coherence & reasoning, helpfulness & instruction-following).
• Retrieval Augmented Generation (RAG): Optimized for RAG scenarios, capable of answering questions based solely on provided documents and citing sources.
• High Context Length: Supports a context window of 32768 tokens.

Good For

• General Language Tasks: Text generation, summarization, and question answering in Polish.
• Domain-Specific Assistants: Particularly suited for applications in Polish public administration, legal, and bureaucratic sectors.
• Research & Development: A strong foundation for building downstream AI applications requiring robust Polish language capabilities.