CYFRAGOVPL/PLLuM-12B-instruct-2412

TEXT GENERATIONConcurrency Cost:1Model Size:12BQuant:FP8Ctx Length:32kTool Calling:SupportedPublished:Feb 7, 2025License:apache-2.0Architecture:Transformer0.0K Open Weights Cold

The PLLuM-12B-instruct-2412 is a 12 billion parameter instruction-tuned causal language model developed by CYFRAGOVPL, based on Mistral-Nemo-Base-2407. It is specialized in Polish and other Slavic/Baltic languages, with additional English data for generalization, and features a 32768 token context length. This model excels at generating contextually coherent text and assisting with tasks like question answering and summarization, particularly in Polish public administration contexts.

Loading preview...

Overview

CYFRAGOVPL's PLLuM-12B-instruct-2412 is a 12 billion parameter instruction-tuned model from the PLLuM family, built upon Mistral-Nemo-Base-2407. It is specifically designed for Polish and other Slavic/Baltic languages, incorporating English data for broader applicability. The model leverages an extensive Polish text corpus (up to 150B tokens for non-commercial versions) and a unique collection of ~40k manually created "organic instructions" in Polish, including multi-turn dialogues, to enhance its fine-tuning and mitigate negative linguistic transfer.

Key Capabilities

  • Multilingual Proficiency: Specialized in Polish, Slavic, and Baltic languages, with English support.
  • Instruction Following: Refined with a large, manually curated Polish instruction dataset for nuanced human-model interactions.
  • Safety and Balance: Features the first Polish-language preference corpus, teaching the model correctness, balance, and safety, especially for sensitive topics.
  • Domain-Specific Excellence: Achieves top scores on custom benchmarks for Polish public administration tasks.
  • Context Length: Supports a substantial context window of 32768 tokens.

Good For

  • General Polish Language Tasks: Text generation, summarization, and question answering in Polish.
  • Domain-Specific Applications: Developing intelligent assistants for Polish public administration, legal, or bureaucratic contexts, especially when paired with RAG.
  • Research and Development: Serving as a foundational model for AI applications requiring strong Polish language command.

Finetunes
1 models