CYFRAGOVPL/PLLuM-12B-chat-2412

TEXT GENERATIONConcurrency Cost:1Model Size:12BQuant:FP8Ctx Length:32kTool Calling:SupportedPublished:Feb 7, 2025License:apache-2.0Architecture:Transformer0.0K Open Weights Cold

PLLuM-12B-chat-2412 is a 12 billion parameter chat-tuned large language model developed by CYFRAGOVPL, based on Mistral-Nemo-Base-2407, with a 32768 token context length. It is specialized in Polish and other Slavic/Baltic languages, incorporating English data for broader generalization. This model excels at generating contextually coherent text and assisting in various tasks, particularly for Polish public administration and general-purpose dialog scenarios. It was refined using extensive Polish data, including 150B tokens for non-commercial versions, and human-authored instruction datasets.

Loading preview...

PLLuM-12B-chat-2412: Polish-Optimized Chat Model

PLLuM-12B-chat-2412 is a 12 billion parameter chat-tuned large language model from the PLLuM family, developed by CYFRAGOVPL. It is built upon the Mistral-Nemo-Base-2407 architecture and features a 32768 token context length. This model is specifically designed for Polish and other Slavic/Baltic languages, augmented with English data to enhance generalization capabilities. Its development involved extensive data collection, including up to 150 billion tokens of Polish text for non-commercial variants, and a unique dataset of ~40k manually created "organic instructions" in Polish, including multi-turn dialogues.

Key Capabilities

  • Specialized Polish Language Processing: Optimized for generating contextually coherent text in Polish, with strong performance in Slavic and Baltic languages.
  • Instruction and Chat Tuning: Refined through instruction tuning and preference learning using a unique Polish preference corpus, ensuring balanced, safe, and contextually appropriate responses.
  • High-Quality Data Foundation: Trained on large-scale, high-quality text corpora, including a significant portion of Polish data.
  • Public Administration Expertise: Achieves top scores on custom benchmarks relevant to Polish public administration tasks.
  • Retrieval Augmented Generation (RAG) Support: Designed to perform well in RAG settings, particularly for question answering based on provided documents.

Good For

  • General Language Tasks: Text generation, summarization, and question answering in Polish.
  • Domain-Specific Assistants: Especially effective for applications in Polish public administration, legal, and bureaucratic contexts.
  • Research & Development: Serving as a foundational model for AI applications requiring strong Polish language command.
  • Dialog and Conversational AI: Its chat-tuned nature makes it suitable for general-purpose conversational scenarios.

Finetunes
1 models