gepardzik/Bielik-11B-v3.0-Instruct-heretic-MPOA

TEXT GENERATIONPricing:Input $0.431 / Cached $0.0862 / Output $1.12Concurrent Unit Cost:1Model Size:15BQuant:FP8Context Size:8kTool Calling:SupportedPublished:Feb 19, 2026License:apache-2.0Architecture:Transformer0.0K Open Weights Featherless Exclusive Cold

The gepardzik/Bielik-11B-v3.0-Instruct-heretic-MPOA is a 15 billion parameter instruction-tuned causal language model, derived from the SpeakLeash/Bielik-11B-v3.0-Instruct. This version has been decensored using Heretic v1.2.0 with Magnitude-Preserving Orthogonal Ablation, significantly reducing refusals compared to the original. Developed by SpeakLeash and ACK Cyfronet AGH, it excels in understanding and processing Polish and 31 other European languages, making it suitable for multilingual applications requiring less restrictive content generation.

Loading preview...

Model Overview

The gepardzik/Bielik-11B-v3.0-Instruct-heretic-MPOA is a 15 billion parameter instruction-tuned language model, building upon the SpeakLeash/Bielik-11B-v3.0-Instruct. This specific variant has undergone a decensoring process using the Heretic v1.2.0 tool with Magnitude-Preserving Orthogonal Ablation (MPOA), resulting in a substantial reduction in refusal rates (5/100 compared to 36/100 for the original model) while maintaining a low KL divergence of 0.0356.

Key Capabilities

  • Multilingual Proficiency: Optimized for Polish, with strong capabilities across 31 other European languages, leveraging extensive training on multilingual text corpora.
  • Decensored Output: Designed to provide less restrictive content generation due to the Heretic modification, making it suitable for use cases where the original model's refusal rate might be prohibitive.
  • Advanced Alignment: The base model was aligned using DPO-Positive and Reinforcement Learning (GRPO, Dr. GRPO) with the VERL framework, focusing on logic, STEM, mathematics, and tool-use domains.
  • Instruction Following: Fine-tuned on over 20 million instructions, including manually verified and synthetic Polish instructions, to enhance its ability to follow user prompts.

Use Cases

This model is particularly well-suited for applications requiring:

  • Multilingual Chatbots: Especially those targeting Polish and other European languages.
  • Creative Content Generation: Where less restrictive output and a lower refusal rate are desired.
  • Research and Development: For exploring the impact of decensoring techniques on large language models.

Limitations

It's important to note that, as a decensored model, it lacks moderation mechanisms and can produce factually incorrect, biased, or offensive outputs. Users should implement their own guardrails for deployment in environments requiring moderated content.

Finetunes
2 models