Hugging Face Bielik-7B-Instruct-v0.1: Polish Language Model
Bielik-7B-Instruct-v0.1 is a 7 billion parameter instruction-tuned model developed by SpeakLeash and ACK Cyfronet AGH. It is fine-tuned from the Bielik-7B-v0.1 base model, with a strong focus on the Polish language. The model was trained using a combination of manually verified Polish instructions and publicly available English instruction datasets like OpenHermes-2.5 and orca-math-word-problems-200k, with specific improvements to counteract potential quality variations.
Key Capabilities and Features
- Polish Language Proficiency: Exhibits exceptional ability to understand and process the Polish language, providing accurate responses for various linguistic tasks.
- Optimized Training: Utilizes techniques such as weighted token-level loss, adaptive learning rates, and masked user instructions to enhance performance.
- Efficient Training Framework: Trained using the original open-source ALLaMo framework, designed for fast and efficient language model training.
- RAG Reader Performance: Achieves a high score of 86.00 in the RAG Reader task on the Open PL LLM Leaderboard, outperforming many other 7B models.
- Quantized Versions: Available in various quantized formats (GGUF, GPTQ, AWQ, EXL2, HQQ) and an MLX format for Apple Silicon, catering to diverse hardware resources.
Use Cases and Limitations
Bielik-7B-Instruct-v0.1 is suitable for applications requiring strong Polish language understanding and generation, especially in tasks like question answering and text processing. It is released under a CC BY NC 4.0 license, permitting non-commercial use. The model is a demonstration of fine-tuning potential and currently lacks moderation mechanisms. Users should be aware that it may produce factually incorrect, biased, or offensive outputs, and should not be relied upon for factual accuracy without verification.