TheTsar1209/qwen-carpmuscle-v0.4
TheTsar1209/qwen-carpmuscle-v0.4 is a 14.8 billion parameter instruction-tuned causal language model developed by TheTsar1209, fine-tuned from unsloth/Qwen2.5-14B-Instruct-bnb-4bit. It was trained using SFT with Unsloth on the ChatML format, featuring an 8k context length. This model is optimized for general conversational tasks, with training data refined to remove "slop" and refusals, and datasets reformatted for improved performance.
Loading preview...
Overview
TheTsar1209/qwen-carpmuscle-v0.4 is a 14.8 billion parameter instruction-tuned model, developed by TheTsar1209. It is fine-tuned from the unsloth/Qwen2.5-14B-Instruct-bnb-4bit base model, utilizing SFT with Unsloth and Huggingface's TRL library. The training process involved using the ChatML format with an 8k context length, and included specific data cleaning steps to remove "slop" and refusals, alongside dataset reformatting.
Key Capabilities
- Instruction Following: Trained with Supervised Fine-Tuning (SFT) to follow instructions effectively.
- Optimized Training: Leverages Unsloth for faster training, indicating efficiency in development.
- Refined Data Quality: Training datasets were curated to enhance response quality by removing undesirable content and reformatting for consistency.
Performance Highlights
Evaluations on the Open LLM Leaderboard show an average score of 35.67. Notable scores include 72.02 on IFEval (0-Shot) and 49.38 on BBH (3-Shot). While not a top-tier performer across all benchmarks, its IFEval score suggests reasonable instruction-following capabilities.
Good for
- General conversational AI applications requiring a 14.8B parameter model.
- Developers looking for a model trained with Unsloth for potential efficiency benefits.
- Use cases where a refined instruction-following model with an 8k context is suitable.