cjnielson44/grizzly-qwen2.5-7b-opus-distilled

Grizzly Qwen2.5 7B Opus Distilled Overview

cjnielson44/grizzly-qwen2.5-7b-opus-distilled is a 7.6 billion parameter language model, fine-tuned by cjnielson44. It is based on Qwen/Qwen2.5-7B-Instruct and has been specifically optimized for instruction-following and reasoning tasks. The model was trained using LoRA adapters on chat data distilled from Claude Opus/Fable, with the adapters subsequently fused into the base model weights.

Key Capabilities

• Instruction Following: Designed to accurately follow instructions provided in prompts.
• Reasoning-Oriented Text Generation: Excels at generating text that demonstrates careful reasoning.
• MLX Compatibility: Provided as a fused BF16 model, tokenizer, chat template, and config, ready for direct loading and use with mlx-lm on Apple Silicon.
• Prompt Masking: Training incorporated prompt masking, applying loss only to assistant responses for focused learning.

Training Details

The model was fine-tuned using the mlx-lm framework with a LoRA rank of 16 and scale of 32, targeting key attention and feed-forward modules. It utilized an AdamW optimizer with a learning rate of 1e-5 and a maximum sequence length of 4096 tokens. The final local validation loss reported was 1.725.

Good For

• Local Apple Silicon Experimentation: Ideal for developers looking to experiment with instruction-tuned LLMs on Apple Silicon hardware.
• Instruction-Following Applications: Suitable for tasks requiring the model to adhere closely to given instructions.
• Reasoning Tasks: Useful for generating responses that involve logical thought processes and careful explanation.