auryn-macmillan/boostedv1
BoostedV1 by auryn-macmillan is a 1.5 billion parameter causal language model, continued LoRA training of DeepSeek-R1-Distill-Qwen-1.5B. It features a 32768 token context length and is specifically optimized for improved reasoning and code generation tasks. This model is a merged output, requiring no LoRA at inference, and is suitable for applications demanding enhanced logical processing and programming capabilities.
Loading preview...
Model Overview
auryn-macmillan/boostedv1 is a 1.5 billion parameter causal language model, built upon the DeepSeek-R1-Distill-Qwen-1.5B base architecture. It has undergone continued LoRA training to enhance its capabilities, particularly in reasoning and code generation. The model is provided as a merged output, meaning it can be used directly for inference without needing separate LoRA adapters.
Key Characteristics
- Base Architecture: DeepSeek-R1-Distill-Qwen-1.5B (Qwen2ForCausalLM).
- Training: Utilized a multi-stage LoRA training process, including 400 MLX LoRA steps on Apple Silicon and 550 Phase-1 continuation steps on dual RTX 3090 GPUs. Training data included OpenCodeInstruct, OpenThoughts, and OpenR1-Math.
- Context Length: Supports a context length of 32768 tokens.
- Inference: The model is a merged output, simplifying deployment as no LoRA is required during inference.
Performance Benchmarks
- GSM8K: Achieved a score of 46.0%.
- HumanEval (pass@1): Scored 7.3%.
Ideal Use Cases
This model is well-suited for applications requiring:
- Improved Reasoning: Tasks that benefit from enhanced logical processing.
- Code Generation: Scenarios where generating programming code is a primary requirement.