brucoder/winter-frost-2
brucoder/winter-frost-2 is a 7.6 billion parameter instruction-tuned causal language model developed by INEZA AIME BRUNO (brucoder), fine-tuned from Qwen/Qwen2.5-7B-Instruct. This model is a merged checkpoint, meaning the LoRA adapter has been integrated into the base model's weights for direct loading. It was fine-tuned using QLoRA on a dataset including general instructions and Python coding examples, primarily aiming to reliably shift small, well-defined behaviors.
Loading preview...
Overview
brucoder/winter-frost-2 is a 7.6 billion parameter language model, fine-tuned by INEZA AIME BRUNO (brucoder) from the Qwen/Qwen2.5-7B-Instruct base model. This version is a standalone, merged checkpoint, allowing it to be loaded and run without a separate adapter-loading step. The fine-tuning process utilized QLoRA (4-bit NF4 quantized base) on a single T4 GPU.
Key Characteristics
- Base Model: Qwen/Qwen2.5-7B-Instruct.
- Fine-tuning Method: QLoRA, with 4-bit NF4 quantization and fp16 compute.
- Training Data: Approximately 690 examples, comprising 300 general instructions from HuggingFaceH4/no_robots, 300 Python coding examples from iamtarun/python_code_instructions_18k_alpaca, and 90 custom identity examples.
- Training Duration: 3 epochs, 132 steps, completed in about 45 minutes.
- Merged Checkpoint: The LoRA adapter is already folded into the base model's weights.
Intended Use and Limitations
This model is designed to reliably shift small, well-defined behaviors, such as specific conversational responses, due to its focused fine-tuning on a relatively small dataset. It is not a new architecture and was not trained from scratch. Users should not expect significant changes in general reasoning, coding ability, or overall knowledge compared to the original Qwen/Qwen2.5-7B-Instruct base model. For those preferring to load the adapter separately, an adapter-only version is available at brucoder/winter-frost-2-adapter.