beita6969/EvoSteer-9b
EvoSteer-9b is a 9 billion parameter language model developed by beita6969, built upon the Qwen3.5-9B architecture. It integrates a LoRA adapter (rank 64, alpha 128) merged into the base weights, specifically targeting projection layers. This model is designed to function as a trained orchestrator, with its merged layers stored in float32 to preserve the LoRA updates, making it suitable for applications requiring precise weight representation.
Loading preview...
EvoSteer-9B Overview
EvoSteer-9B is a 9 billion parameter model developed by beita6969, derived from the Qwen3.5-9B base model. Its primary distinction lies in the integration of a trained LoRA adapter (rank 64, alpha 128) directly merged into the Qwen3.5-9B weights. This adapter specifically modifies the q_proj, k_proj, v_proj, o_proj, out_proj, and in_proj_qkv layers.
Key Characteristics
- Orchestrator Functionality: The model is presented as a "trained EvoSteer orchestrator," suggesting a specialized role in managing or directing AI processes.
- Precision Preservation: The merged LoRA layers are stored in
float32to ensure the fidelity of the LoRA updates, which represent approximately 0.06% of the weight norm. This is crucial because rounding tobfloat16would discard most of these fine-grained adjustments. - Base Model Integration: While the LoRA-modified layers are
float32, all other layers retain the originalbfloat16precision of the Qwen3.5-9B base model.
Good for
- Applications requiring precise LoRA integration: Ideal for scenarios where the subtle modifications introduced by the LoRA adapter are critical to performance.
- Research into orchestrator models: Useful for developers and researchers exploring the capabilities of specialized orchestrator LLMs built via LoRA fine-tuning.
- Reproducing specific training outcomes: Users can load the model in
float32to exactly reproduce the trained orchestrator's behavior.