LSW142857/OPSD-PI-Qwen3.5-9B-Medium-Trailing-1024-A6000-Merged-Update04
LSW142857/OPSD-PI-Qwen3.5-9B-Medium-Trailing-1024-A6000-Merged-Update04 is a 9 billion parameter Qwen3.5-based language model, fine-tuned with 1024 rows of data using the OPSD-PI method and incorporating four optimizer updates. This model is fully merged and ready for direct loading, featuring a 32768-token context length. It is designed for general language tasks, benefiting from a specialized training process that integrates expert-SFT initialization with OPSD main-model and MTP LoRA updates.
Loading preview...
Model Overview
LSW142857/OPSD-PI-Qwen3.5-9B-Medium-Trailing-1024-A6000-Merged-Update04 is a 9 billion parameter Qwen3.5-based language model, representing the fourth completed optimizer update (zero-indexed training iteration 3) from an OPSD run on 8×RTX A6000 GPUs. This public repository provides the fully merged model, requiring no additional adapter or merge steps.
Key Characteristics
- Architecture: Based on Qwen3.5, fine-tuned with a specialized OPSD-PI method.
- Parameter Count: 9 billion parameters.
- Context Length: Supports a 32768-token context window.
- Training Process: Incorporates expert-SFT initialization, OPSD main-model LoRA updates, and MTP LoRA updates, with every directly trained full-MTP tensor included. The merge process restores full trained MTP tensors first, then applies main-model and MTP LoRA deltas with a scaling of 2.0.
- Directly Loadable: The model is provided as a fully merged Hugging Face model, ready for immediate use without further merging.
Usage Considerations
- The PI (teacher) was active only during training. For evaluation, the student model should be assessed without adding PI.
- It is recommended to use held-out tasks for evaluation rather than the 1024 training rows to ensure unbiased performance assessment.
- Integrity checks can be performed using
sha256sum -c SHA256SUMSafter downloading, as all 775 output tensors were verified before upload. Detailed configuration and finite metrics are available inmerge_manifest.jsonandtraining_config.json.