sambal/k2-v2-think-dpo-checkpoint-200
The sambal/k2-v2-think-dpo-checkpoint-200 is a 32.8 billion parameter language model based on the Qwen2ForCausalLM architecture, derived from LLM360/K2-Think. This intermediate checkpoint, trained with sigmoid DPO and RPO objectives, is optimized for preference alignment. It features a 32768 token context length and is suitable for further fine-tuning or evaluation on downstream tasks requiring preference-trained responses.
Loading preview...
Model Overview
The sambal/k2-v2-think-dpo-checkpoint-200 is an intermediate 32.8 billion parameter checkpoint from a DPO (Direct Preference Optimization) training run, built upon the LLM360/K2-Think base model. It utilizes the Qwen2ForCausalLM architecture and has been fine-tuned using a combination of sigmoid DPO (beta=0.1) and RPO (alpha=0.1) objectives. This specific release represents the 200th iteration of a 496-iteration training process.
Key Characteristics
- Base Model: Derived from
LLM360/K2-Think. - Architecture: Qwen2ForCausalLM, a 32B class model.
- Training Objective: Sigmoid DPO with RPO for preference alignment.
- Context Length: Supports a context of 32768 tokens.
- Chat Template: Includes a K2 v2 high chat template (
chat_template.jinja). - Format: Merged Transformers checkpoint, consisting of 14 BF16 safetensors shards.
Usage Considerations
This model is an intermediate preference-training checkpoint. It is intended for developers to evaluate on their specific downstream tasks and potentially for further fine-tuning before deployment. Users should load it using AutoModelForCausalLM and AutoTokenizer from the transformers library, specifying torch_dtype="auto" and device_map="auto" for efficient loading.