thu-ml/STAIR-Qwen2-7B-SFT
STAIR-Qwen2-7B-SFT is a 7.6 billion parameter language model developed by thu-ml, fine-tuned from Qwen2-7B-Instruct. It is specifically trained on the STAIR-SFT dataset, which features step-level Chain-of-Thought (CoT) reasoning formats. This model is designed to align reasoning processes and facilitate self-improvement, making it particularly effective for tasks requiring structured, step-by-step logical deduction and safety alignment.
Loading preview...
Overview
STAIR-Qwen2-7B-SFT is a 7.6 billion parameter model developed by thu-ml, fine-tuned from the Qwen2-7B-Instruct base model. Its core innovation lies in its training on the STAIR-SFT dataset, which comprises 20,000 prompts from UltraFeedback and PKU-SafeRLHF, all formatted with step-level Chain-of-Thought (CoT) answers. This specialized training aims to align the model's reasoning format and enhance its capacity for self-improvement within the STAIR framework.
Key Capabilities
- Structured Reasoning: Generates responses in a step-level CoT format, breaking down complex problems into logical steps.
- Safety Alignment: Incorporates safety considerations, as demonstrated by its refusal to engage with harmful prompts, providing a final, safe answer.
- Fine-tuned for Alignment: Specifically trained to align its reasoning processes, which can be beneficial for tasks requiring transparent and verifiable thought processes.
Good For
- Tasks requiring explainable AI: Ideal for applications where understanding the model's reasoning steps is crucial.
- Safety-critical applications: Its alignment with safety protocols makes it suitable for environments where harmful content generation must be strictly avoided.
- Research in self-improvement and reasoning: Provides a strong base for further experimentation in aligning LLMs with complex reasoning and ethical guidelines.