promotion/Qwen3-8B-Panacea-baseline
Panacea on Qwen3-8B is an 8 billion parameter language model developed by Zhong et al. (NeurIPS 2024), built on the Qwen3-8B backbone with a 32768 token context length. It utilizes SVD-LoRA with a preference-agnostic singular value approach, optimized for instruction following, truthfulness, honesty, and helpfulness. This model is designed for general-capability applications requiring robust and aligned responses.
Loading preview...
Panacea on Qwen3-8B: Aligned Language Model
Panacea is an 8 billion parameter language model developed by Zhong et al. (NeurIPS 2024), leveraging the Qwen3-8B backbone. It incorporates a novel SVD-LoRA technique with k=8 preference-agnostic singular values, injecting the preference vector as the remaining ones, and uses LS aggregation. The model was exported with specific lambda values (0.15, 0.55, 0.15, 0.15) after validation on 100 prompts.
Key Capabilities & Training:
- Objectives: Optimized for instruction following, truthfulness, honesty, and helpfulness.
- Training: Trained with 300 optimizer updates and a global batch size of 16.
- Evaluation: Performance is assessed by independent objective-wise win rates against a common reference, judged by
Llama-3.3-70B-Instructon prompt-disjoint held-out prompts.
Good For:
- Applications requiring a general-purpose language model with strong alignment to instruction following and ethical principles.
- Use cases where truthfulness and helpfulness are critical, as highlighted by its training objectives.
- Scenarios benefiting from a model with a 32768 token context length for processing longer inputs.