Pensez-v0.1-e5: Efficient Bilingual Reasoning
Pensez-v0.1-e5 is a 7.6 billion parameter bilingual (French-English) reasoning model developed by HoangHa, based on the Qwen 2.5 Instruct 7B architecture. It is specifically designed to achieve superior reasoning capabilities with a significantly reduced training dataset of only 2,000 curated samples (1,000 French, 1,000 English).
Key Capabilities & Differentiators
- Optimized Reasoning: Employs distinct strategies for concise reasoning in simple tasks and extended reasoning for complex domains like mathematics, coding, and science.
- Special Token Guidance: Utilizes
<think>...</think> tokens to explicitly guide the model's reasoning process, enhancing its problem-solving approach. - Bilingual Proficiency: Demonstrates strong performance in both French and English, with a focus on French-specific reasoning benchmarks.
- Efficiency: Achieves competitive reasoning performance compared to larger models like DeepSeek-R1-Distill-Qwen-7B, despite being trained on a much smaller dataset.
Benchmarks & Performance
Pensez-v0.1-e5 shows notable improvements in French reasoning tasks:
- Math-hard (fr): Scores 0.3458, outperforming Qwen2.5-7B-Instruct (0.2253).
- BoolQA (fr): Achieves 0.9157, surpassing DeepSeek-R1-Distill-Qwen-7B (0.7079).
Training Details
The model was trained over five epochs using advanced techniques such as Packing Inputs Without Cross-Contamination Attention, Liger Kernel, DeepSpeed 3, and NEFTune Noise for robustness, with a maximum sequence length of 16,384 tokens.