Hugging Face Overview
Model Overview
aloobun/Reyna-CoT-4B-v0.1 is a 4 billion parameter language model developed by aloobun, based on the Qwen/Qwen1.5-4B architecture. This model is the fourth iteration in a series dedicated to improving Chain of Thought (CoT) capabilities in smaller language models.
Key Capabilities
- Enhanced Chain of Thought Reasoning: Specifically fine-tuned on a diverse collection of CoT tasks.
- Problem Solving: Demonstrates proficiency in various reasoning challenges.
- Question Answering: Capable of closed-book question answering.
- Ethical Reasoning: Includes training data related to ethical considerations.
Training Details
The model was fine-tuned using a curated set of datasets, including content from kaist-ai/CoT-Collection, euclaise/TinyCoT, and a small subset of teknium/OpenHermes-2.5. The training utilized an AdamW optimizer with specific hyperparameters (eps=1e-8, cosine decay with 20% warmup, lr=2e-5).
Good For
- Applications requiring explicit reasoning steps: Ideal for tasks where intermediate thought processes are beneficial.
- Resource-constrained environments: Its 4B parameter size makes it suitable for deployment where larger models are impractical.
- Research into CoT mechanisms: Provides a base for further experimentation with Chain of Thought techniques in compact models.