yingfanbot/gsm-cot-llama3b-nl
The yingfanbot/gsm-cot-llama3b-nl is a 3.2 billion parameter causal language model, fine-tuned from Meta's Llama-3.2-3B-Instruct. It specializes in generating natural-language chain-of-thought reasoning steps, specifically optimized for mathematical tasks like GSM8k-Aug. This model serves as the Stage-1 CoT initialization for the natural-language LOTUS model, focusing on explicit reasoning.
Loading preview...
Model Overview
The yingfanbot/gsm-cot-llama3b-nl is a 3.2 billion parameter causal language model, fine-tuned from meta-llama/Llama-3.2-3B-Instruct. Its primary distinction lies in its fine-tuning on the GSM8k-Aug dataset with natural-language reasoning steps, making it adept at generating explicit, step-by-step thought processes for problem-solving.
Key Capabilities
- Natural-Language Chain-of-Thought (CoT) Generation: Excels at producing human-readable reasoning steps, crucial for understanding how a model arrives at its conclusions.
- Mathematical Reasoning: Specifically optimized for tasks requiring mathematical problem-solving, as evidenced by its training on GSM8k-Aug.
- Initialization for LOTUS: This model functions as the foundational Stage-1 CoT initialization for the more advanced natural-language LOTUS model (
yingfanbot/gsm-lotus-llama3b-nl), indicating its role in a larger, sophisticated reasoning architecture.
Usage and Integration
This model is a standard causal language model and can be loaded directly using from_pretrained without requiring any special wrappers for generation. It is designed to generate the Chain-of-Thought directly, making it straightforward to integrate into applications requiring explicit reasoning.
Research Context
This model is a direct result of the research presented in the paper "Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers" by Fan, Svete, and Lee (arXiv:2606.31779), highlighting its scientific backing and purpose in advancing reasoning capabilities in LLMs.