yingfanbot/gsm-cot-llama3b
The yingfanbot/gsm-cot-llama3b model is a supervised chain-of-thought fine-tune of Meta's Llama-3.2-3B-Instruct, specifically optimized for mathematical reasoning tasks. This 3 billion parameter model is fine-tuned on the GSM8K dataset to enhance its ability to generate step-by-step reasoning. It serves as a baseline checkpoint for the LOTUS (Looped Transformers) architecture, focusing on bridging latent and explicit reasoning.
Loading preview...
Overview
The yingfanbot/gsm-cot-llama3b model is a specialized fine-tune of the meta-llama/Llama-3.2-3B-Instruct architecture. This model has undergone a supervised chain-of-thought (CoT) fine-tuning process using the GSM8K dataset, which is designed for mathematical word problems. Its primary purpose is to improve the model's ability to generate explicit, step-by-step reasoning for complex tasks.
Key Capabilities
- Enhanced Mathematical Reasoning: Specifically fine-tuned on GSM8K, this model is optimized for solving mathematical problems by generating detailed chains of thought.
- Baseline for LOTUS: It functions as the Stage-1 CoT baseline and initialization checkpoint for the Looped Transformers (LOTUS) architecture, as detailed in the paper "Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers" (arXiv:2606.31779).
- Llama-3.2-3B-Instruct Foundation: Built upon a robust 3 billion parameter Llama-3.2-Instruct base, providing a strong general language understanding foundation.
Good For
- Research in Reasoning: Ideal for researchers exploring chain-of-thought reasoning, especially in mathematical domains.
- Developing Reasoning Systems: Suitable as a foundational model for building more advanced reasoning systems, particularly those leveraging the LOTUS architecture.
- Educational Applications: Can be used in applications requiring step-by-step solutions to arithmetic and word problems.