ombd/Llama-3.1-8B-Aleph-2
ombd/Llama-3.1-8B-Aleph-2 is an 8 billion parameter reasoning model built from Meta's Llama 3.1 base, optimized for complex problem-solving. It utilizes a unique reinforcement learning approach called the Aleph tree, which learns from the divergence of correct and incorrect reasoning branches. This model excels in tasks like tool calls (99.5%) and extraction (99.0%), outperforming Meta's Llama 3.1 8B Instruct on these specific benchmarks, while also showing significant gains across various math and instruction-following tests.
Loading preview...
Llama-3.1-8B-Aleph-2 Overview
ombd/Llama-3.1-8B-Aleph-2 is an 8 billion parameter reasoning model derived from Meta's Llama 3.1 base. Developed by Omar Badran, this model distinguishes itself through its unique training methodology, the "Aleph tree," which employs reinforcement learning from verifiable rewards. Instead of collecting separate answers, it grows a tree of potential solutions, allowing the trainer to identify where reasoning paths diverge into correct or incorrect outcomes. This approach enables learning directly from programmatic verdicts on final answers, without relying on human labels or judge models.
Key Capabilities
- Enhanced Reasoning: Demonstrates significant improvements across various reasoning tasks compared to its predecessor and the warmup model.
- Superior Tool Calling: Achieves 99.5% on tool call tasks (choosing functions and arguments as JSON), surpassing Meta's Llama 3.1 8B Instruct (82.4%).
- High Extraction Accuracy: Scores 99.0% on information extraction (filling JSON records from text), also outperforming Meta's Instruct (90.0%).
- Instruction Following: Shows strong performance in following complex written instructions (50.8% on IFEval).
- Mathematical Problem Solving: Improved performance on grade-school math (76.0% on GSM8K) and harder mixed math problems (40.6%).
Good For
- Automated Reasoning Tasks: Ideal for applications requiring robust logical deduction and problem-solving, especially where verifiable outcomes are available.
- Function Calling & API Interaction: Highly effective for scenarios involving tool use and structured data extraction due to its high accuracy in these areas.
- Instruction-Following Agents: Suitable for building agents that need to precisely adhere to complex, multi-step instructions.
- Research into RL-based Reasoning: Offers a unique perspective on reinforcement learning for language models, particularly with its Aleph tree training method.