WisdomShell/ADG-CoT-LLaMa3-8B
WisdomShell/ADG-CoT-LLaMa3-8B is an 8 billion parameter LLaMa3-based model developed by WisdomShell, fine-tuned using the Answer Divergence-Guided Selection (ADG) method. This model is specifically optimized for instruction tuning by selecting high-quality training examples based on the geometric structure of multiple sampled answers. It excels in reasoning, knowledge, and coding tasks by improving instruction data selection under a fixed budget, offering enhanced performance over traditional single-reference response methods.
Loading preview...
Overview of WisdomShell/ADG-CoT-LLaMa3-8B
WisdomShell/ADG-CoT-LLaMa3-8B is an 8 billion parameter LLaMa3-based model that leverages the Answer Divergence-Guided Selection (ADG) method for instruction data selection. Developed by WisdomShell, this approach significantly enhances instruction tuning quality by evaluating the geometric structure of multiple sampled answers for each instruction, rather than relying on a single reference response. This method was presented at ACL 2026.
Key Capabilities & Features
- Geometry-Aware Data Selection: ADG scores instructions based on the dispersion magnitude and shape anisotropy of multiple high-temperature sampled answers, providing a more robust quality assessment.
- Improved Instruction Tuning: Consistently improves model performance across reasoning, knowledge, and coding benchmarks under a fixed 10K data budget.
- Semantic Coverage: Employs bin-wise selection to ensure semantic diversity and coverage within the selected instruction subset.
- Practical Pipeline: The repository provides a complete pipeline for multi-sample answer generation, instruction embedding and clustering, ADG scoring, subset selection, model training, and benchmark evaluation.
Ideal Use Cases
- Optimizing Instruction Datasets: Developers looking to select the most impactful instruction examples from large pools for fine-tuning.
- Enhancing Reasoning Tasks: Applications requiring strong performance in complex reasoning, mathematical, and logical problem-solving.
- Improving Code Generation: Scenarios where high-quality code generation and understanding are critical.
- Knowledge-Intensive Applications: Use cases demanding accurate and comprehensive knowledge recall and application.