yanggangu/Llama-3.1-8B-Instruct-SMAT-ScienceQA
The yanggangu/Llama-3.1-8B-Instruct-SMAT-ScienceQA is an 8 billion parameter instruction-tuned language model, based on the Llama 3.1 architecture with an 8192-token context length. It is a SMAT expert model, specifically trained to retain task performance while improving model merging capabilities. This model is particularly optimized for ScienceQA tasks, leveraging the SMAT training methodology for efficient integration.
Loading preview...
Overview
This model, yanggangu/Llama-3.1-8B-Instruct-SMAT-ScienceQA, is an 8 billion parameter instruction-tuned language model built upon the Llama 3.1 architecture. It is identified as a SMAT expert from Table 1 of the research paper "SMAT: Simple and Efficient Merge-Aware Training" (training seed 42).
Key Capabilities
- SMAT Training: Utilizes the SMAT methodology, which focuses on training experts that maintain strong task performance while simultaneously enhancing the efficiency and effectiveness of model merging processes.
- ScienceQA Optimization: Specifically fine-tuned and optimized for performance on ScienceQA tasks, indicating its strength in scientific question answering and reasoning.
- Llama 3.1 Base: Benefits from the robust capabilities and architecture of the Llama 3.1 foundation model.
Good For
- Research in Model Merging: Ideal for researchers exploring techniques for combining multiple models or experts, particularly those interested in the SMAT methodology.
- ScienceQA Benchmarking: Suitable for applications requiring high performance on scientific question answering datasets like ScienceQA.
- Efficient Model Integration: Useful for scenarios where retaining individual task performance while preparing models for merging is a priority.
For more technical details and usage instructions, refer to the original paper and the GitHub repository.