Hjgugugjhuhjggg/mergekit-ties-tzamfyy
Hjgugugjhuhjggg/mergekit-ties-tzamfyy is a 1 billion parameter language model created by Hjgugugjhuhjggg using the TIES merge method, based on meta-llama/Llama-3.2-1B. This model integrates capabilities from multiple Llama-3.2-1B variants, including those fine-tuned for mathematical reasoning (MGSM8K, GSM8K), medical question answering (medmcqa), common sense QA, SQL, and code generation. It is designed to offer a broad range of specialized functionalities within a single compact model.
Loading preview...
Model Overview
Hjgugugjhuhjggg/mergekit-ties-tzamfyy is a merged language model built upon the meta-llama/Llama-3.2-1B base, utilizing the TIES merge method. This approach combines the strengths of multiple specialized Llama-3.2-1B variants into a single, cohesive model.
Key Capabilities
This model integrates diverse functionalities from its constituent models, including:
- Mathematical Reasoning: Incorporates models fine-tuned on datasets like MGSM8K and GSM8K, enhancing its ability to handle mathematical problems.
- Question Answering: Includes components optimized for medical question answering (medmcqa) and common sense QA.
- Code and SQL Generation: Merges models with capabilities in SQL and general code tasks.
- Multilingual Support: Features a component fine-tuned on Hinglish medical datasets, suggesting some multilingual adaptability.
Merge Details
The merge process involved a variety of Llama-3.2-1B instruction-tuned and pre-trained models, each contributing specific expertise. The TIES method, with a density of 0.5, was used to combine these models, aiming for a balanced integration of their respective strengths. This allows the model to perform well across a spectrum of tasks that its individual components were specialized for.