Model Overview
The kth8/Llama-3.2-1B-Instruct-SuperGPQA-Classifier is a specialized 1.23 billion parameter language model, fine-tuned from the unsloth/Llama-3.2-1B-Instruct base model. Its primary function is to act as a classifier, categorizing given problems into predefined disciplines, fields, and subfields.
Key Capabilities
- Structured Classification: Designed to classify problems into a hierarchical structure of discipline, field, and subfield.
- JSON Output: Provides classification results in a consistent JSON format, making it easy for programmatic integration.
- Optimized for Accuracy: Fine-tuned on the
m-a-p/SuperGPQA dataset, which focuses on general knowledge question answering, enhancing its ability to accurately categorize diverse problems. - Low Temperature Recommendation: For optimal classification results, a temperature setting of 0.0 is recommended, indicating a preference for deterministic and precise outputs.
Training Details
The model was trained using PEFT (Parameter-Efficient Fine-Tuning) with a rank of 32 and LoRA alpha of 64, targeting q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, and down_proj modules. It underwent 2 epochs of SFT (Supervised Fine-Tuning) with a batch size of 32 and a learning rate of 0.0004, achieving a final validation loss of 0.05316. The training utilized an NVIDIA RTX PRO 6000 Blackwell Server Edition GPU.
Good For
- Automated content categorization.
- Structuring knowledge bases.
- Routing queries based on subject matter.
- Educational applications requiring precise topic identification.