kth8/Llama-3.2-3B-Instruct-SuperGPQA-Classifier
kth8/Llama-3.2-3B-Instruct-SuperGPQA-Classifier is a 3.2 billion parameter instruction-tuned Llama 3.2 model, fine-tuned by kth8 on the SuperGPQA dataset. This model is specifically designed and optimized for classifying problems into predefined disciplines, fields, and subfields. It excels at structured categorization tasks, providing JSON-formatted outputs for academic and technical problem classification.
Loading preview...
Model Overview
This model, kth8/Llama-3.2-3B-Instruct-SuperGPQA-Classifier, is a specialized fine-tune of the unsloth/Llama-3.2-3B-Instruct base model, featuring 3.2 billion parameters. Its primary function is to act as a classifier, categorizing problems into specific disciplines, fields, and subfields based on the SuperGPQA dataset.
Key Capabilities
- Structured Classification: Designed to output classifications in a precise JSON format, specifying
discipline,field, andsubfield. - Academic/Technical Categorization: Optimized for problems spanning a wide range of academic and technical domains, including Philosophy, Science, Engineering, Medicine, and more.
- Instruction-Following: Benefits from its instruction-tuned base, making it responsive to classification prompts.
Training Details
The model was fine-tuned using PEFT (LoRA) with a rank of 32 and an alpha of 64, targeting q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, and down_proj modules. Training involved 2 epochs with a batch size of 32, a learning rate of 0.0004, and an AdamW optimizer. It achieved an average training loss of 0.068 and a final validation loss of 0.050.
Usage Recommendation
For optimal classification results, it is recommended to set the inference temperature to 0.0.