andresnowak/Qwen3-0.6B-MNLP_mcqa_model_text
The andresnowak/Qwen3-0.6B-MNLP_mcqa_model_text is a 0.8 billion parameter Qwen3-based language model fine-tuned by andresnowak. It is specifically optimized for multiple-choice question answering (MCQA) tasks, trained using a Seq2Seq method with TRL. This model excels at providing direct answers to MCQA problems across various domains, including STEM and general knowledge. It features a context length of 32768 tokens, making it suitable for detailed question analysis.
Loading preview...
Model Overview
The andresnowak/Qwen3-0.6B-MNLP_mcqa_model_text is a specialized 0.8 billion parameter language model built upon the Qwen3-0.6B-Base architecture. Developed by andresnowak, this model has undergone supervised fine-tuning (SFT) using the TRL library, specifically targeting Multiple Choice Question Answering (MCQA) tasks. Its training methodology involves a Seq2Seq approach, where the model is trained to generate answers in a "Letter. Answer" format.
Key Capabilities
- Multiple Choice Question Answering (MCQA): The model is explicitly fine-tuned for MCQA, making it proficient in selecting correct answers from a given set of options.
- Broad Domain Coverage: Training datasets include a diverse range of MCQA benchmarks such as Math QA, ScienceQA, MMLU, AI2 ARC (Challenge and Easy), MedMCQA, OpenBookQA, and SciQ.
- Language Modelling: The training process included language modeling with loss calculated on both the prompt and completion, enhancing its generative capabilities within the MCQA context.
- Context Length: Supports a substantial context length of 32768 tokens, allowing for comprehensive question and option processing.
Performance Highlights
Evaluated across several MCQA benchmarks, the model demonstrates varying levels of accuracy:
- ARC Challenge: 60.6% Accuracy
- ARC Easy: 78.6% Accuracy
- MMLU: 48.6% Accuracy
- Overall Accuracy: 43.9%
Good for
- Automated MCQA Systems: Ideal for applications requiring automated answering of multiple-choice questions.
- Educational Tools: Can be integrated into platforms for quizzes, assessments, or study aids.
- Research in MCQA: Provides a fine-tuned base for further research and development in multiple-choice question answering models, particularly for its size and specialized training.