NakJun/Llama-3.2-1B-Instruct-korQuAD-v1
NakJun/Llama-3.2-1B-Instruct-korQuAD-v1 is a 1 billion parameter Llama-3.2-Instruct based model developed by NakJun, fine-tuned specifically for Korean Question Answering tasks. Utilizing the KorQuAD v1.0 dataset and LoRA, this model excels at extracting precise answers from provided Korean contexts. With a 32768 token context length, it offers strong performance on Korean QA benchmarks, making it suitable for applications requiring accurate information retrieval in Korean.
Loading preview...
Model Overview
This model, NakJun/Llama-3.2-1B-Instruct-korQuAD-v1, is a 1 billion parameter language model built upon the Llama-3.2-1B-Instruct architecture. Developed by NakJun, its primary distinction lies in its specialized fine-tuning for Korean Question Answering (QA) tasks using the KorQuAD v1.0 dataset. The model leverages LoRA (Low-Rank Adaptation) for efficient training.
Key Capabilities
- Specialized Korean QA: Optimized for understanding and answering questions in Korean based on provided contexts.
- Llama-3.2-Instruct Base: Benefits from the foundational capabilities of the Llama-3.2-Instruct series.
- Efficient Fine-tuning: Utilizes LoRA with specific configurations (r=16, lora_alpha=16) targeting key attention and feed-forward modules.
- Performance on KorQuAD: Achieves an F1 Score of 59.03 and an Exact Match score of 36.07 on the KorQuAD v1.0 evaluation script, demonstrating its proficiency in Korean QA.
Use Cases
This model is particularly well-suited for applications requiring accurate and context-aware question answering in the Korean language. Potential use cases include:
- Information Retrieval Systems: Extracting specific answers from Korean documents or knowledge bases.
- Customer Support Chatbots: Providing precise answers to user queries in Korean.
- Educational Tools: Assisting with comprehension and fact-checking for Korean texts.
Training Details
The model was trained for 5 epochs with a batch size of 1 and a learning rate of 2e-4 using the AdamW optimizer. Its 32768 token context length allows for processing substantial input texts for question answering.