sonyalfauzan/legal-rag-qwen-grpo
The sonyalfauzan/legal-rag-qwen-grpo is a 0.5 billion parameter Qwen2 model developed by sonyalfauzan, specifically fine-tuned for legal Retrieval Augmented Generation (RAG) tasks. This model leverages Unsloth and Huggingface's TRL library for accelerated training, building upon the sonyalfauzan/legal-rag-qwen-sft base. With a 32768 token context length, it is optimized for processing and generating responses based on extensive legal documents.
Loading preview...
Model Overview
The sonyalfauzan/legal-rag-qwen-grpo is a 0.5 billion parameter Qwen2 model developed by sonyalfauzan, specifically fine-tuned for legal Retrieval Augmented Generation (RAG) applications. It is an iteration built upon the sonyalfauzan/legal-rag-qwen-sft model.
Key Characteristics
- Architecture: Based on the Qwen2 model family.
- Parameter Count: Features 0.5 billion parameters, offering a balance between performance and computational efficiency.
- Context Length: Supports a substantial context window of 32768 tokens, crucial for handling lengthy legal texts.
- Training Optimization: The model was trained with significant speed improvements, utilizing Unsloth and Huggingface's TRL library, resulting in 2x faster fine-tuning.
Primary Use Case
This model is specifically designed and fine-tuned for legal RAG tasks, making it suitable for applications requiring accurate information retrieval and generation within legal contexts. Its extended context length and specialized training aim to enhance its ability to understand and respond to complex legal queries.