dzakk/qwen2.5-0.5b-pgabl-legal-sft-dzakwan
The dzakk/qwen2.5-0.5b-pgabl-legal-sft-dzakwan model is a 0.5 billion parameter Qwen2.5-based language model, fine-tuned by Dzakwan Fadhlullah using QLoRA SFT on an Indonesian Alpaca-GPT4 dataset. It is specifically optimized for text generation in Indonesian, particularly for legal assistant applications. This model is designed to work in conjunction with a RAG pipeline for factual legal knowledge, rather than relying solely on its internal weights. It features a context length of 32768 tokens.
Loading preview...
dzakk/qwen2.5-0.5b-pgabl-legal-sft-dzakwan: Indonesian Legal Assistant
This model is a 0.5 billion parameter variant of the Qwen2.5 architecture, developed by Dzakwan Fadhlullah. It has been fine-tuned using QLoRA SFT on the Ichsan2895/alpaca-gpt4-indonesian dataset, making it specialized for Indonesian language text generation.
Key Capabilities
- Indonesian Text Generation: Optimized for generating text in the Indonesian language.
- Legal Assistant Focus: Specifically trained to assist with legal-related queries and content.
- RAG Pipeline Integration: Designed to be used with a Retrieval-Augmented Generation (RAG) pipeline to provide factual legal knowledge, ensuring accuracy beyond its inherent model weights.
- Efficient Fine-tuning: Utilizes QLoRA for efficient fine-tuning, with two experiments run for 800 steps each.
- Extended Context: Supports a substantial context length of 32768 tokens.
Good For
- Indonesian Legal Tech: Applications requiring text generation for legal contexts in Indonesian.
- RAG-based Legal Systems: Serving as the generative component within a RAG system for legal information retrieval and synthesis.
- Resource-Efficient Deployment: Its 0.5B parameter size makes it suitable for scenarios where computational resources are a consideration, while still offering specialized capabilities.