The neginr/multisubject_law_mc model is a 7.6 billion parameter language model fine-tuned from Qwen/Qwen2.5-7B-Instruct. This model is specifically optimized for legal domain tasks, leveraging a specialized dataset to enhance its performance in multi-subject legal contexts. It is designed to provide improved understanding and generation capabilities for legal inquiries and analyses.
Loading preview...
Overview
This model, neginr/multisubject_law_mc, is a fine-tuned version of the 7.6 billion parameter Qwen/Qwen2.5-7B-Instruct base model. It has been specialized using the neginr/multisubject_law_mc dataset, indicating an optimization for tasks within the legal domain, particularly those involving multiple legal subjects or categories.
Key Capabilities
- Legal Domain Specialization: Enhanced understanding and generation of legal text due to fine-tuning on a specific legal dataset.
- Instruction Following: Inherits instruction-following capabilities from its Qwen2.5-7B-Instruct base, adapted for legal queries.
- Multi-subject Legal Contexts: Designed to handle legal information spanning various subjects, suggesting applicability to complex legal research or analysis.
Training Details
The model was trained with a learning rate of 2e-05, a total batch size of 96 (achieved with train_batch_size 1 and gradient_accumulation_steps 3 across 32 GPUs), and for 7 epochs. It utilized the AdamW optimizer with cosine learning rate scheduling and a warmup ratio of 0.1. This training configuration aims to adapt the base model effectively to the nuances of legal language and concepts.