Model Overview
macadeliccc/magistrate-3.2-3b-it is a 3.2 billion parameter instruction-tuned language model based on the LlamaForCausalLM architecture, developed by macadeliccc. It is a fine-tuned version of macadeliccc/magistrate-3.2-3b-base, with a notable context length of 32768 tokens. The model's primary specialization is in legal assistance, particularly concerning US Supreme Court case law and US Federal regulations.
Key Capabilities & Training
This model has been trained using a diverse set of datasets, including standard open-source resources like OpenHermes-2.5 and hermes-function-calling, alongside a comprehensive, non-synthetic argument dataset. The training methodology incorporates a "Spectrum top 35% finetune" approach for both pretraining and instruction finetuning, drawing inspiration from Cohere's research on the impact of code in pre-training. The instruction finetuning is largely based on OpenHermes-2.5 and hermes-function-calling, with additional synthetic data for the instruct version.
Intended Use Cases
- Legal Research: Designed for specialized tasks related to US Supreme Court case law and US Federal regulations.
- Legal Specialty Development: Suitable for ongoing research and development in the legal AI domain.
Limitations
This model is intended for research purposes and continued development. Users are responsible for all outputs generated by the model.