ricdomolm/lawma-70b

Lawma 70B: Specialized Legal Classification Model

Lawma 70B is a 70 billion parameter model fine-tuned from Llama 3 70B Instruct, specifically designed for legal classification tasks. It was trained on over 500,000 examples (2 billion tokens) derived from 260 legal classification tasks from the Supreme Court and Songer Court of Appeals databases.

Key Capabilities & Performance

• Superior Legal Classification: Lawma 70B achieves an average accuracy of 81.9% across all 260 legal classification tasks, significantly outperforming GPT-4 (62.9%) and Llama 3 70B Instruct (58.4%).
• Outperforms GPT-4: It surpasses GPT-4 on 95% of these tasks, with an average improvement of over 17 accuracy points.
• Specialized Output: The model is fine-tuned for multiple-choice questions and outputs only letters (e.g., A, B, C) or numbers, not general instructions.

When to Use Lawma 70B

• Legal Classification: Ideal for specific legal classification problems, particularly those related to Supreme Court and Court of Appeals case variables.
• Further Fine-tuning: Recommended as a base for practitioners to further fine-tune on their exact legal tasks, as even a small number of examples can yield significant performance gains.
• Research in Legal AI: Useful for academic and research applications exploring the power of specialization in legal AI, as detailed in its arXiv preprint.

While Lawma 70B offers superior performance for its specialized domain, Lawma 8B might be preferred for its lower inference and fine-tuning costs, with only a small performance trade-off on average.