nshportun/usa-immigration-llama-3.2-3b
The nshportun/usa-immigration-llama-3.2-3b model is a 3.2 billion parameter Llama 3.2 Instruct variant, fine-tuned by nshportun on 17,058 source-grounded Q&A pairs covering U.S. immigration law. This model specializes in providing answers to immigration-related queries, outperforming a zero-shot Llama 3 8B baseline on fully correct answers despite its smaller size. It is optimized for research and educational applications in U.S. immigration law, particularly for RAG-based legal assistants and domain-specific benchmarking.
Loading preview...
Overview
This model, nshportun/usa-immigration-llama-3.2-3b, is a 3.2 billion parameter Llama 3.2 Instruct model fine-tuned by nshportun. It was trained on the nshportun/usa-immigration-law-qa dataset, comprising 17,058 question-answer pairs derived from official U.S. immigration sources like USCIS Policy Manual, 8 CFR, and BIA Precedent Decisions. The fine-tuning process used LoRA (r=8, alpha=32) over a single epoch, achieving an evaluation perplexity of 2.47.
Key Capabilities & Performance
- Specialized Domain Knowledge: Excels in U.S. immigration law, covering subdomains such as family-based immigration, naturalization, asylum, and nonimmigrant visas.
- Improved Accuracy: Achieves 7.9% fully correct answers on a benchmark of 101 questions, outperforming a zero-shot Llama 3 8B baseline (2.0% fully correct) despite being 2.7x smaller.
- Benchmark Insights: While its mean score (0.68/3.0) is below larger models like Claude Sonnet 4.6 (1.47/3.0), its relative improvement over a larger Llama 3 base model highlights the effectiveness of domain-specific fine-tuning for exact correctness.
- Strongest Subdomains: Shows best performance in procedural topics like travel documents (1.83 mean score) and naturalization (1.13 mean score).
Intended Use Cases
- RAG-based Immigration Legal Assistants: Ideal for building systems that retrieve and generate answers based on specific immigration law documents.
- Domain-Specific LLM Benchmarking: Useful for evaluating the performance of other models on highly specialized legal Q&A tasks.
- Immigration Law Q&A Research: Supports academic and research efforts in understanding and applying U.S. immigration regulations.
Disclaimer: This model is for research and educational purposes only and does not constitute legal advice. Users should consult a licensed immigration attorney for legal guidance.