tu-ericngo/llama-3.1-8B-StructuredIE
tu-ericngo/llama-3.1-8B-StructuredIE is an 8 billion parameter Llama-3.1-8B-Instruct model fine-tuned by Tu 'Eric' Ngo for structured information extraction (IE). Specifically, it targets joint named entity recognition (NER) and relation extraction (RE) from political elite biographies, outputting information in a structured JSON format. The model excels at extracting details about political elites, their associations, events, timeframes, and family members, making it suitable for highly specialized biographical data analysis. It was fine-tuned using a combination of manually collected and GPT-4 generated synthetic data.
Loading preview...
Model Overview
This model, tu-ericngo/llama-3.1-8B-StructuredIE, is an 8 billion parameter Llama-3.1-8B-Instruct variant fine-tuned by Tu 'Eric' Ngo for highly specialized structured information extraction (IE). It focuses on joint named entity recognition (NER) and relation extraction (RE) from political elite biographies, generating structured JSON output.
Key Capabilities
- Specialized Information Extraction: Designed to identify and extract specific information about political elites, including educational and professional associations, events, timeframes, and family members.
- Structured JSON Output: Generates extracted information in a predefined JSON schema, facilitating downstream data processing and analysis.
- Targeted Fine-tuning: Fine-tuned using Unsloth's procedure with a dataset comprising both manually collected and GPT-4 generated synthetic data, structured in an Alpaca format.
Training Details
The model underwent a focused training regimen for 3 epochs (99 steps) with a total batch size of 8, utilizing bf16 non-mixed precision. Only 1.05% of the model's parameters were trained, amounting to 83,886,080 parameters. The training process took approximately 38.48 minutes, with a peak reserved memory of 10.107 GB.
Evaluation Metrics
Evaluation was conducted using F1, Precision, and Recall, calculated for each root-level field due to the complex, nested JSON schema. Key results include:
- JSON Valid: 70.270%
- Exact Match: 2.700%
- Average Jaccard Similarity: 0.535
- Average Cosine Similarity: 0.672
Intended Use
This model is specifically fine-tuned for a particular research project involving political elite biographies and follows a very specific JSON schema and prompt template. While it may perform similar structured IE tasks, its primary strength lies in its intended, highly specific application. Future iterations aim to expand its range of structured IE capabilities.