HiTZ/GoLLIE-34B
HiTZ/GoLLIE-34B is a 34 billion parameter Large Language Model developed by HiTZ Basque Center for Language Technology, fine-tuned from Code LLaMA2. It specializes in zero-shot Information Extraction by following annotation guidelines defined on the fly, outperforming previous approaches. With a 32768 token context length, GoLLIE-34B is designed for precise information extraction tasks where detailed schema definitions are provided.
Loading preview...
GoLLIE-34B: Guideline-Following LLM for Information Extraction
GoLLIE-34B is a 34 billion parameter Large Language Model developed by the HiTZ Basque Center for Language Technology, specifically designed for zero-shot Information Extraction (IE). Fine-tuned from Code LLaMA2, this model distinguishes itself by its ability to follow detailed annotation guidelines provided by the user, rather than solely relying on pre-encoded knowledge.
Key Capabilities & Features
- Guideline-Driven IE: GoLLIE allows users to define annotation schemas and instructions on the fly, using Python classes and docstrings, enabling highly flexible and precise information extraction.
- Zero-Shot Performance: It significantly outperforms previous approaches in zero-shot IE tasks, demonstrating robust performance even on unseen tasks.
- Context Length: Supports a substantial context window of 32768 tokens.
- Performance: Achieves a 57.2% zero-shot average F1 score and a 75.0% supervised average F1 score on evaluation benchmarks, making it the strongest model in the GoLLIE series.
Ideal Use Cases
- Custom Information Extraction: Extracting specific entities and relationships from text based on user-defined schemas.
- Rapid Prototyping: Quickly setting up IE tasks without extensive re-training or data labeling.
- Complex Annotation Tasks: Handling scenarios where detailed, nuanced guidelines are crucial for accurate extraction.
Users should note that GoLLIE requires trust_remote_code=True and a CUDA-enabled GPU for optimal performance, as it leverages Flash Attention 2.