wordcab/llama-natural-instructions-13b
wordcab/llama-natural-instructions-13b is a 13 billion parameter LLaMA model, fine-tuned by wordcab using the LoRA technique on the Natural Instructions dataset. This model is designed for research purposes, focusing on instruction-following tasks. It offers a balance between performance and efficiency, achieving faster inference times through 8-bit quantization compared to the original LLaMA 13B model.
Loading preview...
Model Overview
wordcab/llama-natural-instructions-13b is a 13 billion parameter language model, fine-tuned from Meta's llama-13b using the Low-Rank Adaptation (LoRA) technique. The model was trained on the Natural Instructions dataset from AllenAI, enhancing its ability to follow diverse instructions.
Key Characteristics
- Architecture: LLaMA 13B base model, fine-tuned with LoRA adapters.
- Parameter Count: 13 billion parameters.
- Training: Utilizes LoRA for efficient fine-tuning on instruction-following tasks.
- Efficiency: Designed for faster inference, achieving approximately 1.2 seconds inference time in 8-bit precision, which is significantly quicker than the original LLaMA 13B.
- Research Focus: This model is intended strictly for research purposes, as indicated by its license.
Performance Considerations
While offering improved inference speed due to 8-bit quantization and LoRA adapters, the model exhibits some performance degradation compared to the original LLaMA 13B on benchmarks like BoolQ, PIQA, WinoGrande, and OpenBookQA. Specifically, it scored 70% on BoolQ, 63.93% on PIQA, 51.6% on WinoGrande, and 50.4% on OpenBookQA. Complex tasks such as WinoGrande and OpenBookQA show a more pronounced drop in performance.
Usage
Developers can load the model either with just the LoRA adapters or as a full merged model. It provides utility functions for generating prompts and extracting responses, following a specific ### Instruction:\n{instruction}\n\n### Input:\n{input}\n\n### Response:\n format.