lgaalves/llama-2-13b-chat-platypus
lgaalves/llama-2-13b-chat-platypus is a 13 billion parameter instruction fine-tuned language model developed by Luiz G A Alves, based on the LLaMA2 transformer architecture with a 4096 token context length. This model is specifically fine-tuned on STEM and logic-based datasets, demonstrating strong performance in areas like TruthfulQA. It is designed for general English language tasks, particularly those requiring logical reasoning and factual accuracy.
Loading preview...
Model Overview
lgaalves/llama-2-13b-chat-platypus is a 13 billion parameter instruction fine-tuned model developed by Luiz G A Alves. It is built upon the LLaMA2 transformer architecture and has a context length of 4096 tokens. The model was fine-tuned using LoRA on a single A100-40GB GPU, completing training in approximately 2 hours.
Key Capabilities & Performance
This model was instruction fine-tuned using the garage-bAInd/Open-Platypus dataset, which is rich in STEM and logic-based content. Benchmarking against garage-bAInd/Platypus2-13B and llama-2-13b-chat-hf shows competitive performance, notably achieving the highest score in TruthfulQA (46.23) among the compared models. Other benchmark scores include ARC (53.84), HellaSwag (80.67), and MMLU (54.44).
Training Details
The model was trained exclusively on English language data. The fine-tuning process leveraged LoRA (Low-Rank Adaptation) for efficient training. As with all large language models, it carries inherent risks of producing inaccurate, biased, or objectionable responses, and developers are advised to conduct thorough safety testing for specific applications.