GOAT-7B-Community: A LLaMA 2-based Chatbot Research Model

GOAT-7B-Community is a 7 billion parameter language model developed by GOAT.AI, built upon the LLaMA 2 architecture. It has been supervised fine-tuned (SFT) using a substantial dataset of 72,000 multi-turn dialogues collected from user conversations within the GoatChat application and OpenAssistant.

Key Capabilities & Features

• Base Architecture: LLaMA 2 7B, providing a robust foundation.
• Training Data: Fine-tuned on a unique dataset of 72K multi-turn dialogues, enhancing conversational abilities.
• Context Window: Supports a context length of 4096 tokens, allowing for more extensive interactions.
• Research Focus: Primarily designed to support research and development in large language models and chatbot technologies.

Performance & Evaluation

The model's performance has been evaluated on standard benchmarks, with results including:

• MMLU (5-shot): 49.58
• BigBench Hard (BBH): 35.7
• Open LLM Leaderboard Average: 42.74

Use Cases

GOAT-7B-Community is intended for:

• Researchers: Exploring advancements in natural language processing, machine learning, and artificial intelligence.
• Hobbyists: Experimenting with and developing chatbot applications.

It's important to note that while the model is a valuable research tool, it may produce factually incorrect or biased outputs, consistent with the limitations of current LLMs.