Model Overview

BlouseJury/Mistral-7B-Discord-0.2 is a 7-billion parameter language model developed by BlouseJury. It is a fine-tuned version of the original Mistral-7B-v0.1 model, specifically adapted for conversational contexts. The model underwent training for 4 epochs on a dataset comprising approximately 40 million tokens of anonymized, largely unformatted Discord messages.

Key Characteristics

• Base Model: Fine-tuned from Mistral-7B-v0.1.
• Training Data: Specialized on a large corpus of Discord messages, aiming to capture the nuances of informal chat.
• Parameter Count: 7 billion parameters, offering a balance between performance and computational efficiency.
• Context Length: Supports an 8192-token context window.

Performance Benchmarks

Evaluated on the Open LLM Leaderboard, BlouseJury/Mistral-7B-Discord-0.2 demonstrates competitive performance for its size, with an overall average score of 59.55. Specific metric scores include:

• HellaSwag (10-Shot): 82.49
• MMLU (5-Shot): 62.82
• AI2 Reasoning Challenge (25-Shot): 60.58
• Winogrande (5-Shot): 77.74

Ideal Use Cases

This model is particularly well-suited for applications that require generating or understanding text in an informal, chat-like style, such as:

• Discord Bot Development: Creating bots that can interact naturally within Discord channels.
• Conversational AI: Building chatbots for informal communication platforms.
• Social Media Content Generation: Producing text that mimics casual online discourse.
• Text Analysis: Understanding sentiment or context in user-generated content from chat platforms.