SynapseLLM: A Finetuned Mistral 7B Model
SynapseLLM, developed by WebraftAI, is a 7 billion parameter, decoder-only transformer model finetuned from Mistral 7B v0.1. This model is designed to excel in code generation and general question-answering scenarios, leveraging a custom dataset of 1.54 million rows. The training data includes 361k Maths Instruct Q/A, 143k GPT-3.5 Q/A, 140k General Code, 63k Python code, and 900k General Q/A generated via GPT-4.
Key Capabilities & Features
- Specialized Finetuning: Optimized for both code and chat-based Q/A instructions.
- Mistral-based Architecture: Utilizes the efficient Mistral 7B v0.1 as its base.
- Apache 2.0 License: Allows for broad usage and distribution.
- Prompt Format: Adheres to the standard Mistral instruct prompt format.
Performance & Limitations
Evaluations on the Open LLM Leaderboard show an average score of 55.93. Specific scores include 75.54 on HellaSwag (10-Shot) and 73.09 on Winogrande (5-shot), while GSM8k (5-shot) scored 27.60. Users should be aware of potential biases, including the possibility of factually incorrect outputs, lack of system prompt adherence, and no inherent memory. Researchers can experiment with external memory integration.
Ideal Use Cases
This model is particularly well-suited for applications requiring:
- Code generation and completion.
- Instruction-following for general Q/A.
- Mathematical problem-solving in a Q/A format.