neph1/bellman-7b-mistral-instruct-v0.2

Hugging Face
TEXT GENERATIONConcurrent Unit Cost:1Model Size:7BQuant:FP8Context Size:4kTool Calling:SupportedPublished:Dec 16, 2023License:apache-2.0Architecture:Transformer0.0K Open Weights Featherless Exclusive Warm

neph1/bellman-7b-mistral-instruct-v0.2 is a 7 billion parameter Mistral-based instruction-tuned language model developed by neph1. It has been Qlora trained for approximately two epochs on 14,000 rows of Q&A, Python examples, and general instruction-type questions, with a 4096 token context length. This model is specifically optimized to improve knowledge in Swedish topics and enhance the quality of the Swedish language, making it particularly suitable for Swedish-centric applications.

Loading preview...

Model Overview

neph1/bellman-7b-mistral-instruct-v0.2 is a 7 billion parameter language model built upon the Mistral architecture, developed by neph1. This version has undergone Qlora training for about two epochs on a diverse dataset of 14,000 rows, including question-and-answer pairs, Python code examples, and general instruction-following tasks. It features a 4096 token context length and has received a light DPO (Direct Preference Optimization) pass to enhance alignment and reduce factual inaccuracies.

Key Capabilities

  • Swedish Language Proficiency: Specifically trained to improve knowledge and quality in Swedish topics and language.
  • Instruction Following: Fine-tuned on a broad range of instruction-type questions for general utility.
  • Code Understanding: Includes Python examples in its training data, suggesting some capability in code-related tasks.
  • Alignment: Utilizes a DPO pass to make the model less prone to generating untrue statements.

Training Details

  • Base Model: Mistral-7B-Instruct-v0.2.
  • Training Method: Qlora for approximately 2 epochs.
  • Dataset Size: 14,002 rows, with increased diversity in the instruct dataset.
  • Context Length: 4096 tokens.
  • Configuration: Rank 64, Alpha 128, Learning rate 3e-5.

Good For

  • Applications requiring strong performance in Swedish language generation and understanding.
  • Instruction-following tasks where factual accuracy and reduced hallucination are important.
  • Use cases benefiting from a 7B parameter model with a 4K context window for efficiency and performance.

Finetunes
1 models