Overview

Hermes-Instruct-7B-217K is an instruction-tuned language model built upon the Mistral-7B-Instruct-v0.2 architecture. Developed by lodrick-the-lafted, this model leverages Mistral's robust base, which inherently supports a 32K token context window and a 1M rope theta.

Key Characteristics

• Base Model: Mistral-7B-Instruct-v0.2, a powerful 7 billion parameter model.
• Fine-tuning Data: Trained on 217,000 rows from the teknium/openhermes dataset, formatted in the Alpaca instruction style.
• Prompt Format Flexibility: Supports both the default Mistral-Instruct prompt format (<s>[INST] {sys_prompt} {instruction} [/INST]) and the Alpaca format ({sys_prompt}\n\n### Instruction:\n{instruction}\n\n### Response:\n). The tokenizer's default is set to Alpaca.

Use Cases

This model is particularly well-suited for applications requiring:

• Instruction Following: Generating responses based on explicit instructions.
• Chatbot Development: Engaging in conversational AI where clear instruction adherence is crucial.
• Text Generation: Creating diverse text outputs, from recipes to creative writing, guided by user prompts.

By further fine-tuning the Mistral-7B-Instruct-v0.2 with a substantial instruction dataset, Hermes-Instruct-7B-217K aims to enhance its ability to understand and execute user commands effectively.