lodrick-the-lafted/Hermes-Instruct-7B-217K
Hermes-Instruct-7B-217K is a 7 billion parameter instruction-tuned causal language model developed by lodrick-the-lafted, based on Mistral-7B-Instruct-v0.2. It was fine-tuned using 217K rows of the OpenHermes dataset in Alpaca format, leveraging Mistral's native 32K context and 1M rope theta. This model is optimized for following instructions and generating responses in both Mistral-Instruct and Alpaca prompt formats.
Loading preview...
Overview
Hermes-Instruct-7B-217K is an instruction-tuned language model built upon the Mistral-7B-Instruct-v0.2 architecture. Developed by lodrick-the-lafted, this model leverages Mistral's robust base, which inherently supports a 32K token context window and a 1M rope theta.
Key Characteristics
- Base Model: Mistral-7B-Instruct-v0.2, a powerful 7 billion parameter model.
- Fine-tuning Data: Trained on 217,000 rows from the
teknium/openhermesdataset, formatted in the Alpaca instruction style. - Prompt Format Flexibility: Supports both the default Mistral-Instruct prompt format (
<s>[INST] {sys_prompt} {instruction} [/INST]) and the Alpaca format ({sys_prompt}\n\n### Instruction:\n{instruction}\n\n### Response:\n). The tokenizer's default is set to Alpaca.
Use Cases
This model is particularly well-suited for applications requiring:
- Instruction Following: Generating responses based on explicit instructions.
- Chatbot Development: Engaging in conversational AI where clear instruction adherence is crucial.
- Text Generation: Creating diverse text outputs, from recipes to creative writing, guided by user prompts.
By further fine-tuning the Mistral-7B-Instruct-v0.2 with a substantial instruction dataset, Hermes-Instruct-7B-217K aims to enhance its ability to understand and execute user commands effectively.
Top 3 parameter combinations used by Featherless users for this model. Click a tab to see each config.