Model Overview

The cgato/TheSpice-7b-FT-ExperimentalOrca is a 7 billion parameter language model developed by cgato. This model was specifically fine-tuned using a cleaned version of the Orca dataset alongside a proprietary roleplay (RP) data package. The primary objective behind its creation was to serve as an experimental testbed to assess whether an overwhelming amount of instruction tokens would negatively affect the model's roleplay evaluation performance.

Key Characteristics

• Parameter Count: 7 billion parameters.
• Training Data: Utilizes a cleaned Orca dataset combined with custom roleplay data.
• Context Length: Supports a context window of 4096 tokens.
• Experimental Focus: Designed to explore the interplay between instruction tuning and roleplay capabilities.

Intended Use

This model is particularly suited for researchers and developers interested in:

• Evaluating the robustness of roleplay performance in models exposed to significant instruction-tuning data.
• Experimenting with different prompting strategies for instruction-tuned models in conversational or roleplay contexts.
• Understanding the trade-offs between instruction following and creative generation in fine-tuned LLMs.