cgato/TheSpice-7b-FT-ExperimentalOrca
TheSpice-7b-FT-ExperimentalOrca is a 7 billion parameter language model developed by cgato, fine-tuned using a cleaned version of the Orca dataset combined with roleplay data. This model was created to investigate the impact of instruction tokens on roleplay evaluations. It features a 4096-token context length and is designed for experimental evaluation of instruction-tuned models in roleplay scenarios.
Loading preview...
Model Overview
The cgato/TheSpice-7b-FT-ExperimentalOrca is a 7 billion parameter language model developed by cgato. This model was specifically fine-tuned using a cleaned version of the Orca dataset alongside a proprietary roleplay (RP) data package. The primary objective behind its creation was to serve as an experimental testbed to assess whether an overwhelming amount of instruction tokens would negatively affect the model's roleplay evaluation performance.
Key Characteristics
- Parameter Count: 7 billion parameters.
- Training Data: Utilizes a cleaned Orca dataset combined with custom roleplay data.
- Context Length: Supports a context window of 4096 tokens.
- Experimental Focus: Designed to explore the interplay between instruction tuning and roleplay capabilities.
Intended Use
This model is particularly suited for researchers and developers interested in:
- Evaluating the robustness of roleplay performance in models exposed to significant instruction-tuning data.
- Experimenting with different prompting strategies for instruction-tuned models in conversational or roleplay contexts.
- Understanding the trade-offs between instruction following and creative generation in fine-tuned LLMs.