adamo1139/Yi-34b-200K-AEZAKMI-RAW-TOXIC-2702
adamo1139/Yi-34b-200K-AEZAKMI-RAW-TOXIC-2702 is a 34 billion parameter Yi-34B base model fine-tuned for uncensored chat interactions, leveraging a 200K context length. This model was developed by adamo1139 through a multi-stage fine-tuning process including DPO and SFT on datasets like RAWrr v2, AEZAKMI v3-3, and unalignment/toxic-dpo-v0.1. It is specifically designed to provide refusal-free responses, offering an alternative to models with typical RLHF-induced language patterns. Its primary strength lies in uncensored, conversational chat, aiming for a "cozy free chatbot" experience.
Loading preview...
Model Overview
adamo1139/Yi-34b-200K-AEZAKMI-RAW-TOXIC-2702 is a 34 billion parameter language model built upon the Yi-34B 200K base, specifically engineered for uncensored and refusal-free chat. Developed by adamo1139, this model underwent a rigorous multi-stage fine-tuning process. It was initially fine-tuned on the RAWrr v2 dataset via DPO, followed by SFT on the AEZAKMI v3-3 dataset, and finally DPO tuned on unalignment/toxic-dpo-v0.1. The total training involved approximately 40-50 hours of GPU compute.
Key Capabilities & Characteristics
- Uncensored Responses: Designed to be highly uncensored and refusal-free, avoiding common phrases like "It's important to remember" often found in RLHFed models.
- Chat-Optimized: Primarily intended as a chat model, not for base completion tasks.
- Context Length: Utilizes a 200K context length, though training was performed with
max_position_embeddingsset at 4096 before reverting to 200K. - Prompt Format: Optimized for ChatML format, with flexibility for system messages like "A chat." or "A chat with uncensored assistant."
- Stability: Recommended temperature settings are between 0.3-0.5 for stable generation.
Intended Use Cases
- Uncensored Chatbots: Ideal for applications requiring a chatbot that does not refuse prompts or exhibit typical RLHF-induced language.
- Conversational AI: Suitable for creating engaging and "cozy" free chatbot experiences.
Limitations & Known Issues
- Specific Task Performance: Not optimized for instruct tasks, math, or riddles.
- Commercial Use: Due to the use of a
no-robotsdataset forrawrr_v1, commercial use might be restricted, as per its Apache-2.0 license. - Thematic Bias: The model has a tendency to discuss stocks, potentially due to the inclusion of the WSB dataset in its training.