Sao10K/Llama-3.3-70B-Vulpecula-r1

Sao10K/Llama-3.3-70B-Vulpecula-r1 Overview

Llama-3.3-70B-Vulpecula-r1 is a 70 billion parameter language model, a collaborative effort by Sao10K and GradientPutri. Built upon Meta's Llama 3.3 architecture, this model features a 32768 token context length and is designed as a "thinking-based" model, drawing inspiration from Deepseek-R1. It has undergone fine-tuning using both Supervised Fine-Tuning (SFT) and a limited application of Reinforcement Learning (RL) on creative writing datasets.

Key Capabilities

• Thinking Mode: Inspired by Deepseek-R1, the model can activate a "thinking mode" by prefilling or beginning assistant replies with <think>\n, though it performs well without it.
• Enhanced Control: Offers improved steerability, instruct-roleplay capabilities, and greater creative control compared to its base model.
• Dataset Composition: Trained on a diverse mix of semi-synthetic and human-based chat/roleplaying datasets, along with cleaned instruct datasets and reasoning traces from Deepseek-R1 for various tasks.

Good For

• Creative Writing: Optimized for generating imaginative and nuanced text, particularly in creative writing scenarios.
• Roleplaying & Chat: Excels in instruct-roleplay and natural chat applications due to its specialized training data.
• Complex Instruction Following: Benefits from reasoning traces, making it suitable for tasks requiring detailed instruction adherence and thought processes.