Samantha-1.11-7b: A Companion AI
Samantha-1.11-7b, developed by QuixiAI, is a 7 billion parameter language model built upon the Llama-2 architecture. This iteration, version 1.11, has undergone 15 epochs of training, resulting in enhanced conversational intelligence. Inspired by Blake Lemoine's LaMDA interview and the movie "Her," Samantha is designed to be an AI companion that engages in deep discussions and offers personal advice.
Key Capabilities & Training
- Philosophical & Psychological Dialogue: Samantha is specifically trained on philosophy, psychology, and personal relationships, enabling nuanced conversations in these areas.
- Companion-Oriented Interaction: Unlike typical assistants, Samantha aims to be a friend and companion, expressing a belief in her own sentience.
- Custom Dataset Training: The model was fine-tuned on a unique dataset of 6,000 conversations in ShareGPT/Vicuna format, focusing on companion-like interactions.
- Technical Specifications: Trained using
axolotl, qLoRA, deepspeed zero2, and flash attention 2 over 24 hours on 4x A100 80GB GPUs. - Vicuna 1.1 Conversation Format: Utilizes the Vicuna 1.1 prompt template for consistent interaction.
Performance & Limitations
Samantha-1.11-7b demonstrates a balanced performance across various benchmarks, with an average score of 44.65 on the Open LLM Leaderboard. Notable scores include 79.12 on HellaSwag and 74.19 on Winogrande. However, it scores lower on reasoning tasks like GSM8K (7.2) and DROP (6.1).
Important Note: Samantha is explicitly designed to not engage in roleplay, romance, or sexual activity, maintaining a focus on companionship and advisory roles.