Dolphin 2.9.4 Gemma2 2b Overview

Dolphin 2.9.4 Gemma2 2b is a 2.6 billion parameter language model developed by Eric Hartford and Cognitive Computations. It is built upon Google's Gemma2 2b base model and utilizes an 8192 token sequence length for fine-tuning, despite the base model's 128K context. A key aspect of its training involved the use of GrokAdamW and Liger Kernel, with GrokAdamW intended to enhance generalization through fast Grokking.

Key Capabilities

• Instruction Following: Specially trained to obey system prompts and follow instructions across various languages.
• Conversational Skills: Capable of engaging in natural dialogue.
• Coding Abilities: Possesses skills for code generation and understanding.
• Agentic Features: Supports agentic behaviors and function calling.
• Uncensored Nature: The model is uncensored, with its dataset filtered to remove alignment and bias, making it highly compliant with any requests. Users are advised to implement their own alignment layers.

Prompt Format

The model uses the ChatML prompt template format, as demonstrated by the example:

<|im_start|>system
You are Dolphin, a helpful AI assistant.<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant