OpenChat-3.5-0106-Gemma: High-Performance Gemma Model

OpenChat-3.5-0106-Gemma is an 8.5 billion parameter model developed by OpenChat, representing a significant advancement in the Gemma family. It leverages OpenChat's C-RLFT (Controlled-Reinforcement Learning from Human Feedback) training methodology, applied to the openchat-3.5-0106 dataset, to achieve superior performance.

Key Capabilities & Performance

This model demonstrates strong capabilities across a range of benchmarks, often outperforming its base Gemma counterparts and showing comparable results to Mistral-based OpenChat models. Notable performance highlights include:

• Reasoning: Achieves 52.7 on BBH MC and 50.2 on AGIEval, surpassing other 7B models like OpenChat-3.5-0106 Mistral and Gemma-7B-it.
• Mathematics: Scores 81.5 on GSM8K, indicating strong mathematical problem-solving abilities.
• General Performance: Boasts an average score of 64.4, placing it among the highest-performing Gemma models.

Usage and Deployment

OpenChat provides an optimized OpenAI-compatible API server for deploying this model, supporting high-throughput inference with vLLM. It can run on consumer GPUs with 24GB RAM and supports tensor parallelism. Users can interact with the model via a local API server or the OpenChat Web UI. The model uses a specific conversation template with <end_of_turn> as the end-of-turn token.

Considerations

Users should be aware of potential hallucination of non-existent or inaccurate information. Additionally, the model may generate harmful or biased responses, necessitating the application of AI safety measures for sensitive use cases. The model and its code are distributed under the Apache License 2.0.