tenyx/TenyxChat-7B-v1

TEXT GENERATIONConcurrency Cost:1Model Size:7BQuant:FP8Ctx Length:4kPublished:Jan 5, 2024License:apache-2.0Architecture:Transformer0.0K Open Weights Cold

TenyxChat-7B-v1 is a 7 billion parameter instruction-tuned causal language model developed by Tenyx, based on OpenChat 3.5. It is aligned using Direct Preference Optimization (DPO) on the UltraFeedback dataset, leveraging Tenyx's fine-tuning technology to mitigate catastrophic forgetting. This model is designed as a useful assistant, excelling in multi-turn chat benchmarks like MT-Bench while maintaining performance on general reasoning tasks.

Loading preview...

TenyxChat-7B-v1: An Aligned Chat Model

TenyxChat-7B-v1 is a 7 billion parameter chat model developed by Tenyx, built upon the OpenChat 3.5 base model. It utilizes Tenyx's advanced fine-tuning technology and the Direct Preference Optimization (DPO) framework, trained on the UltraFeedback dataset, to create a useful assistant.

Key Capabilities & Differentiators

  • Catastrophic Forgetting Mitigation: Tenyx's proprietary fine-tuning approach aims to prevent performance degradation on existing knowledge during continual fine-tuning, a common challenge for LLMs.
  • High MT-Bench Performance: At its release in January 2024, TenyxChat-7B-v1 was the highest-ranked 7B chat model on the MT-Bench evaluation, scoring an average of 8.103125. This benchmark assesses multi-turn conversational abilities across various categories like writing, reasoning, and coding.
  • Robust General Reasoning: Despite its chat optimization, the model demonstrates strong performance on the Open LLM Leaderboard benchmarks, including MMLU, Winogrande, GSM8k, ARC, HellaSwag, and TruthfulQA, indicating a balance between conversational and general reasoning skills.
  • Efficient Training: The model was trained using eight A100 GPUs (80GB) for only two hours, highlighting an efficient fine-tuning process.

Good For

  • Chatbot Applications: Its strong performance on MT-Bench makes it suitable for building responsive and coherent multi-turn conversational agents.
  • Assistant-like Interactions: Designed to function as a useful assistant, it can handle a variety of user queries in a conversational context.
  • Applications Requiring Continual Learning: The focus on mitigating catastrophic forgetting suggests potential for scenarios where models need to be updated or fine-tuned further without losing previously acquired knowledge.

Finetunes
3 models