allenai/llama-3.1-tulu-2-dpo-70b

TEXT GENERATIONConcurrency Cost:4Model Size:70BQuant:FP8Ctx Length:32kPublished:Aug 9, 2024License:apache-2.0Architecture:Transformer Open Weights Cold

allenai/llama-3.1-tulu-2-dpo-70b is a 70 billion parameter language model developed by AllenAI, fine-tuned from Meta-Llama-3.1-70B. It is designed as a helpful assistant, trained on a diverse mix of public, synthetic, and human datasets, and further aligned using DPO on the UltraFeedback dataset. This model excels in instruction following and general conversational tasks, offering improved truthfulness and safety compared to its base model.

Loading preview...

Model Overview

allenai/llama-3.1-tulu-2-dpo-70b is a 70 billion parameter language model developed by AllenAI, building upon the Meta-Llama-3.1-70B base model. It is part of the Tulu series, which focuses on creating helpful assistant models. This specific iteration undergoes a two-stage training process: initial fine-tuning on a diverse mix of publicly available, synthetic, and human-created datasets, followed by further alignment using Direct Preference Optimization (DPO) on the UltraFeedback dataset.

Key Capabilities & Performance

  • Instruction Following: Designed to act as a helpful assistant, demonstrating strong capabilities in understanding and executing user instructions.
  • DPO Alignment: Utilizes DPO training on human preference data (UltraFeedback) to enhance helpfulness, harmlessness, and truthfulness.
  • Improved Benchmarks: Shows competitive performance across various benchmarks, including MMLU (76.0), GSM8k (88.5), BBH (79.9), and TruthfulQA (%Info+True 78.3), often outperforming its Tulu 2 Llama 3.1 70b predecessor and the base Llama 3.1 70b instruct model in specific areas like TruthfulQA.
  • Primary Language: Primarily English.

Intended Uses & Limitations

This model is suitable for applications requiring a capable conversational AI assistant. It is important to note that while DPO training improves alignment, the model has not undergone extensive RLHF for safety and may still produce problematic outputs, especially when intentionally prompted. Users should implement their own safety measures and content filtering.

Finetunes
1 models