zai-org/agentlm-70b

AgentLM-70B: Enhancing LLMs for Agentic Tasks

AgentLM-70B is a 69 billion parameter language model developed by THUDM, specifically designed to imbue Large Language Models (LLMs) with advanced agent capabilities. It is built upon the Llama-2-chat architecture and features a 32,768-token context length.

Key Capabilities and Training

AgentLM models are the result of AgentTuning, a novel approach that involves instruction-tuning LLMs using interaction trajectories collected across a variety of agent tasks. This mixed training methodology, utilizing both the AgentInstruct dataset and the ShareGPT dataset, allows AgentLM-70B to:

• Generalize robustly to previously unseen agent tasks.
• Maintain strong performance on general language understanding and generation abilities.
• Follow the Llama-2-chat conversation format, with a fixed system prompt: "You are a helpful, respectful and honest assistant."

What Makes AgentLM-70B Different?

AgentLM-70B stands out as it represents the first attempt to instruction-tune LLMs specifically for agent capabilities through interaction trajectories. This focus on agentic behavior, rather than just general instruction following, makes it particularly suited for applications requiring complex, multi-step reasoning and interaction. The underlying research and dataset, AgentInstruct, have been open-sourced, providing transparency and a foundation for further development.

Use Cases

AgentLM-70B is particularly well-suited for applications requiring:

• Automated task execution and planning.
• Complex problem-solving that involves sequential decision-making.
• Interactive agents that can adapt to various scenarios.

For more details, refer to the AgentTuning project page and the research paper.