Overview
This model, developed by Rasa Technologies, is a Dialogue Understanding (DU) model specifically designed to power AI assistants built with the Conversational AI with Language Models (CALM) approach. Fine-tuned from CodeLlama 13b Instruct, it takes a conversation transcript and structured business logic as input.
Key Capabilities
- Command Generation: Outputs a short sequence of predefined commands (e.g.,
StartFlow(flow_name), SetSlot(slot_name, slot_value), Clarify, ChitChat, KnowledgeAnswer, HumanHandoff, Error). - Internal Grammar Translation: Translates user messages into an internal command grammar, enabling CALM to progress conversations.
- Specialized Functionality: Explicitly fine-tuned for command output; it cannot generate arbitrary free-form text.
Use Cases and Limitations
This model is primarily intended for use within Rasa's CALM paradigm for applications like customer-facing chatbots, voice assistants, and IVR systems. It can be used directly if your assistant's flows are similar to the rasa-calm-demo assistant or as a base for further fine-tuning. Due to its specialized nature, it is out-of-scope for general text generation. While it does not generate problematic user-facing text, its predictions are susceptible to biases, and performance has primarily been tested in English.
Training and Evaluation
The model was trained on the train split of rasa/command-generation-calm-demo-v1 and evaluated on its test split. Key metrics include F1 score per command type, with notable scores such as 0.9722 for StartFlow and 0.9239 for SetSlot.