Bonito-v1: Conditional Task Generation for Zero-Shot Adaptation

Bonito-v1, developed by Nihal V. Nayak, Yiyang Nan, Avi Trost, and Stephen H. Bach at BatsResearch, is a 7 billion parameter model based on mistralai/Mistral-7B-v0.1. Its core function is conditional task generation, which involves transforming unannotated text into synthetic instruction tuning datasets. This capability allows for the adaptation of large language models to specialized, private data without requiring manual annotations.

Key Capabilities

• Synthetic Dataset Generation: Creates instruction tuning datasets from raw text for various NLP tasks.
• Zero-Shot Task Adaptation: Enables fine-tuning of LLMs for specific tasks using generated data, as demonstrated in their paper.
• Supported Task Types: Proficient in generating tasks for summarization, sentiment analysis, multiple-choice QA, extractive QA, topic classification, natural language inference, question generation, text generation, paraphrase identification, and more.
• Training: Trained using Q-LoRA for 100,000 steps on a remixed dataset called ctga-v1.

Use Cases and Limitations

Bonito-v1 is ideal for developers looking to quickly generate training data for instruction tuning on specific tasks. It is particularly useful for adapting models to domains where annotated data is scarce. However, its effectiveness relies on the availability of sufficient unannotated text, and performance may drop with limited input. The model's task generation is limited to its predefined set of task types, and it carries inherent risks associated with large language models, such as generating factually incorrect data or exhibiting biases from its base model, Mistral-7B. Users are advised to thoroughly inspect generated tasks and benchmark performance before deployment.