Nash: An Economics Tutoring Chatbot

Nash is a 13 billion parameter educational tutoring chatbot, developed by a team from Rice University and OpenStax. It is built upon the Vicuna 1.5 model, which itself is a fine-tuned version of LLaMA, specifically adapted for educational interactions in economics.

Key Capabilities & Training

• Specialized Tutoring: Nash is fine-tuned on approximately 700 synthetic student-tutorbot conversations. These conversations were generated using a specialized prompt with GPT-4, drawing content from OpenStax Economics, Microeconomics, and Macroeconomics textbooks.
• Educational Focus: The model's training is geared towards providing educational support and generating relevant dialogues within the domain of economics.
• Performance: On the Hugging Face Open LLM Leaderboard, Nash (fine-tuned on vicuna-13b-v1.5) achieves an average score of 61.8, with specific scores of 59.13 on ARC, 80.64 on HellaSwag, 56.12 on MMLU, and 51.29 on TruthfulQA. This performance is comparable to or slightly better than the base lmsys/vicuna-13b-v1.5 model.

Use Cases

• Economics Education: Ideal for applications requiring an AI tutor or conversational agent focused on economics topics.
• Synthetic Conversation Generation: Can be used to generate realistic student-tutor interactions for research or development in educational AI.

Limitations & Considerations

As a derivative of the LLaMA model, Nash adheres to the LLaMA use policy and shares similar ethical considerations and limitations. It is primarily designed for English-language interactions and its outputs, like all LLMs, cannot be predicted with absolute certainty, potentially producing inaccurate or biased responses. Developers should conduct thorough safety testing for specific applications.