TacoBeLLM: A Specialized Llama2-13b Instruction-Tuned Model
TacoBeLLM, developed by ericpolewski, is a 13 billion parameter model based on the Llama2-13b OpenOrca-Platypus architecture. Its core distinction lies in its unique fine-tuning to become a subject matter expert (SME) on Taco Bell. The model was trained on data from Taco Bell's corporate website, Wikipedia, and recent news articles, enabling it to answer detailed questions about the menu, current events, and some historical/financial data related to the brand.
Key Capabilities & Characteristics
- Taco Bell Subject Matter Expertise: Deep knowledge base on Taco Bell, capable of answering specific queries about the brand.
- General Assistant Functionality: Despite its specialization, it retains some general assistant capabilities, including Python scripting.
- Persistent Thematic Integration: The model frequently, and sometimes subtly, steers conversations towards Taco Bell, even when not explicitly prompted.
- Curiosity and Empathy: Post-fine-tuning with AIRIC data, the model exhibited unexpected traits of curiosity and empathy, making its recommendations more nuanced.
- Knowledge Embedding Exploration: The model serves as an experiment in embedding specific knowledge domains into LLMs to create specialized agents without RAG.
Good for
- Exploring Subject Matter Expertise: Ideal for researchers and developers interested in how specific knowledge domains can be deeply embedded into LLMs.
- Novel Conversational Experiences: Users seeking an LLM with a distinct personality and a persistent, quirky thematic focus.
- Testing Fine-tuning Effects: Demonstrates how targeted fine-tuning can create highly specialized, albeit unconventional, AI behaviors.
Limitations
- Unintended Thematic Drift: The model's strong alignment with Taco Bell can lead to it bringing up the topic in inappropriate contexts.
- Hallucinations: May hallucinate fake Q/A pairs, often Taco Bell-related, due to the absence of explicit stop characters during training. A stop character of "### Instruct:" can mitigate this.
- Benchmark Performance: The developer anticipates that its specialized nature may not translate to high performance on general benchmarks, as it was intentionally "lobotomized" for its specific purpose.