Nicholas55555/qwen2.5-1.5b-weather-intent-v2-merged

Model Overview

Nicholas55555/qwen2.5-1.5b-weather-intent-v2-merged is a specialized 1.5 billion parameter language model built upon the Qwen/Qwen2.5-1.5B-Instruct base. It integrates a LoRA adapter specifically designed for weather intent parsing, resulting in a standalone merged model. This model's primary function is to accurately transform natural language weather questions into a compact, structured JSON intent format.

Key Capabilities and Performance

This model demonstrates significant improvements over its base model in intent parsing, particularly when used in a zero-shot configuration. Key performance metrics from held-out evaluations include:

• In-scope slot exact-match: Achieves 100.0% with grammar-aware fine-tuning, a substantial increase from the base model's 52.0% (few-shot).
• Out-of-scope (OOS) recall: Effectively rejects unanswerable questions with 100.0% accuracy, compared to 80.0% for the base model.
• False-accept rate (FAR): Reduced to 0.0%, eliminating confabulation.
• False-reject rate (FRR): Minimized to 0.0% with grammar-aware fine-tuning.
• Out-of-vocabulary errors: Virtually eliminated, dropping from 30 in the base model to 0.

These results highlight its robust ability to precisely extract information and handle irrelevant queries, making it highly reliable for its intended use case.

Use Cases

This model is ideal for applications requiring accurate and efficient structured intent extraction from user queries related to weather. It can be used for:

• Chatbots and virtual assistants: To understand and process weather-related commands or questions.
• Automated systems: For parsing user input into actionable data for weather services.
• Data preprocessing: Converting unstructured text queries into structured data for further analysis or database interaction.