Model Overview
The astom-M/matsuo-llm-advanced-phase-imdb1 is a 7.6 billion parameter language model, fine-tuned from the Qwen2.5-7B-Instruct base model. It leverages QLoRA with instruction masking for efficient training, resulting in a model optimized for specific agentic applications. The model was developed by astom-M for the Matsuo Lab LLM Advanced Competition 2025.
Key Capabilities
- Database Operations: Proficient in generating SQL queries, trained on a diverse dataset including Spider/BIRD public datasets and distilled samples from Qwen2.5-72B-Instruct.
- Household Navigation: Capable of understanding and executing tasks related to household navigation and manipulation, based on synthetic agent trajectories.
- Efficient Fine-Tuning: Utilizes 4-bit QLoRA (r=32, alpha=64) during training, with loss computed only on assistant response tokens for focused learning.
Training Details
The model was trained on 6,750 samples, comprising both DB operation data and synthetic household task data. Training involved 1.0 epoch with a batch size of 4 (effective 16) and a learning rate of 5e-6, using a maximum sequence length of 4096 tokens. The final model is merged to bf16 for inference.
Good For
- Applications requiring specialized SQL generation capabilities.
- Developing agents for simulated household environments or robotics with navigation and manipulation tasks.
- Researchers and developers participating in agentic LLM competitions focusing on structured data interaction and environmental control.