astom-M/matsuo-llm-advanced-phase-imdb1

TEXT GENERATIONConcurrency Cost:1Model Size:7.6BQuant:FP8Ctx Length:32kPublished:Mar 1, 2026License:apache-2.0Architecture:Transformer Open Weights Cold

The astom-M/matsuo-llm-advanced-phase-imdb1 is a 7.6 billion parameter language model, fine-tuned from Qwen2.5-7B-Instruct with a 32768 token context length. Developed by astom-M, this model specializes in agentic tasks, specifically excelling at database operations (SQL query generation) and household navigation. It was trained using Supervised Fine-Tuning (SFT) with QLoRA and instruction masking on a dataset of 6,750 samples combining DB operation and synthetic household task trajectories.

Loading preview...

Model Overview

The astom-M/matsuo-llm-advanced-phase-imdb1 is a 7.6 billion parameter language model, fine-tuned from the Qwen2.5-7B-Instruct base model. It leverages QLoRA with instruction masking for efficient training, resulting in a model optimized for specific agentic applications. The model was developed by astom-M for the Matsuo Lab LLM Advanced Competition 2025.

Key Capabilities

  • Database Operations: Proficient in generating SQL queries, trained on a diverse dataset including Spider/BIRD public datasets and distilled samples from Qwen2.5-72B-Instruct.
  • Household Navigation: Capable of understanding and executing tasks related to household navigation and manipulation, based on synthetic agent trajectories.
  • Efficient Fine-Tuning: Utilizes 4-bit QLoRA (r=32, alpha=64) during training, with loss computed only on assistant response tokens for focused learning.

Training Details

The model was trained on 6,750 samples, comprising both DB operation data and synthetic household task data. Training involved 1.0 epoch with a batch size of 4 (effective 16) and a learning rate of 5e-6, using a maximum sequence length of 4096 tokens. The final model is merged to bf16 for inference.

Good For

  • Applications requiring specialized SQL generation capabilities.
  • Developing agents for simulated household environments or robotics with navigation and manipulation tasks.
  • Researchers and developers participating in agentic LLM competitions focusing on structured data interaction and environmental control.

Finetunes
1 models