TIGER-Lab/FIM-Mid-14B
TIGER-Lab/FIM-Mid-14B is a 14.8 billion parameter mid-trained checkpoint from the TIGER-Lab FIM pipeline, based on Qwen2.5-Coder-14B-Instruct. This model has undergone function-aware Fill-in-the-Middle (FIM) mid-training, utilizing a 32768 token context length. It is specifically released for reproducibility and as a foundation for further post-training, particularly for developing coding agent models.
Loading preview...
TIGER-Lab/FIM-Mid-14B Overview
FIM-Mid-14B is a 14.8 billion parameter model developed by TIGER-Lab, serving as a crucial mid-trained checkpoint within their Function-Aware Fill-in-the-Middle (FIM) pipeline. It is built upon the Qwen/Qwen2.5-Coder-14B-Instruct base model and has been specifically enhanced through FIM mid-training, which focuses on function-aware code completion and generation.
Key Characteristics
- Mid-Training Focus: The model has undergone specialized FIM mid-training on the TIGER-Lab/FIM-Midtraining-400K dataset, designed to improve its understanding and generation of code, particularly in fill-in-the-middle scenarios.
- Foundation for Agents: This checkpoint is intended as a base for subsequent post-training, specifically for creating coding agent models. For instance, post-training this checkpoint on R2E-Gym agent trajectories yields TIGER-Lab/FIM-14B.
- Reproducibility: Released to facilitate research and development, allowing others to reproduce the FIM pipeline and further fine-tune the model for agent-specific tasks.
- Context Length: Supports a native context length of 32768 tokens, suitable for handling extensive codebases.
Intended Use Cases
- Agent Development: Ideal for researchers and developers looking to build and train advanced coding agents, serving as a robust starting point before agent-specific post-training.
- Code Generation Research: Useful for exploring and reproducing the effects of function-aware FIM mid-training on large language models for code.
- Further Fine-tuning: Provides a strong foundation for various code-related fine-tuning tasks, especially those requiring sophisticated code completion and understanding capabilities.