TIGER-Lab/FIM-Mid-7B
TIGER-Lab/FIM-Mid-7B is a 7.6 billion parameter language model developed by TIGER-Lab, serving as the mid-trained checkpoint of the FIM 7B pipeline. It is based on Qwen2.5-Coder-7B-Instruct and has undergone function-aware Fill-in-the-Middle (FIM) mid-training, supporting a 32768 token context length. This model is released for reproducibility and further post-training, specifically as a foundational step before agent post-training to produce models like TIGER-Lab/FIM-7B.
Loading preview...
Overview
TIGER-Lab/FIM-Mid-7B is a 7.6 billion parameter model from TIGER-Lab, representing a crucial mid-training checkpoint in the development of coding agent foundation models. It is derived from the Qwen/Qwen2.5-Coder-7B-Instruct base model and has been specifically enhanced through function-aware Fill-in-the-Middle (FIM) mid-training. This model supports a substantial context length of 32768 tokens.
Key Characteristics
- Base Model: Built upon
Qwen/Qwen2.5-Coder-7B-Instruct. - Training Focus: Underwent function-aware FIM mid-training using the TIGER-Lab/FIM-Midtraining-400K dataset.
- Purpose: Released primarily for reproducibility and as a starting point for further post-training, particularly for developing agent models like TIGER-Lab/FIM-7B.
- Context Length: Supports a native context length of 32768 tokens.
Intended Use
This model is not intended for direct use as a standalone instruction-following model, as its instruction-following capabilities are noted to be degraded before agent post-training. Instead, it is designed for:
- Reproducibility: Allowing researchers to replicate the FIM mid-training stage.
- Further Post-training: Serving as the foundational checkpoint for subsequent agent post-training, such as the R2E-Gym trajectory SFT to create models like FIM-7B.