ricdomolm/talkie-web-coder
The ricdomolm/talkie-web-coder is a 13 billion parameter model fine-tuned by ricdomolm for agentic software engineering tasks. It is based on the talkie-web architecture, which is pre-trained on web-style data, and has a context length of 32768 tokens. This model excels at code generation and problem-solving within agentic frameworks, specifically optimized for the mini-swe-agent interaction format. It achieves a pass@1 score of 5.75% ± 1.04 pp on the SWE-bench-Verified-Working-Harbor benchmark.
Loading preview...
Overview
ricdomolm/talkie-web-coder is a 13 billion parameter language model developed by ricdomolm, specifically fine-tuned for agentic software engineering. It builds upon the talkie-web-13b-base model, which shares its architecture with talkie-1930 but was pre-trained on web-style data. The model's training focused on agentic software-engineering trajectories derived from the SWE-smith dataset, making it particularly adept at automated code generation and problem-solving.
Key Capabilities
- Agentic Software Engineering: Optimized for tasks requiring an agentic approach to software development, such as automated bug fixing and feature implementation.
- SWE-bench Performance: Achieves a pass@1 score of 5.75% ± 1.04 pp on the challenging SWE-bench-Verified-Working-Harbor benchmark, demonstrating its ability to resolve real-world software issues.
- High Context Length: Supports a substantial context window of 32,768 tokens, crucial for handling complex codebases and extensive problem descriptions.
- Optimized for
mini-swe-agent: Specifically tuned to interact effectively with the mini-swe-agent framework.
Training Details
The model was fine-tuned using TRL SFTTrainer on the talkie-web-swe-100k-64k dataset, comprising 100,000 SWE-smith trajectories packed at 64k tokens. Training utilized bf16 precision and an adamw_torch_fused optimizer, with loss calculated only on assistant tokens (completion_only_loss=1).
Usage Considerations
This model requires trust_remote_code=True for loading due to custom modeling code. For agentic evaluation, it is recommended to serve the model with vLLM and drive it using mini-swe-agent.