hauser458original/lfm2.5-350m-python-math
hauser458original/lfm2.5-350m-python-math is a 350 million parameter instruction-tuned causal language model, fine-tuned from LiquidAI/LFM2.5-350M. It specializes in Python code generation and mathematical word problem-solving, while maintaining general chat capabilities. The model features a 32768-token context length and is optimized for reliable performance in its specialized domains.
Loading preview...
Model Overview
hauser458original/lfm2.5-350m-python-math is a 350 million parameter language model, fine-tuned from LiquidAI/LFM2.5-350M (instruct version). This model specifically targets Python code generation and math word-problem solving, while also retaining general conversational abilities. It addresses limitations found in previous smaller models by incorporating a mixed dataset, custom fix-it examples, and optimized training parameters.
Key Capabilities
- Python Code Generation: Generates complete and runnable Python scripts, including complex structures like Pygame loops, file I/O, classes, and algorithms.
- Mathematical Reasoning: Excels at GSM8K-style word problems, providing step-by-step reasoning for algebra, percentages, geometry, and multi-step arithmetic.
- General Chat: Maintains coherent conversational ability, correctly handling negative constraints and distinguishing between similar concepts (e.g., baking cookies vs. HTTP cookies).
- Efficiency: At 350M parameters, it offers fast generation speeds, achieving approximately 157 tokens/second on a laptop CPU with Q5_K_S quantization.
Training and Improvements
This model was fine-tuned using a full fine-tuning method on a balanced dataset including iamtarun/python_code_instructions_18k_alpaca for Python, openai/gsm8k for math, and yahma/alpaca-cleaned for general chat. Key improvements over prior versions include starting from an instruct checkpoint, using a longer 2048-token sequence length to prevent code truncation, and reducing epochs and learning rate to prevent overfitting. Custom examples were injected to specifically address issues like negative constraints and incomplete code generation.
Limitations
- Python-only: Specialized exclusively for Python code; does not support other programming languages.
- Scale-dependent: As a 350M parameter model, it may not exhibit the deep multi-step reasoning or long-form creative writing capabilities of much larger models.
- May occasionally misattribute its developer or struggle with strict sentence count constraints.