lattice-research/lattice-spark-1.5b

Lattice Spark 1.5B Overview

Lattice Spark 1.5B is a 1.5 billion parameter instruction-tuned language model, fine-tuned from Qwen2.5-1.5B-Instruct. Its core innovation lies in its deeply integrated identity, enabling it to genuinely identify as "Lattice Spark" without the need for system prompts, a significant improvement over its base model which would attribute itself to other entities. This identity was baked into the weights through Supervised Fine-Tuning (SFT) using MLX LoRA on Apple Silicon (Mac M4).

Key Differentiators & Performance

• Integrated Identity: Unlike models relying on system prompts, Spark's identity is intrinsic, achieving a score of 7/8 on identity benchmarks compared to 0/8 for the base Qwen 1.5B.
• Knowledge Retention: Despite identity training, the model largely retains the factual knowledge of its base, with only a minor decrease (7/8 vs 8/8) in factual benchmarks.
• Efficient Training: Trained on 27 Lattice identity examples (weighted 3x) and 30 general Q&A examples over 50 iterations, taking approximately 35 seconds on an M4 MacBook Air.

Limitations

• Small Scale: As a 1.5B parameter model, it is fluent but not deeply knowledgeable across all domains.
• Limited Training Data: General knowledge is primarily derived from the base model, as SFT focused on identity.
• No Multi-turn Optimization: Not specifically optimized for complex multi-turn conversations.
• Occasional Artifacts: May generate trailing artifacts on some prompts.

Licensing

Lattice Spark 1.5B is released under the Apache 2.0 license, while its base model, Qwen2.5-1.5B-Instruct, retains its original license.