electron-rare/spikingkiki-27b
SpikingKiKi 27B is a 27 billion parameter Spiking Neural Network (SNN) variant developed by Hypneum Lab, based on the Qwen3.5 architecture. It replaces standard activations with Leaky Integrate-and-Fire (LIF) spiking neurons in selected layers. This model is a research artifact exploring biologically-inspired sparse activation for energy-efficient inference, with a context length of 32768 tokens.
Loading preview...
SpikingKiKi 27B Overview
SpikingKiKi 27B, developed by Hypneum Lab, is a 27 billion parameter research model built upon the Qwen3.5 dense architecture. Its primary distinction lies in the integration of Leaky Integrate-and-Fire (LIF) spiking neurons, which replace conventional activations within specific attention and feed-forward network layers. This modification is part of an ongoing research effort to explore biologically-inspired sparse activation mechanisms, aiming for more energy-efficient inference in large language models.
Key Characteristics
- Architecture: Based on the Qwen3.5 dense model, comprising approximately 27 billion parameters.
- Spiking Neuron Integration: Utilizes LIF spiking neurons in selected layers, with specific parameters (threshold, decay, refractory period) detailed in
lif_metadata.json. - Context Length: Supports a context window of 32768 tokens.
- Research Focus: Developed as a research artifact within the SpikingKiki line, contributing to the GENIAL framework for energy-efficient AI.
Current Status
This model is currently in a research stage and has not undergone end-to-end benchmarking. It represents an experimental approach to neural network design rather than a production-ready solution.
Good For
- Researchers and developers interested in Spiking Neural Networks (SNNs) and their application to large language models.
- Experiments with biologically-inspired AI architectures and energy-efficient inference methods.
- Exploring the impact of sparse activation on model performance and resource consumption.