SymbioticLM-14B: Hybrid Symbolic–Transformer Model
SymbioticLM-14B is a 17.8 billion parameter hybrid model developed by reaperdoesntknow, combining a Qwen-14B transformer backbone with advanced symbolic cognition modules. It aims to excel in symbolic domains by tightly coupling neural representation with structured symbolic processing.
Key Architectural Features
- Base Model: Qwen-14B transformer with rotary embeddings and FlashAttention.
- Symbolic Modules: Incorporates ThoughtDynamicsLNN, LiquidThoughtProcessor, CrystallineProcessor (DNAConv GNN), and HelicalDNAProcessor for complex symbolic operations.
- Persistent Memory: Utilizes 4096 symbolic states in FP32, retrieved via entropy and contextual similarity, enabling "true memory" for multi-step interactions.
- Dream Mode: Features a background symbolic simulation for open-ended cognitive processes.
- Router: Employs an intent classifier and entropy gating to select appropriate processing paths.
Intended Use Cases
- Advanced Reasoning Agents: Ideal for multi-step conversational agents requiring persistent memory.
- Symbolic Generation: Suited for long-form symbolic theorem generation, proof planning, and symbolic math/code synthesis.
- Scientific Applications: Useful for scientific dialogue, symbolic simulations, and reasoning in fuzzy or discontinuous problem domains.
Limitations
- Memory utility is enhanced with curation and seeding.
- Symbolic cognition is not instruction-tuned for general question answering.
- FlashAttention and symbolic modules increase VRAM usage during generation.