What is Goppa-LogiLlama?

Goppa-LogiLlama is a 1 billion parameter Small Language Model (SLM) developed by Goppa AI, fine-tuned from a LLaMA base. Unlike the trend of ever-larger models, LogiLlama focuses on making smaller models smarter by injecting advanced logical reasoning techniques and knowledge. This approach aims to deliver enhanced reasoning and problem-solving capabilities while maintaining efficiency.

Key Capabilities & Features

• Enhanced Reasoning: Improved logical thinking and knowledge integration for more accurate and context-aware responses.
• Efficiency: Designed for on-device processing with a low memory and energy footprint, making it suitable for resource-constrained environments.
• Transparency: Goppa AI provides open-source training processes and configuration files, promoting reproducible research.
• Custom Architecture: Incorporates a customized ROPE scaling (llama3 type) and a custom tokenizer with an extensive set of special tokens.

Why is it different?

LogiLlama challenges the notion that "bigger is always better" by demonstrating that significant reasoning improvements can be achieved in a 1B parameter model through specialized fine-tuning. It prioritizes efficiency and logical depth over raw parameter count, making it ideal for applications where computational resources are limited.

Good for:

• On-device AI applications requiring logical reasoning.
• Scenarios where memory and energy efficiency are critical.
• Developers interested in transparent and reproducible research in SLMs.