Overview

appvoid/palmer-002 is a 1.1 billion parameter Llama 2-based language model developed by appvoid. It is designed to serve as a "better base model," offering a unique blend of assistant-like characteristics and robust next-word prediction capabilities derived from its internet knowledge base. This model can be used directly for various downstream tasks or further fine-tuned with custom prompts, providing flexibility for developers.

Key Capabilities & Performance

• Improved Base Model: Fine-tuned to act as a base model with some inherent "bias" for assistant-like interactions while retaining strong general knowledge.
• Zero-Shot Evaluation: Demonstrates competitive zero-shot performance against other models in its size class, including tinyllama-2.5 and babbage-002, across benchmarks like ARC_C, HellaSwag, PIQA, and Winogrande.
• Efficient Training: Trained on 15,000 GPT-4 shuffled samples using lower learning rates to preserve general knowledge, with training completed in approximately 3.5 P100 GPU hours.
• Llama 2 Compatibility: Built on the Llama 2 architecture, ensuring compatibility with existing tools and frameworks.

When to Use This Model

palmer-002 is particularly suitable for:

• Developers seeking a small, efficient base model (around 1 billion parameters) that offers a good starting point for various NLP tasks.
• Use cases where a model with a balance of general knowledge and a slight assistant-like predisposition is beneficial.
• Scenarios requiring a base model that can be easily fine-tuned further without extensive initial prompt engineering.