Cyrema/Llama-2-7b-Bogpit
Cyrema/Llama-2-7b-Bogpit is a Llama-2-based language model developed by Cyrema, fine-tuned on a specialized dataset of image board posts. This model is designed for text completion based on a unique, filtered dataset, focusing on generating coherent and relevant continuations to user input. It is not instruction-tuned, making it suitable for direct text generation rather than chat or instructional formats. The model's training on specific subject matter from image boards differentiates its output style and content focus.
Loading preview...
Model Overview
Cyrema/Llama-2-7b-Bogpit is a specialized language model built upon the LLaMA-2 backbone. Developed by Cyrema, this model is uniquely fine-tuned on a heavily filtered dataset derived from image board posts, focusing on a particular subject. The training process involved 361,050 entries, augmented with various filtering techniques to enhance coherency and relevance to its origin and goals.
Key Characteristics
- Backbone: LLaMA-2, indicating a strong foundational architecture.
- Training Data: Unique dataset scraped from image board posts, processed with extensive filtering.
- Training Method: Utilized Axolotl for training, with specific parameters including a rank and alpha of 128 and 16 respectively, a learning rate of 2e-4, and sample packing.
- Inference Style: Not instruction or chat-style trained. Users should provide direct input for text completion, as the model will attempt to continue the given text.
Use Cases and Limitations
This model is best suited for generating text completions based on direct input, particularly for content aligned with its specialized training data. Due to its unique training on image board content and non-instructional format, it is recommended that users thoroughly understand its behavior and limitations before deployment in real-world environments. The model's license is governed by Meta's LLaMA-2 terms.