jingyeom/freeze_KoSoLAR-10.7B-v0.2_1.4_dedup
TEXT GENERATIONConcurrency Cost:1Model Size:15BQuant:FP8Ctx Length:8kTool Calling:SupportedPublished:Jan 29, 2024License:apache-2.0Architecture:Transformer Open Weights Cold
The jingyeom/freeze_KoSoLAR-10.7B-v0.2_1.4_dedup model is a 15 billion parameter instruction-tuned causal language model based on the yanolja/KoSOLAR-10.7B-v0.2 architecture. It was developed by jingyeom with a focus on instruction tuning and utilizes a deduplicated training dataset. This model is optimized for general language tasks, leveraging its instruction-tuned nature for improved response generation.
Loading preview...
Model Overview
The jingyeom/freeze_KoSoLAR-10.7B-v0.2_1.4_dedup model is an instruction-tuned large language model built upon the yanolja/KoSOLAR-10.7B-v0.2 base. It features 15 billion parameters and a context length of 8192 tokens, making it suitable for a variety of language understanding and generation tasks.
Key Characteristics
- Base Model: Derived from
yanolja/KoSOLAR-10.7B-v0.2. - Training Objective: Primarily focused on instruction tuning, indicating an optimization for following user prompts and generating coherent responses.
- Dataset: Training involved a publicly collected dataset, processed with a deduplication algorithm, specifically referencing the "Deduplicating Training Data Makes Language Models Better" approach. This suggests an effort to enhance data quality and model performance by reducing redundancy.
- Instruction Version: Utilizes instruction version 1.4, implying a specific methodology or format for instruction-based training.
Good For
- Applications requiring a model that can effectively follow instructions.
- General natural language processing tasks where a robust, instruction-tuned model is beneficial.
- Scenarios where a model trained with deduplicated data might offer improved generalization or reduced memorization.