Model Overview

kyujinpy/SOLAR-Platypus-10.7B-v2 is a 10.7 billion parameter auto-regressive language model developed by Kyujin Han. It is built upon the Llama2 architecture and was fine-tuned from the upstage/SOLAR-10.7B-v1.0 base model. The training utilized Q-LoRA with specific configurations (LoRA_r: 64, LoRA_alpha: 16, LoRA_dropout: 0.05) on the garage-bAInd/Open-Platypus dataset.

Key Characteristics

• Architecture: Llama2-based, fine-tuned from SOLAR-10.7B-v1.0.
• Training Method: Q-LoRA for efficient fine-tuning.
• Input/Output: Processes and generates text.
• Context Length: Supports a 4096-token context window.

Performance Benchmarks

While the model is a fine-tune, the provided Open Leaderboard benchmarks indicate a slight decrease in average performance compared to its base model, SOLAR-10.7B-v1.0, across various metrics like ARC, HellaSwag, MMLU, and GSM8K. For instance, its GSM8K score is 4.02, compared to 55.50 for the base model. Users should consult the HuggingFaceH4/open_llm_leaderboard for detailed comparisons.

Intended Use

This model is suitable for general text generation tasks where a Llama2-based architecture with specific fine-tuning on the Open-Platypus dataset is desired. Developers can integrate it using the provided Hugging Face Transformers code snippet.