CallComply/SOLAR-10.7B-Instruct-v1.0-128k is a 10.7 billion parameter instruction-tuned large language model developed by Upstage, based on the SOLAR-10.7B architecture. It leverages a depth up-scaling (DUS) methodology, integrating Mistral 7B weights and continued pretraining, to achieve strong performance. This model is specifically fine-tuned for single-turn conversations and features an extended 128k context window, making it suitable for tasks requiring extensive context processing in direct interactions.
Loading preview...
Model Overview
CallComply/SOLAR-10.7B-Instruct-v1.0-128k is an instruction-tuned version of Upstage's 10.7 billion parameter SOLAR model, featuring an impressive 128k context window. It is built upon a novel depth up-scaling (DUS) methodology, which involves integrating Mistral 7B weights into upscaled layers and subsequent pretraining. This approach allows it to achieve competitive performance, even surpassing models with significantly more parameters like Mixtral 8x7B in certain evaluations.
Key Capabilities
- High Performance: Demonstrates strong results on various NLP tasks, outperforming many models under 30B parameters.
- Extended Context: Supports a 128k context length, enabling processing of very long inputs.
- Instruction Following: Fine-tuned using a combination of Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) on diverse datasets, including in-house generated data.
- Data Contamination Tested: Rigorously tested to ensure freedom from benchmark data contamination.
Good For
- Single-Turn Conversations: Optimized specifically for direct, single-turn conversational interactions.
- Fine-tuning: Designed to be a robust and adaptable base model for further fine-tuning on specific tasks.
- Applications Requiring Long Context: Ideal for use cases where processing extensive textual information in a single interaction is crucial.
Top 3 parameter combinations used by Featherless users for this model. Click a tab to see each config.