lik07/Stheno-TheSpice-v1

TEXT GENERATIONPricing:Input $0.37 / Cached $0.074 / Output $0.38Concurrent Unit Cost:1Model Size:8BQuant:FP8Context Size:8kTool Calling:SupportedPublished:Jun 16, 2024Architecture:Transformer Featherless Exclusive Cold

lik07/Stheno-TheSpice-v1 is an 8 billion parameter language model created by lik07, resulting from a DARE TIES merge of Sao10K/L3-8B-Stheno-v3.2 and cgato/L3-TheSpice-8b-v0.8.3. This model leverages the strengths of its merged components, offering a versatile foundation for various natural language processing tasks. Its 8192-token context window supports handling moderately long inputs and generating coherent, extended responses.

Loading preview...

Overview

lik07/Stheno-TheSpice-v1 is an 8 billion parameter language model developed by lik07. It was created using the mergekit tool, specifically employing the DARE TIES merge method. This technique combines the parameters of pre-trained models to potentially achieve enhanced performance or specialized capabilities.

Merge Details

This model is a composite of two distinct base models:

The DARE TIES (Disentangled Attribution Regularization for Efficient Training and Inference of Ensembles) merge method was applied, with specific density and weight parameters configured for each component model. The tokenizer source is derived from the base model, and the merge process utilized bfloat16 precision with int8_mask enabled.

Key Characteristics

  • Architecture: Merged model based on L3-8B-Stheno-v3.2 and L3-TheSpice-8b-v0.8.3.
  • Parameter Count: 8 billion parameters, offering a balance between performance and computational efficiency.
  • Context Length: Supports an 8192-token context window, suitable for processing and generating substantial text passages.

Potential Use Cases

Given its merged nature, Stheno-TheSpice-v1 is likely suitable for a range of general-purpose NLP applications where the combined strengths of its constituent models are beneficial. This could include text generation, summarization, question answering, and conversational AI, particularly in scenarios that benefit from the specific characteristics inherited from the Sao10K/L3-8B-Stheno-v3.2 and cgato/L3-TheSpice-8b-v0.8.3 models.

Finetunes
1 models