Supichi/BBAI_QWEEN_V000000_LUMEN_14B

TEXT GENERATIONPricing:Input $0.431 / Cached $0.0862 / Output $1.12Concurrent Unit Cost:1Model Size:14.8BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Feb 26, 2025Architecture:Transformer Featherless Exclusive Cold

Supichi/BBAI_QWEEN_V000000_LUMEN_14B is a 14.8 billion parameter language model, created by Supichi through a SLERP merge of Qwen/Qwen2.5-14B and v000000/Qwen2.5-Lumen-14B. This model leverages the Qwen2.5 architecture and supports a 32768 token context length. Its primary characteristic is being a composite model, combining features from its base components to potentially offer enhanced general-purpose language understanding and generation capabilities.

Loading preview...

Model Overview

Supichi/BBAI_QWEEN_V000000_LUMEN_14B is a 14.8 billion parameter language model developed by Supichi. This model was created using the SLERP merge method via mergekit, combining two distinct Qwen2.5-based models: Qwen/Qwen2.5-14B and v000000/Qwen2.5-Lumen-14B. The merge process involved specific layer ranges and parameter weighting for self-attention and MLP components, aiming to synthesize their respective strengths.

Key Characteristics

  • Architecture: Based on the Qwen2.5 family, known for strong general-purpose language capabilities.
  • Parameter Count: 14.8 billion parameters, offering a balance between performance and computational efficiency.
  • Context Length: Supports a substantial context window of 32768 tokens, suitable for processing longer inputs and generating coherent extended outputs.
  • Merge Method: Utilizes the SLERP (Spherical Linear Interpolation) method, which is designed to create a smooth interpolation between model weights, potentially leading to a more robust and balanced merged model.

Potential Use Cases

Given its merged nature and base models, BBAI_QWEEN_V000000_LUMEN_14B is likely suitable for a variety of general-purpose NLP tasks, including:

  • Text Generation: Creating coherent and contextually relevant text for various applications.
  • Question Answering: Responding to queries based on provided context.
  • Summarization: Condensing long documents or conversations.
  • Code Assistance: Potentially aiding in code generation or understanding, depending on the capabilities inherited from its base models.

Finetunes
1 models