nbeerbower/bruphin-lambda

TEXT GENERATIONConcurrency Cost:1Model Size:7BQuant:FP8Ctx Length:4kPublished:Mar 30, 2024License:apache-2.0Architecture:Transformer Open Weights Cold

The nbeerbower/bruphin-lambda is a 7 billion parameter language model created by nbeerbower, resulting from a SLERP merge of chihoonlee10/T3Q-Mistral-Orca-Math-DPO and nbeerbower/bruphin-kappa. This model leverages the strengths of its merged components, particularly benefiting from the math-focused DPO fine-tuning of T3Q-Mistral-Orca-Math-DPO. With a 4096-token context length, it is designed for tasks requiring robust language understanding and potentially enhanced mathematical reasoning capabilities.

Loading preview...

bruphin-lambda: A Merged Language Model

nbeerbower/bruphin-lambda is a 7 billion parameter language model developed by nbeerbower, created through a strategic merge of two pre-trained models: chihoonlee10/T3Q-Mistral-Orca-Math-DPO and nbeerbower/bruphin-kappa. This model aims to combine the distinct strengths of its constituents.

Key Characteristics

  • Merge Method: Utilizes the SLERP (Spherical Linear Interpolation) merge method, known for smoothly combining model weights.
  • Component Models: Integrates chihoonlee10/T3Q-Mistral-Orca-Math-DPO, which is fine-tuned for mathematical reasoning, and nbeerbower/bruphin-kappa.
  • Configuration: The merge process involved specific layer ranges for each component and a detailed parameter configuration for the SLERP method, focusing on self_attn and mlp layers.
  • Context Length: Supports a context window of 4096 tokens.

Potential Use Cases

Given its lineage, particularly the inclusion of a math-optimized model, bruphin-lambda is likely well-suited for:

  • Tasks requiring enhanced mathematical problem-solving.
  • General language understanding and generation where robust reasoning is beneficial.
  • Applications benefiting from a model that combines diverse pre-training characteristics.

Finetunes
2 models