ryan0712/llama-3-8b-slow-DUS-method-1

TEXT GENERATIONPricing:Input $0.431 / Cached $0.0862 / Output $1.12Concurrent Unit Cost:1Model Size:15BQuant:FP8Context Size:8kTool Calling:SupportedPublished:May 23, 2024License:llama3Architecture:Transformer Featherless Exclusive Cold

ryan0712/llama-3-8b-slow-DUS-method-1 is a 15 billion parameter merged language model, combining components from ryan0712/llama-3-8b-DUS-initialized and ryan0712/llama-3-8b-slow-DUS-layer-SLERP using a passthrough merge method. This model leverages specific layer ranges from its base models to create a distinct configuration. It is designed for general text generation tasks, offering a unique blend of its constituent models' characteristics.

Loading preview...

Model Overview

ryan0712/llama-3-8b-slow-DUS-method-1 is a 15 billion parameter language model created through a merging process using LazyMergekit. It integrates specific layer ranges from two distinct base models:

  • ryan0712/llama-3-8b-DUS-initialized
  • ryan0712/llama-3-8b-slow-DUS-layer-SLERP

This model employs a "passthrough" merge method, combining layers from the source models in a structured manner. The configuration specifies that layers 0-21 and 21-48 are sourced from llama-3-8b-DUS-initialized, while layers 0-1 are taken from llama-3-8b-slow-DUS-layer-SLERP.

Key Characteristics

  • Architecture: Merged model derived from Llama-3-8B variants.
  • Parameter Count: 15 billion parameters.
  • Merge Method: Utilizes a passthrough merge strategy, allowing for precise control over layer integration.
  • Data Type: Configured to use bfloat16 for efficient computation.

Intended Use Cases

This model is suitable for general text generation tasks, leveraging the combined strengths of its constituent models. Developers can use it for various applications requiring a capable language model, such as:

  • Content creation
  • Chatbot development
  • Text summarization
  • Code generation (if base models have such capabilities)

Finetunes
1 models