Alignment-Lab-AI/Mistral-nemo-3b-unhealed

TEXT GENERATIONPricing:Input $0.87 / Cached $0.2 / Output $0.99Concurrent Unit Cost:1Model Size:12BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Jan 25, 2025Architecture:Transformer0.0K Featherless Exclusive Cold

Alignment-Lab-AI/Mistral-nemo-3b-unhealed is a 12 billion parameter language model created by Alignment-Lab-AI, merged using the passthrough method from ArliAI/Mistral-Nemo-12B-ArliAI-RPMax-v1.3 and mistralai/Mistral-Nemo-Base-2407. This model leverages specific layer ranges from its base models to combine their characteristics. It is designed for general language generation tasks, offering a blend of capabilities from its constituent Mistral-Nemo architectures.

Loading preview...

Model Overview

Alignment-Lab-AI/Mistral-nemo-3b-unhealed is a 12 billion parameter language model developed by Alignment-Lab-AI. It was constructed using the passthrough merge method via mergekit, combining layers from two distinct Mistral-Nemo-based models.

Merge Details

This model integrates specific layer ranges from:

  • ArliAI/Mistral-Nemo-12B-ArliAI-RPMax-v1.3: Contributes the initial 13 layers (0-12).
  • mistralai/Mistral-Nemo-Base-2407: Provides layers 18-19 and 32-39, intended to enhance output stability.

This selective merging approach aims to combine the strengths of both base models into a single, cohesive unit. The configuration specifies bfloat16 as the data type for the merged model.

Intended Use Cases

  • General Text Generation: Suitable for a wide array of language generation tasks, benefiting from the combined characteristics of its base models.
  • Experimentation with Merged Architectures: Developers interested in exploring models created through advanced merging techniques may find this model particularly useful for understanding how different layer contributions influence overall performance and behavior.

Finetunes
1 models