Pyroserenus/Orthrus-12b-v0.8

TEXT GENERATIONPricing:Input $1.2 / Output $4.8Concurrent Unit Cost:1Model Size:12.2BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Aug 7, 2024License:cc-by-nc-4.0Architecture:Transformer0.0K Open Weights Featherless Exclusive Cold

Pyroserenus/Orthrus-12b-v0.8 is a 12.2 billion parameter language model, created by Pyroserenus, designed as a component for larger merges but usable independently. This model was developed using the TIES merge method, based on IntervitensInc/Mistral-Nemo-Base-2407-chatml, and incorporates anthracite-org/magnum-12b-v2 and unsloth/Mistral-Nemo-Instruct-2407. It is optimized for chat-based interactions, preferring ChatML format, and is suitable for general conversational AI tasks.

Loading preview...

Orthrus-12b-v0.8 Overview

Orthrus-12b-v0.8 is a 12.2 billion parameter language model developed by Pyroserenus. While primarily intended as a foundational component for more extensive model merges, it functions effectively as a standalone model. This model was constructed using the TIES merge method, a technique that combines multiple pre-trained language models to leverage their strengths.

Key Capabilities

  • Chat-Optimized: Designed with a preference for the ChatML format, ensuring robust performance in conversational AI applications.
  • Flexible Instruction Following: While preferring ChatML, it also works well with Mistral Instruct formats and demonstrates adaptability to other instruction styles.
  • Merge Architecture: Built upon IntervitensInc/Mistral-Nemo-Base-2407-chatml as its base, integrating anthracite-org/magnum-12b-v2 and unsloth/Mistral-Nemo-Instruct-2407 to enhance its capabilities.

Good For

  • Conversational AI: Ideal for chatbots, virtual assistants, and interactive dialogue systems due to its ChatML optimization.
  • Model Experimentation: Suitable for developers exploring model merging techniques or requiring a robust base for further fine-tuning.
  • General Text Generation: Capable of various text generation tasks where a 12B parameter model with a 32K context window is appropriate.

Finetunes
5 models