FallenMerick/Space-Whale-Lite-13B

TEXT GENERATIONPricing:Input $1.5 / Output $2.1Concurrent Unit Cost:1Model Size:13BQuant:FP8Context Size:4kPublished:May 16, 2024License:otherArchitecture:Transformer0.0K Featherless Exclusive Cold

FallenMerick/Space-Whale-Lite-13B is a 13 billion parameter language model created by FallenMerick, built by merging components of TeeZee/Orca-2-13b_flat and KoboldAI/LLaMA2-13B-Psyfighter2. This model is a restack of the Psyonic Cetacean, utilizing a passthrough merge method to combine the strengths of its constituent models. It is designed for general language generation tasks, leveraging its merged architecture for potentially enhanced performance in areas where its base models excel. With a context length of 4096 tokens, it offers a balanced solution for various natural language processing applications.

Loading preview...

Space-Whale-Lite-13B Overview

Space-Whale-Lite-13B is a 13 billion parameter language model developed by FallenMerick, created through a sophisticated merging process. It is specifically designed as a "restack" of the well-known jebcarter/psyonic-cetacean-20B model, aiming to provide similar capabilities in a more compact form.

Merge Details

This model was constructed using the passthrough merge method via mergekit. The architecture combines layers from two distinct base models:

  • TeeZee/Orca-2-13b_flat: Integrated across specific layer ranges (0-8 and 24-32).
  • KoboldAI/LLaMA2-13B-Psyfighter2: Incorporated in other layer ranges (8-24 and 32-40).

This strategic merging of components from different models is intended to leverage their individual strengths, potentially leading to a versatile model for various language tasks. GGUF quantized versions are also available from FaradayDotDev/Space-Whale-Lite-13B-GGUF.

Potential Use Cases

Given its merged heritage from models like Orca-2 and LLaMA2-Psyfighter2, Space-Whale-Lite-13B is likely suitable for:

  • General text generation: Creating coherent and contextually relevant text.
  • Instruction following: Responding to prompts and instructions effectively.
  • Creative writing and roleplay: Benefiting from the characteristics of its base models that are often used in these domains.

Developers seeking a 13B parameter model with a unique merged architecture, particularly those familiar with the Psyonic Cetacean family, may find this model a compelling option.

Finetunes
2 models