kingbri/chronolima-airo-grad-l2-13B

Model Overview

The kingbri/chronolima-airo-grad-l2-13B is a 13 billion parameter language model developed by kingbri, created through a sophisticated gradient merge process. It combines three distinct models: Chronos 13b v2, Airoboros l2 13b gpt4 2.0, and LimaRP llama 2 Lora.

Unique Merging Strategy

Unlike traditional ratio merges, this model utilizes a gradient merging technique, specifically BlockMerge_Gradient by Gryphe. This method allows Airoboros to contribute its "core response" at the initial layers, with Chronos and LimaRP progressively refining the output in later layers. LimaRP was integrated at a lower weight (0.25) to subtly correct Chronos's tendencies rather than completely altering character personalities, which was a common issue with higher-weight Lora merges.

Key Characteristics & Use Cases

This model is particularly suited for applications requiring expressive and long-form conversational responses, especially in roleplay scenarios. It inherits a bias for verbose and detailed replies from Chronos and incorporates human roleplay data from LimaRP. Users should be aware that due to its training on specific datasets, it is not intended for providing factual information or advice.

Instruction Formats

Given its merged origins, the model supports multiple instruction formats, including Alpaca 2 and Airoboros styles, to facilitate interaction.