Dans-Archive/Dans-QuestionableCocktail-13b is a 13 billion parameter language model developed by Dans-Archive, created by merging Llama-2-chat, Holodeck, and Chronos. This model aims to introduce novel prose and structuring capabilities while retaining the core strengths of Llama-2-chat. It is specifically designed to explore the effects of layer-wise merging on generative text characteristics, offering a unique blend for creative text generation. The model has a context length of 4096 tokens and is intended for experimental use in text generation tasks.
Loading preview...
Model Overview
Dans-Archive/Dans-QuestionableCocktail-13b is a 13 billion parameter experimental language model developed by Dans-Archive. It is constructed through a specific merge of three base models: 70% Llama-2-chat, 15% Holodeck, and 15% Chronos. The primary objective behind this merge was to infuse Llama-2-chat with more diverse prose and novel text structuring without compromising its original generative qualities.
Merge Methodology
The merging process involved first combining Holodeck and Chronos 50/50 into an intermediate model named 'HoloChronos'. Subsequently, 'HoloChronos' was merged with Llama-2-chat using a layer-specific weighting pattern: [0.3, 0.2, 0.2, 0.2, 0.3, 0.5, 0.4, 0.2, 0.3, 0.4]. This intricate merging strategy aims to selectively integrate features from the contributing models.
Recommended Usage
Users are advised to employ the Llama-2-chat prompt format when interacting with this model. However, due to its experimental nature as a merged model, performance may vary depending on the specific use case. This model is particularly suited for those interested in exploring the nuances of text generation and the impact of model merging on creative writing and prose style.