yamatazen/Ayla-Light-12B-v2
yamatazen/Ayla-Light-12B-v2 is a 12 billion parameter language model developed by yamatazen, created by merging several pre-trained models using the Model Stock method. This model leverages a 32768 token context length and is designed to combine the strengths of its constituent models. It is suitable for general language generation tasks, benefiting from its merged architecture.
Loading preview...
Model Overview
yamatazen/Ayla-Light-12B-v2 is a 12 billion parameter language model developed by yamatazen. This model was created through a merge of multiple pre-trained language models using the Model Stock method, with yamatazen/Ayla-Light-12B serving as the base model. The merging process utilized mergekit to combine the capabilities of its components.
Merge Details
The model integrates contributions from the following pre-trained models:
intervitens/mini-magnum-12b-v1.1NeverSleep/Lumimaid-v0.2-12Binflatebot/MN-12B-Mag-Mell-R1
This approach aims to consolidate the strengths of these individual models into a single, more versatile language model. The merge was configured to use bfloat16 data type and included normalization during the parameter combination process. With a context length of 32768 tokens, Ayla-Light-12B-v2 is designed for various language generation and understanding tasks, benefiting from its composite architecture.