ajtaltarabukin2022/merged_champion_v5_m1
The ajtaltarabukin2022/merged_champion_v5_m1 is a 32 billion parameter language model created by ajtaltarabukin2022, formed by merging several pre-trained models using the DARE TIES method. This model integrates components from multiple 'affine' models, leveraging a weighted combination of their layers. It is designed for general language tasks, benefiting from the combined strengths of its constituent models.
Loading preview...
Model Overview
The ajtaltarabukin2022/merged_champion_v5_m1 is a 32 billion parameter language model developed by ajtaltarabukin2022. It was constructed using MergeKit with the DARE TIES merge method, which combines parameters from multiple pre-trained models to create a new, potentially more capable model.
Merge Details
This model is a composite of several 'affine' models, with dura-lori/affine-5ED5dwT4fztHjgjyR6vXpbGfnooeuWfr3VueaZrrfWJSou7y serving as the base. The merge incorporated contributions from:
voidai001/affine-rl0-5HeJuQB4ZcVaU8yfgwYCm3AvdiA7dPA34nvB5HwSubVoFREmchouchouM/Affine-5DhGPvYiBChDerVjSgyt1vuuwQyZWJJgsEdQHAkXRuSYji4ddura-lori/affine-5CtqFaxMkR1rZfP3cWiW6ywTszxd6dKqFoPtKdLQzMkT1kCf
The merging process involved a weighted combination of layers from these models, with specific weights assigned to each source model across layers 0 to 64. The configuration utilized bfloat16 for data types and applied a density of 0.7.
Intended Use
As a merged model, merged_champion_v5_m1 is expected to inherit and combine the general language understanding and generation capabilities of its constituent models, making it suitable for a broad range of natural language processing tasks.