cookinai/CatMacaroni14
CatMacaroni14 is a 7 billion parameter language model created by cookinai, developed through a slerp merge of shadowml/Marcoro14-7B-slerp and rishiraj/CatPPT. This model integrates components from EmbeddedLLM/Mistral-7B-Merge-14-v0.1 via Marcoro14-7B-slerp, offering a unique blend of capabilities from its constituent models. It is designed for general language tasks, leveraging its merged architecture to provide a distinct performance profile.
Loading preview...
Overview
CatMacaroni14 is a 7 billion parameter language model developed by cookinai. It is the result of a slerp merge operation combining two base models: shadowml/Marcoro14-7B-slerp and rishiraj/CatPPT. The Marcoro14-7B-slerp component itself incorporates elements from EmbeddedLLM/Mistral-7B-Merge-14-v0.1, suggesting a lineage that includes Mistral-based architectures.
Merge Details
The model was created using a specific mergekit configuration, applying a slerp merge method. The merge parameters involved differential weighting for self_attn and mlp layers, with a fallback value for other tensors, indicating a fine-tuned approach to combining the strengths of its parent models. The merge was performed with bfloat16 precision.
Considerations
The creator notes potential contamination concerns regarding AIDC-ai-business/Marcoroni-7B-v3, a model that influenced one of CatMacaroni14's base components. Users are encouraged to report any findings to the HuggingFaceH4/open_llm_leaderboard to maintain data integrity.