Sorihon/Reforged-Memories-12B
Sorihon/Reforged-Memories-12B is a 12 billion parameter language model created by Sorihon, built through a multi-step merge process incorporating various 12B models, including Mistral-Nemo-Instruct-2407 as a base. This model is designed to combine the strengths of its constituent models, focusing on general language generation capabilities. With a context length of 32768 tokens, it aims to provide robust performance across a range of conversational and text-based tasks.
Loading preview...
Model Overview
Sorihon/Reforged-Memories-12B is a 12 billion parameter language model developed by Sorihon, distinguished by its intricate multi-step merging process. This model integrates components from numerous other 12B models, with mistralai/Mistral-Nemo-Instruct-2407 serving as a foundational base throughout its development.
Key Characteristics
- Architecture: A merged model, combining multiple 12B parameter models to synthesize diverse capabilities.
- Parameter Count: 12 billion parameters.
- Context Length: Supports a substantial context window of 32768 tokens.
- Development Process: Built through a series of iterative merges (Steps 1-7) using methods like Nuslerp and Dare Ties, progressively refining the model's characteristics.
Intended Use Cases
This model is suitable for general-purpose language generation tasks where a blend of capabilities from various specialized models is beneficial. Its large context window makes it potentially useful for applications requiring understanding and generation over longer texts.