Honkware/Manticore-13b-Landmark
Honkware/Manticore-13b-Landmark is a 13 billion parameter language model developed by Honkware, featuring Landmark Attention for enhanced context handling. This model is a blend of Manticore-13B and a QLoRA adaptation, specifically designed to leverage a large context size of approximately 18,000 tokens. It is optimized for tasks requiring extensive contextual understanding, making it suitable for applications that benefit from processing long documents or conversations.
Loading preview...
Overview
Honkware/Manticore-13b-Landmark is a 13 billion parameter language model that integrates Landmark Attention to significantly extend its effective context window. This model is a composite, built upon the base of Manticore-13B and further refined with a Manticore-13B-Landmark-QLoRA adaptation.
Key Capabilities
- Extended Context Handling: Utilizes Landmark Attention, a technique detailed in this paper, to manage a substantially larger context size, estimated around 18,000 tokens. This allows the model to process and understand much longer inputs than typical models of its size.
- Blended Architecture: Combines a robust base model with a QLoRA fine-tuning layer, indicating a focus on efficient adaptation and performance.
Good For
- Applications requiring deep understanding of long documents, articles, or extensive conversations.
- Use cases where maintaining coherence and extracting information across a vast amount of text is critical.
- Developers looking for a 13B parameter model with a significantly expanded context window for complex tasks.