TheBloke/Tulu-13B-SuperHOT-8K-fp16: Extended Context LLM

This model is a 13 billion parameter LLaMa-based language model, developed by TheBloke, which combines the instruction-tuned capabilities of Allen AI's Tulu 13B with the extended context handling of Kaio Ken's SuperHOT 8K. The primary differentiator of this model is its ability to process inputs up to an 8192-token context length, significantly more than the base Tulu 13B's original 4096 tokens.

Key Capabilities & Features

• Extended Context Window: Achieves an 8K (8192 token) context length through the integration of Kaio Ken's SuperHOT 8K LoRA and specific configuration settings (trust_remote_code=True).
• Instruction Following: Inherits instruction-tuning from Allen AI's Tulu 13B, which was fine-tuned on a diverse mixture of datasets including FLAN V2, CoT, Dolly, Open Assistant 1, GPT4-Alpaca, Code-Alpaca, and ShareGPT.
• LLaMa Architecture: Built upon the LLaMa model architecture, providing a robust foundation for language understanding and generation.
• Flexible Deployment: Available in fp16 PyTorch format, suitable for GPU inference and as a base for further quantization or conversion.

When to Use This Model

• Long-form Content Processing: Ideal for applications requiring the model to understand and generate text based on extensive input, such as summarizing long documents, extended conversations, or detailed code analysis.
• Instruction-tuned Tasks: Benefits from the Tulu 13B's instruction-tuning, making it effective for various prompt-based tasks like question answering, creative writing, and code generation.
• Research and Development: The fp16 format makes it a good candidate for researchers and developers looking to experiment with extended context models or perform further fine-tuning and quantization.