Vire Protocol 70B: Conversational Llama 3.1 Fine-tune
Vire Protocol 70B is a 70 billion parameter language model developed by Fireheart92, built upon the robust NousResearch/Hermes-3-Llama-3.1-70B base model. This iteration focuses on refining conversational abilities through a LoRA (Low-Rank Adaptation) fine-tuning process.
Key Characteristics
- Base Model: Derived from NousResearch/Hermes-3-Llama-3.1-70B, inheriting its foundational capabilities.
- Fine-tuning Method: Utilizes LoRA with specific parameters (rank 16, alpha 32, dropout 0.05) to efficiently adapt the base model.
- Training: Underwent 3 epochs of Supervised Fine-Tuning (SFT) with a learning rate of 1e-5 and a cosine scheduler, enhancing its dialogue generation.
- Context Length: Supports a substantial context window of 32768 tokens, beneficial for extended conversations and complex interactions.
Primary Use Case
This model is specifically designed and optimized for conversational applications. Its fine-tuning aims to improve the quality and coherence of dialogue, making it suitable for chatbots, interactive assistants, and other use cases requiring natural language interaction. Users should be aware that the model operates under the Meta Llama 3.1 Community License Agreement and its associated Acceptable Use Policy.