hemlang/Hemlock-Apothecary-7B
Hemlock-Apothecary-7B by hemlang is a 7.6 billion parameter instruction-tuned language model, fine-tuned from the Hemlock-Codex-7B base model. It was trained using a Supervised Fine-Tuning (SFT) approach with a maximum sequence length of 2048 tokens. This model is designed for general language understanding and generation tasks, leveraging its SFT training for improved conversational and instructional capabilities.
Loading preview...
Hemlock-Apothecary-7B Overview
Hemlock-Apothecary-7B is a 7.6 billion parameter language model developed by hemlang, built upon the hemlang/Hemlock-Codex-7B base model. It has undergone Supervised Fine-Tuning (SFT) to enhance its ability to follow instructions and engage in conversational tasks.
Key Training Details
The model was fine-tuned with a focus on specific parameters to optimize its performance:
- Base Model:
hemlang/Hemlock-Codex-7B - Training Mode: SFT (Supervised Fine-Tuning)
- Max Sequence Length: 2048 tokens
- Quantization: 4-bit (NF4) for efficient deployment
- LoRA Configuration: Utilized LoRA with a rank of 128 and alpha of 128, targeting key attention and feed-forward projection modules (
up_proj,down_proj,gate_proj,k_proj,q_proj,v_proj,o_proj).
This configuration suggests an emphasis on adapting the base model's capabilities to instruction-following tasks while maintaining computational efficiency.
Potential Use Cases
Given its SFT training, Hemlock-Apothecary-7B is suitable for applications requiring:
- Instruction Following: Responding to prompts and commands effectively.
- General Text Generation: Creating coherent and contextually relevant text.
- Conversational AI: Engaging in dialogue and generating human-like responses.