yellowtown/Japanese-Qwen2.5-14B-Instruct-V1
yellowtown/Japanese-Qwen2.5-14B-Instruct-V1 is a 14.8 billion parameter instruction-tuned causal language model, fine-tuned from Qwen2.5-14B-Instruct. This model is optimized for instruction following and general language tasks, leveraging a 32768 token context length. Its architecture is based on the Qwen2.5 series, making it suitable for applications requiring robust natural language understanding and generation.
Loading preview...
Overview
yellowtown/Japanese-Qwen2.5-14B-Instruct-V1 is a 14.8 billion parameter instruction-tuned language model, building upon the Qwen2.5-14B-Instruct base. This model is designed for general-purpose instruction following and conversational AI applications, benefiting from a substantial 32768 token context window.
Key Capabilities
- Instruction Following: Optimized to accurately interpret and execute user instructions.
- General Language Understanding: Capable of various natural language processing tasks.
- Extended Context: Supports processing and generating text over long input sequences due to its 32768 token context length.
Training Details
The model was fine-tuned using a learning rate of 5e-06, a total batch size of 64 (achieved with gradient accumulation steps of 16 on 4 GPUs), and trained for 1 epoch. The optimizer used was ADAMW_TORCH with a cosine learning rate scheduler and a warmup ratio of 0.1.
Good For
- Applications requiring a robust instruction-tuned model with a large context window.
- General conversational agents and chatbots.
- Tasks benefiting from strong language understanding and generation capabilities.