AQuarterMile/Writing-Model-Qwen-7B
AQuarterMile/Writing-Model-Qwen-7B is a 7.6 billion parameter instruction-tuned causal language model, fine-tuned from Qwen/Qwen2.5-7B-Instruct. Developed by AQuarterMile, this model is specifically optimized for writing evaluation tasks, leveraging a 12K SFT dataset. With a 32K context length, it excels at generating and assessing written content, making it suitable for applications requiring nuanced text analysis and creation.
Loading preview...
Overview
This model, AQuarterMile/Writing-Model-Qwen-7B, is a 7.6 billion parameter language model fine-tuned from the Qwen/Qwen2.5-7B-Instruct base model. Its primary specialization is in writing evaluation tasks, having been trained on a 12K Supervised Fine-Tuning (SFT) dataset. It is part of a larger ecosystem, including a dedicated Critic Model and a 32B parameter variant.
Key Capabilities
- Specialized in Writing Evaluation: Fine-tuned specifically for tasks related to assessing and generating written content.
- Instruction-Following: Built upon an instruction-tuned base model, enhancing its ability to follow complex prompts for writing tasks.
- Context Length: Supports a substantial context window of 32,768 tokens, allowing for processing and generating longer pieces of text.
Training Details
The model underwent training with a learning rate of 7e-06, a batch size of 1 per device across 32 GPUs, and a total effective batch size of 128. It utilized the AdamW optimizer with a cosine learning rate scheduler over 5 epochs. The training leveraged Transformers 4.46.1 and PyTorch 2.4.0.
Good For
- Applications requiring automated assessment or generation of written text.
- Tasks involving detailed analysis of writing quality or style.
- Use cases where a model specialized in generative writing is beneficial.