HaimingW/qwen2.5-1.5b-faithful-summarization
HaimingW/qwen2.5-1.5b-faithful-summarization is a 1.5 billion parameter causal language model, fine-tuned from Qwen/Qwen2.5-1.5B. Developed by HaimingW, this model specializes in faithful document summarization, achieving a combined faithfulness and coverage score of 0.476 on a held-out evaluation set. It is optimized for generating accurate and concise summaries from various document types, leveraging a 32768 token context length.
Loading preview...
Model Overview
HaimingW/qwen2.5-1.5b-faithful-summarization is a 1.5 billion parameter causal language model, fine-tuned by HaimingW from the Qwen/Qwen2.5-1.5B base model. Its primary purpose is to generate faithful document summaries.
Key Capabilities
- Faithful Summarization: Specifically trained to produce summaries that accurately reflect the source document's content, minimizing hallucination.
- Decontaminated Training Data: Fine-tuned on public summarization datasets (XSum, CNN/DailyMail, BillSum) that were decontaminated against evaluation sets to ensure robust performance.
- Efficient Fine-tuning: Utilizes LoRA SFT with specific hyperparameters (r=32, alpha=64) for efficient adaptation.
- Context Length: Supports a maximum sequence length of 2048 tokens during training, suitable for summarizing moderately long documents.
Performance Highlights
Evaluated on 360 held-out summarization items, the model achieved:
- Faithfulness Score: 0.547
- Coverage Score: 0.422
- Combined Score: 0.476 (exceeding a target of 0.45)
- Degenerate Fraction: 11.9%
When to Use This Model
This model is ideal for applications requiring:
- Generating concise and accurate summaries of English documents.
- Tasks where the fidelity of the summary to the original text is critical.
- Deployment in environments where a smaller, specialized model is preferred over larger, general-purpose LLMs for summarization tasks.