martimfasantos/tinyllama-1.1b-sum-dpo-full_LR2e-8_3epochs_old
The martimfasantos/tinyllama-1.1b-sum-dpo-full_LR2e-8_3epochs_old model is a 1.1 billion parameter language model fine-tuned from martimfasantos/tinyllama-1.1b-sum-sft-full_old. It was trained using Direct Preference Optimization (DPO) on the openai/summarize_from_feedback dataset. This model is specifically optimized for summarization tasks, demonstrating a validation loss of 0.6872 and a rewards accuracy of 0.5943.
Loading preview...
Model Overview
This model, martimfasantos/tinyllama-1.1b-sum-dpo-full_LR2e-8_3epochs_old, is a 1.1 billion parameter language model. It is a fine-tuned variant of the martimfasantos/tinyllama-1.1b-sum-sft-full_old model, specifically optimized for summarization tasks.
Key Capabilities
- Summarization: Fine-tuned using Direct Preference Optimization (DPO) on the
openai/summarize_from_feedbackdataset, indicating a focus on generating high-quality summaries. - Performance Metrics: Achieved a validation loss of 0.6872 and a rewards accuracy of 0.5943 on the evaluation set, suggesting its proficiency in summarization.
Training Details
The model was trained for 3 epochs with a learning rate of 2e-08, a batch size of 8 (total batch size of 16 with gradient accumulation), and an Adam optimizer. The training utilized a cosine learning rate scheduler with a warmup ratio of 0.1.
Good for
- Applications requiring efficient text summarization, particularly where preference-based learning is beneficial.
- Research and development in DPO techniques on smaller language models for specific tasks.