chargoddard/prometheus-2-llama-3-8b
chargoddard/prometheus-2-llama-3-8b is an 8 billion parameter language model, a replication of prometheus-7b-v2.0, built upon the Llama 3 8B Instruct base model with an 8192 token context length. It is specifically designed for fine-grained evaluation of other language models, achieved by training on preference and feedback datasets and linearly merging two models. This model excels at assessing and providing feedback on LLM outputs, making it suitable for automated evaluation tasks.
Loading preview...
prometheus-2-llama-3-8b Overview
This model is a replication of prometheus-7b-v2.0, utilizing the Llama 3 8B Instruct as its foundational architecture. It is specifically engineered to perform fine-grained evaluation of other language models, a capability derived from its unique training methodology.
Key Capabilities
- Automated LLM Evaluation: Specialized in assessing the quality and nuances of outputs generated by other language models.
- Feedback Generation: Trained on preference and feedback datasets to provide detailed insights.
- Llama 3 Instruct Compatibility: Uses the standard Llama 3 Instruct prompt format for seamless integration.
Training Methodology
The model was developed by training two distinct models on separate preference and feedback datasets, which were then linearly merged at equal weight. The training involved 1 epoch with a learning rate of 1e-5, an effective batch size of 128, and cosine annealing with approximately 5% warmup. This approach, detailed in the original Prometheus papers, aims to induce robust evaluation capabilities.
Good For
- Developers and researchers needing an automated system for evaluating LLM performance.
- Applications requiring fine-grained feedback on language model outputs.
- Benchmarking and quality assurance pipelines for generative AI models.
Top 3 parameter combinations used by Featherless users for this model. Click a tab to see each config.