nbeerbower/mistral-nemo-gutades-12B
The nbeerbower/mistral-nemo-gutades-12B is a 12-billion parameter language model, fine-tuned by nbeerbower using the ORPO method on the jondurbin/gutenberg-dpo-v0.1 dataset. This model is based on nbeerbower/mistral-nemo-bophades-12B and specializes in tasks related to the Gutenberg dataset's characteristics. With a context length of 32768 tokens, it is suitable for applications requiring processing of longer texts, particularly those with a literary or classical focus.
Loading preview...
Overview
nbeerbower/mistral-nemo-gutades-12B is a 12-billion parameter language model developed by nbeerbower. It is a fine-tuned version of the nbeerbower/mistral-nemo-bophades-12B model, specifically optimized using the ORPO (Optimized Reward Prompting) method. The fine-tuning was performed over 3 epochs on the jondurbin/gutenberg-dpo-v0.1 dataset, which suggests a specialization in content derived from the Project Gutenberg corpus.
Key Capabilities
- ORPO Fine-tuning: Utilizes the ORPO method for alignment, potentially enhancing instruction following and response quality based on the DPO dataset.
- Gutenberg Dataset Specialization: Fine-tuned on a dataset derived from Project Gutenberg, indicating potential strengths in processing, generating, or understanding literary and classical texts.
- Extended Context Window: Features a context length of 32768 tokens, allowing for the handling of longer documents and more complex conversational turns.
Good for
- Literary Analysis: Tasks involving classic literature, historical texts, or content similar to the Project Gutenberg collection.
- Long-form Text Processing: Applications requiring the model to maintain coherence and context over extended passages.
- Research and Development: As a base for further fine-tuning on domain-specific literary or historical datasets.