nbeerbower/Qwen2.5-Gutenberg-Doppel-32B

Hugging Face
TEXT GENERATIONPricing:Input $2.72 / Output $4.8Concurrent Unit Cost:2Model Size:32.8BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Nov 17, 2024License:apache-2.0Architecture:Transformer0.0K Open Weights Featherless Exclusive Warm

The nbeerbower/Qwen2.5-Gutenberg-Doppel-32B is a 32.8 billion parameter language model based on the Qwen2.5-32B-Instruct architecture, developed by nbeerbower. It has been fine-tuned using the ORPO method on the jondurbin/gutenberg-dpo-v0.1 and nbeerbower/gutenberg2-dpo datasets, featuring a 32768 token context length. This model is optimized for tasks benefiting from its specific DPO fine-tuning on Gutenberg-derived datasets, supporting a wide range of languages including English, Chinese, French, and Spanish.

Loading preview...

Qwen2.5-Gutenberg-Doppel-32B Overview

This model is a 32.8 billion parameter language model, building upon the robust Qwen/Qwen2.5-32B-Instruct base architecture. Developed by nbeerbower, it leverages a substantial 32768 token context length, making it suitable for processing longer texts.

Key Characteristics

  • Base Model: Qwen/Qwen2.5-32B-Instruct, known for its strong general language understanding.
  • Fine-tuning Method: Utilizes the ORPO (Odds Ratio Preference Optimization) technique, trained over 1.25 epochs using 2x A100 GPUs. This method aims to align the model's outputs with human preferences more effectively.
  • Training Data: Fine-tuned on specific DPO datasets: jondurbin/gutenberg-dpo-v0.1 and nbeerbower/gutenberg2-dpo. These datasets suggest a focus on text generation and understanding derived from literary or structured text sources.
  • Multilingual Support: Inherits multilingual capabilities from its base, supporting languages such as Chinese, English, French, Spanish, Portuguese, German, Italian, Russian, Japanese, Korean, Vietnamese, Thai, and Arabic.

Potential Use Cases

  • Text Generation: Ideal for tasks requiring high-quality, coherent text generation, potentially with a stylistic leaning influenced by its Gutenberg-derived training.
  • Content Creation: Suitable for generating creative content, summaries, or expanding on existing texts.
  • Multilingual Applications: Can be applied to tasks across its supported languages, benefiting from the Qwen2.5 base's broad linguistic understanding.

Finetunes
1 models