mlfoundations-dev/stackexchange_vegetarianism

Overview

The mlfoundations-dev/stackexchange_vegetarianism model is an 8 billion parameter language model, fine-tuned from the meta-llama/Meta-Llama-3.1-8B base model. This specialization focuses its capabilities on content derived from the StackExchange Vegetarianism dataset.

Key Characteristics

• Base Model: Fine-tuned from meta-llama/Meta-Llama-3.1-8B.
• Parameter Count: 8 billion parameters.
• Context Length: Supports a context length of 32768 tokens.
• Training Data: Specialized on the mlfoundations-dev/stackexchange_vegetarianism dataset.
• Performance: Achieved a validation loss of 1.0397 on the evaluation set after 3 epochs of training.

Training Details

The model was trained with a learning rate of 5e-06, using a total batch size of 512 across 8 GPUs. The optimizer used was AdamW with default betas and epsilon, and a constant learning rate scheduler. The training process involved 3 epochs.

Intended Use Cases

This model is best suited for applications requiring deep understanding or generation of text related to vegetarianism, veganism, plant-based diets, and associated discussions found on platforms like StackExchange. Its fine-tuning on a specific domain dataset makes it particularly effective for tasks such as:

• Answering questions about vegetarianism.
• Summarizing discussions on plant-based topics.
• Generating content relevant to vegetarian lifestyles and diets.