mlfoundations-dev/b2_math_fasttext_pos_numina_neg_natural_reasoning

Overview

This model, b2_math_fasttext_pos_numina_neg_natural_reasoning, is a 7.6 billion parameter language model. It is a fine-tuned variant of the Qwen/Qwen2.5-7B-Instruct base model, developed by mlfoundations-dev. The fine-tuning process specifically utilized the mlfoundations-dev/b2_math_fasttext_pos_numina_neg_natural_reasoning dataset, indicating a specialized focus on tasks related to mathematical reasoning and the interpretation of natural language with positive and negative numerical values.

Training Details

The model was trained using a learning rate of 4e-05, with a total batch size of 128 across 32 GPUs. The training involved 5 epochs, utilizing an AdamW optimizer and a cosine learning rate scheduler with a 0.1 warmup ratio. The development environment included Transformers 4.46.1, Pytorch 2.5.1, Datasets 3.0.2, and Tokenizers 0.20.3.

Key Characteristics

• Base Model: Qwen/Qwen2.5-7B-Instruct
• Parameter Count: 7.6 billion
• Context Length: 131072 tokens
• Specialization: Fine-tuned on a dataset implying a focus on mathematical reasoning and natural language processing involving numerical sentiment or context.

Intended Use Cases

While specific intended uses and limitations are not detailed in the provided model card, its fine-tuning on a specialized dataset suggests potential applications in:

• Mathematical problem-solving and reasoning.
• Analysis of text containing numerical data with positive or negative implications.
• Tasks requiring the understanding of numerical relationships within natural language.