huihui-ai/MicroThinker-3B-Preview-v2
MicroThinker-3B-Preview-v2 by huihui-ai is a 3.2 billion parameter language model, fine-tuned from MicroThinker-3B-Preview, with a 32768-token context length. This iteration focuses on enhancing AI reasoning capabilities, demonstrating improved performance over its predecessor. It was fine-tuned using the FineQwQ-142k dataset, making it suitable for tasks requiring advanced reasoning.
Loading preview...
MicroThinker-3B-Preview-v2 Overview
huihui-ai's MicroThinker-3B-Preview-v2 is a 3.2 billion parameter language model, representing an advancement over its base model, MicroThinker-3B-Preview. This version has been specifically fine-tuned to improve AI reasoning capabilities, leveraging a substantial context length of 32768 tokens.
Key Training Details
- Base Model: Fine-tuned from
huihui-ai/MicroThinker-3B-Preview. - Dataset: Utilized 142k samples from the
FineQwQ-142kdataset. - Training Environment: Fine-tuning was performed on a single RTX 4090 GPU (24GB).
- Methodology: Employed Supervised Fine-Tuning (SFT) with LoRA (rank 8, alpha 32) for efficient adaptation.
- Quantization: Trained with 4-bit quantization, using
bfloat16for compute and storage. - Epochs: Trained for 1 epoch with a
max_lengthof 21710 tokens during fine-tuning.
Intended Use Cases
This model is designed for applications requiring enhanced reasoning, building upon the capabilities of its predecessor. Its fine-tuning process and focus on reasoning suggest suitability for tasks that benefit from step-by-step thinking and logical deduction.