CAS-SIAT-XinHai/MeasHalu-3B-Instruct
MeasHalu-3B-Instruct by CAS-SIAT-XinHai is a 3.1 billion parameter instruction-tuned language model, fine-tuned on a reasoning dataset. This model demonstrates a low loss of 0.0904 on its evaluation set, indicating its potential for tasks requiring logical inference and problem-solving. It is designed for applications where precise reasoning capabilities are crucial, distinguishing it from general-purpose LLMs.
Loading preview...
Model Overview
MeasHalu-3B-Instruct is a 3.1 billion parameter instruction-tuned language model developed by CAS-SIAT-XinHai. It has been fine-tuned on a specific reasoning_test dataset, aiming to enhance its capabilities in logical inference and problem-solving tasks. The model achieved a final validation loss of 0.0904 during its training, with the lowest training loss reaching 0.0.
Training Details
The model was trained using the following key hyperparameters:
- Learning Rate: 1e-05
- Batch Size: 1 (train), 1 (eval)
- Gradient Accumulation Steps: 16, resulting in a total train batch size of 32
- Optimizer: AdamW with default betas and epsilon
- LR Scheduler: Cosine type with a warmup ratio of 0.1
- Epochs: 25.0
Training was conducted on a multi-GPU setup (2 devices) and utilized Transformers 4.51.3, Pytorch 2.6.0+cu124, Datasets 3.2.0, and Tokenizers 0.21.0.
Potential Use Cases
Given its fine-tuning on a reasoning dataset and low loss, MeasHalu-3B-Instruct is likely suitable for:
- Tasks requiring logical deduction.
- Problem-solving scenarios.
- Applications where precise, instruction-following reasoning is critical.