huihui-ai/MicroThinker-8B-Preview

TEXT GENERATIONConcurrency Cost:1Model Size:8BQuant:FP8Ctx Length:32kPublished:Jan 12, 2025License:apache-2.0Architecture:Transformer0.0K Open Weights Cold

MicroThinker-8B-Preview is an 8 billion parameter language model developed by huihui-ai, fine-tuned from Meta-Llama-3.1-8B-Instruct-abliterated. This model focuses on enhancing AI reasoning capabilities, leveraging a 32768 token context length. It was trained using the FineQwQ-142k dataset with a 4-bit quantization, demonstrating good performance despite being a test fine-tune. The model is optimized for reasoning tasks, as indicated by its training on specific problem-solving examples.

Loading preview...

MicroThinker-8B-Preview Overview

MicroThinker-8B-Preview is an 8 billion parameter language model developed by huihui-ai, built upon the huihui-ai/Meta-Llama-3.1-8B-Instruct-abliterated base model. This iteration aims to advance AI reasoning capabilities, with the 8B version noted to outperform its 3B and 1B counterparts.

Key Characteristics & Training

  • Base Model: Fine-tuned from Meta-Llama-3.1-8B-Instruct-abliterated.
  • Parameter Count: 8 billion parameters.
  • Context Length: Supports a maximum context length of 21710 tokens during fine-tuning.
  • Training Data: Utilized 142k samples from the FineQwQ-142k dataset for supervised fine-tuning (SFT).
  • Quantization: Trained with 4-bit quantization (quant_bits 4) and bfloat16 compute/storage dtypes.
  • Hardware: Fine-tuned on a single RTX 4090 GPU (24GB).
  • Reasoning Focus: The training process included examples like counting characters, lily pad growth problems, and handshake calculations, suggesting an emphasis on logical and mathematical reasoning.

Usage

This model can be easily integrated and run using Ollama:

ollama run huihui_ai/microthinker:8b

Potential Use Cases

  • Reasoning Tasks: Suitable for applications requiring step-by-step logical deduction and problem-solving.
  • Instruction Following: Benefits from its instruction-tuned base, making it responsive to specific prompts.
  • Experimental AI Development: Given its nature as a "test" fine-tune with promising results, it's a good candidate for further experimentation and development in reasoning-focused LLM applications.

Finetunes
1 models