NotASI/FineTome-Llama3.2-1B-0929
NotASI/FineTome-Llama3.2-1B-0929 is a 1 billion parameter Llama 3.2 model, fine-tuned by NotASI from unsloth/Llama-3.2-1B-Instruct-bnb-4bit. This model was trained on the mlabonne/FineTome-100k dataset for 2 epochs using rslora + qlora, achieving a final training loss of 0.796700. It is optimized for general conversational tasks and follows the Llama 3.1 chat template, offering efficient inference due to Unsloth integration.
Loading preview...
Overview
NotASI/FineTome-Llama3.2-1B-0929 is a 1 billion parameter language model developed by NotASI, fine-tuned from the unsloth/Llama-3.2-1B-Instruct-bnb-4bit base model. It was trained for 2 epochs on the mlabonne/FineTome-100k dataset using rslora + qlora techniques, resulting in a final training loss of 0.796700. The model leverages Unsloth for 2x faster training and inference, making it efficient for deployment.
Key Capabilities
- Instruction Following: Adheres to the Llama 3.1 chat template for structured conversations.
- General Conversational AI: Capable of engaging in various dialogue scenarios, as demonstrated by its ability to continue sequences and introduce itself.
- Efficient Inference: Optimized for speed through Unsloth integration, suitable for applications requiring quick responses.
Performance Metrics
Evaluated on the Open LLM Leaderboard, the model achieved an average score of 9.56. Specific scores include:
- IFEval (0-Shot): 39.91
- BBH (3-Shot): 5.74
- MATH Lvl 5 (4-Shot): 1.28
- GPQA (0-shot): 3.02
- MuSR (0-shot): 2.66
- MMLU-PRO (5-shot): 4.76
When to Use This Model
This model is suitable for developers looking for a compact (1B parameter) Llama 3.2-based model that is efficient for inference. Its fine-tuning on a diverse dataset makes it a good candidate for general-purpose conversational AI, quick prototyping, and applications where resource efficiency is crucial. It's particularly useful for tasks that benefit from a model trained with Unsloth's speed optimizations.