Model Overview

The huihui-ai/Llama-3.2-3B-Instruct-abliterated-finetuned is a 3.2 billion parameter instruction-tuned language model, building upon the Llama-3.2-3B-Instruct-abliterated base. This fine-tuned version aims to enhance performance across various benchmarks, particularly in instruction-following and reasoning tasks. It maintains a substantial context length of 32768 tokens, making it suitable for processing longer inputs.

Key Capabilities & Performance

This model demonstrates notable improvements in specific evaluation metrics:

• IF_Eval: Achieves 77.80, surpassing both the base Llama-3.2-3B-Instruct (76.55) and Llama-3.2-3B-Instruct-abliterated (76.76).
• BBH (Big-Bench Hard): Scores 42.20, an improvement over the base models (41.81 and 41.86).
• GPQA: Reaches 28.74, outperforming the previous versions (28.39 and 28.41).

While showing gains in these areas, its performance on MMLU Pro and TruthfulQA is slightly lower than the abliterated base model. The model is designed for general instruction-following, where its enhanced reasoning on complex tasks (BBH, GPQA) and instruction adherence (IF_Eval) can be beneficial.

Usage Notes

Users can clear the conversation and retry if the initial output is not satisfactory. The evaluation script used for these benchmarks is available within the model's repository.