Hugging Face Badger \u039b Llama 3 8B Instruct Overview
maldv/badger-lambda-llama-3-8b is an 8 billion parameter instruct model built upon the Llama 3 architecture. Its core innovation lies in its "recursive maximally pairwise disjoint normalized denoised fourier interpolation" method, which merges 20 distinct Llama 3-based models. This process involves extracting and subtracting base model layers, then recursively merging layer deltas based on cosine similarity, normalizing them, and applying a denoised Fourier transform.
Key Characteristics
- Unique Merging Technique: Utilizes a complex fourier interpolation method to combine multiple Llama 3 variants, avoiding artifacts seen in traditional merges.
- Abliteration Focus: The merging process, particularly the "abliteration" step, aims for coherent outputs, though responses might be concise or slightly rigid.
- Llama 3 Instruct Format: Designed to be used with the standard Llama 3 Instruct prompting format.
Potential Use Cases
- Instruction Following: Excels in tasks requiring adherence to specific instructions.
- Creative Writing Assistance: Can be used as a writing assistant to continue stories.
- Roleplay Scenarios: Suitable for uncensored fictional roleplay, focusing on emotional, logical, and temporal coherence, with characters taking initiative.
Performance Insights
Evaluations on the Open LLM Leaderboard show an average score of 20.76, with specific metrics including IFEval (0-Shot) at 48.61 and MMLU-PRO (5-shot) at 30.74. The model's output block is directly influenced by Llama-3-8B-Instruct-Gradient-4194k, contributing to its characteristic response style.