MayaPH/GodziLLa2-70B
MayaPH/GodziLLa2-70B is a 69 billion parameter LLaMA 2-based experimental instruction-following language model developed by Maya Philippines. It combines proprietary LoRAs with the Guanaco LLaMA 2 1K dataset. This model is designed to stress test the limitations of composite LLMs and has demonstrated strong performance on the Open LLM Leaderboard, notably outperforming GPT-3.5 and GPT-4 on TruthfulQA.
Loading preview...
GodziLLa2-70B: An Experimental LLaMA 2-Based Model
GodziLLa2-70B, developed by Maya Philippines, is a 69 billion parameter experimental language model built upon LLaMA 2. It integrates proprietary LoRAs from Maya Philippines with the Guanaco LLaMA 2 1K dataset. The primary objective of this model is to evaluate the performance and limitations of composite, instruction-following LLMs.
Key Performance Highlights
GodziLLa2-70B has achieved notable rankings on the Open LLM Leaderboard, debuting at #4 in August 2023 and reaching #2 in November 2023. Key benchmark results (Fall 2023 update) include:
- MMLU (5-shot): 69.88
- ARC (25-shot): 71.42
- HellaSwag (10-shot): 87.53
- TruthfulQA (0-shot): 61.54
- Winogrande (5-shot): 83.19
- GSM8K (5-shot): 43.21
Notably, GodziLLa2-70B surpasses GPT-3.5 (ChatGPT) in average performance and the HellaSwag benchmark, and outperforms both GPT-3.5 and GPT-4 on the TruthfulQA benchmark. It also performs comparably to GPT-3.5 on MMLU.
Technical Considerations
Users should be aware that the model's default precision is fp32, requiring approximately 275 GB of VRAM/RAM. Lower precision options (fp16, int8, int4) are recommended to conserve memory. Quantized versions (GPTQ and GGML) are available from TheBloke for further memory optimization.
Ethical Considerations
As an experimental model, users are advised to exercise caution regarding privacy, potential biases from training data, and the presence of uncensored content, including NSFW material. Guardrails are encouraged for safe and aligned usage.