Ghost 7B v0.9.1 Overview

Ghost 7B v0.9.1 is an early alpha release of the Ghost 7B model, developed by Lam Hieu. This model is a 7 billion parameter language model built on the Mistral 7B architecture, with a primary focus on enhancing reasoning capabilities and multi-task knowledge. A key differentiator is its optimization for multilingual comprehension, particularly Vietnamese, achieved through QLoRA fine-tuning on a dataset that is approximately 70% Vietnamese and 30% English.

Key Capabilities & Features

• Multilingual Proficiency: Demonstrates strong comprehension and generation in both Vietnamese and English, including handling Vietnamese with missing accents, abbreviations, or slang.
• Reasoning & Multi-task Knowledge: Designed with an emphasis on improving reasoning abilities and general multi-task knowledge.
• Efficient Training: Utilizes QLoRA for training, allowing for effective fine-tuning with a relatively small dataset (estimated 150MB) and leveraging Unsloth's features for cost-efficiency.
• Quantized Versions Available: Provided in GGUF and AWQ formats for reduced resource requirements.

Performance & Evaluation

On the Open LLM Leaderboard, Ghost 7B v0.9.1 achieves an average score of 55.10, with notable scores in HellaSwag (77.03) and Winogrande (72.53). Crucially, it ranks 3rd on the VMLU (Vietnamese Multitask Language Understanding) Leaderboard for fine-tuned models, highlighting its strong performance in Vietnamese language understanding and reasoning tasks.

Use Cases

This model is suitable for creating AI assistants that require robust multilingual capabilities, especially for applications involving Vietnamese and English. It is designed to be adaptable for various tasks, with the developer suggesting it can replace applications for certain functions, similar to how one might use ChatGPT for specific tasks.