Gigi-Llama3-8B-Chinese-zh Overview

Gigi-Llama3-8B-Chinese-zh is the inaugural model in the Gigi series, developed by yaojialzc. It is an 8 billion parameter model based on the Llama-3-8B-Instruct architecture, specifically fine-tuned to significantly enhance its Chinese language capabilities while maintaining strong English performance.

Key Capabilities & Training

• Bilingual Proficiency: Fine-tuned on over 1.3 million high-quality Chinese-English bilingual data points, enabling robust performance in mixed-language contexts.
• Diverse Training Data: Incorporates a wide range of datasets including OpenHermes-2.5 (over 1 million GPT-4 generated instruction-tuning data), glaive-function-calling, refgpt_fact_v2, and extensive Chinese datasets like COIG-CQIA and alpaca-gpt4-data-zh, some translated using GPT-3.5.
• Instruction Following: Designed to handle various downstream tasks effectively, leveraging its instruction-tuned base and specialized fine-tuning.
• Llama-3 Compatibility: Adheres to the Llama-3-8B-Instruct dialogue template, using <|end_of_text|> as the pad token.

Performance Metrics

Evaluations on the Open LLM Leaderboard show competitive performance:

• Average Score: 67.04
• MMLU (5-Shot): 66.91
• HellaSwag (10-Shot): 80.28
• GSM8k (5-Shot): 66.79

Use Cases

This model is particularly well-suited for applications requiring high-quality Chinese-English bilingual processing and instruction following, making it a strong candidate for tasks in mixed-language environments.