Shisa V2.1 Llama 3.2 3B Overview

Shisa V2.1 Llama 3.2 3B is a 3.2 billion parameter, bilingual Japanese and English (JA/EN) chat model developed by Shisa.AI, built on the Llama 3.2 architecture. It is part of the Shisa V2.1 series, which represents an update to the Shisa V2 family, incorporating a re-sampled dataset and various data fixes for improved performance. The model features a 32K context length and is designed to offer class-leading Japanese language capabilities at its size, alongside robust English performance.

Key Capabilities & Differentiators

• Enhanced Japanese Performance: Achieves a JA AVG score of 57.9 and an EN AVG score of 43.2 on Shisa.AI's internal multieval test battery, showing notable gains over previous Shisa V2 models.
• Reduced Cross-Lingual Token Leakage (CLTL): Demonstrates a significant 47.8x improvement in reducing CLTL compared to its base model, with leakage reduced from 11.48% to 0.24%. This addresses a critical issue for Japanese language production use cases.
• Comprehensive Evaluation: Developed using a rigorous evaluation suite including Shaberi v2.1, ELYZA Tasks 100, Japanese MT-Bench, and custom Shisa.AI benchmarks like shisa-jp-ifeval and shisa-jp-tl-bench.
• Optimized for Local/Edge Use: At 3.2 billion parameters, it is suitable for deployment in local and edge-based applications.

Good For

• Applications requiring strong, accurate Japanese language generation and understanding.
• Use cases where minimizing language confusion and ensuring pure Japanese output is critical, such as translation, customer service, or content creation.
• Developers seeking a compact yet powerful bilingual model for local or edge deployments.