Llama 3.1 Future Code Ja: Code-Optimized Multilingual LLM
future-architect/Llama-3.1-Future-Code-Ja-8B is an 8 billion parameter model based on Meta Llama 3.1, developed by Future Corporation. It underwent continual pre-training on a diverse dataset comprising 204.9B code tokens (from The Stack V2) and 85.7B mostly-Japanese natural language tokens (from LLM-jp Corpus v3). The model was further fine-tuned using SFT and DPO with custom instruction data, enhancing its ability to follow general task instructions.
Key Capabilities
- Multilingual Support: Officially supports Japanese and English for natural language tasks.
- Extensive Code Support: Proficient in over 40 programming languages, from Python and Java to COBOL.
- Fill-in-the-Middle (FIM): Supports bidirectional context infilling, crucial for IDE-like code completion.
- Performance: Outperforms the original Llama 3.1 in both Japanese and English code completion across various languages. It also surpasses Qwen families in Japanese generation tasks, balancing code specialization with general Japanese language ability.
Good for
- Code Generation and Completion: Excels in generating and completing code snippets in a wide array of programming languages.
- Japanese and English Development Tasks: Ideal for software development tasks requiring proficiency in either Japanese or English.
- IDE Integration: Its FIM capability makes it suitable for integration into development environments for intelligent code suggestions.
- Balanced Performance: Offers a strong balance between specialized code-related tasks and general Japanese language understanding.