LumiOpen/Llama-Poro-2-8B-Instruct

Poro 2 8B Instruct: A Bilingual Conversational AI

Poro 2 8B Instruct is an 8.03 billion parameter instruction-following model, a collaborative effort by AMD Silo AI, TurkuNLP, and HPLT. It is built upon the Llama 3.1 8B architecture and has undergone extensive fine-tuning, including supervised fine-tuning (SFT) and Direct Preference Optimization (DPO), to enhance its conversational abilities in both Finnish and English.

Key Capabilities & Training Highlights

• Bilingual Proficiency: Demonstrates substantial improvements in Finnish instruction-following, outperforming Llama 3.1 8B Instruct, Gemma-2-9B-it, and EuroLLM-9B-Instruct in Finnish benchmarks, while maintaining strong English performance.
• Robust Training: The base model was continuously pretrained on 165 billion tokens of Finnish, English, code, and math data. SFT involved 1.4 million instruction examples, and DPO utilized the HelpSteer3 dataset for alignment.
• Context Length: Supports a maximum sequence length of 8,192 tokens.
• Performance: Achieves a 66.54 on IFEval Finnish and 6.75 on MTBench Finnish, with an 85% win rate against Llama 3.1 8B Instruct in Finnish MTBench pairwise comparisons.

Good for

• Conversational AI applications in Finnish and English.
• Question answering and information retrieval.
• Content generation and creative writing.
• Educational applications requiring bilingual support.
• Customer service and support applications.
• Translation between Finnish and English.