Hugging Face Overview
Poro 2 8B Instruct: Multilingual Conversational AI
Poro 2 8B Instruct is an 8 billion parameter instruction-following chatbot model, developed by LumiOpen in collaboration with AMD Silo AI, TurkuNLP, and HPLT. It is built upon the Llama 3.1 8B architecture and has been extensively fine-tuned for conversational AI applications in both Finnish and English.
Key Capabilities & Training:
- Bilingual Proficiency: Optimized for instruction following and conversations in both Finnish and English.
- Advanced Fine-tuning: Created through a multi-stage process including continued pretraining on 165B tokens (Finnish, English, code, math), Supervised Fine-Tuning (SFT) with 1.4M instruction examples, and Direct Preference Optimization (DPO) using the HelpSteer3 dataset for improved response quality.
- Performance: Achieves substantial improvements in Finnish instruction-following benchmarks (e.g., 66.54 on IFEval Finnish, 6.75 on MTBench Finnish) compared to Llama 3.1 8B Instruct, Gemma-2-9B-it, and EuroLLM-9B-Instruct, while maintaining strong English performance.
- Context Length: Supports a maximum sequence length of 8192 tokens.
Intended Use Cases:
- Conversational AI applications in Finnish and English.
- Question answering and information retrieval.
- Content generation and creative writing.
- Educational applications and customer service.
- Translation between Finnish and English.
Limitations:
- Limited proficiency in languages other than English and Finnish.
- Potential for biased, inappropriate, or factually incorrect content.
- Performance variations in specialized domains and a knowledge cutoff for recent events.