Remek/Llama-3-8B-Omnibus-1-PL-v01-INSTRUCT
Remek/Llama-3-8B-Omnibus-1-PL-v01-INSTRUCT is an 8 billion parameter instruction-tuned model based on Meta's Llama-3-8B, specifically fine-tuned for fluent Polish language generation. Developed by Remek, this model utilizes the synthetic Omnibus-1-PL dataset, focusing on Polish facts, people, and places. It excels at conversational tasks and instruction following in Polish, addressing the base Llama-3-8B's English preference.
Loading preview...
Remek/Llama-3-8B-Omnibus-1-PL-v01-INSTRUCT Overview
This model is an instruction-tuned variant of Meta's Llama-3-8B, developed by Remek, with a primary focus on enhancing its proficiency in the Polish language. The base Llama-3-8B model, while powerful, tends to favor English, and this fine-tuning effort aims to make it a fluent Polish conversationalist.
Key Capabilities
- Polish Language Fluency: Significantly improved ability to converse and follow instructions in Polish, addressing the base model's English bias.
- Instruction Following: Trained on the synthetic Omnibus-1-PL dataset, which includes a diverse range of Polish instructions covering mathematics, writing, dialogues, medical topics, logical puzzles, and translations.
- Uncensored: The model is explicitly noted as uncensored, offering a less restricted conversational experience.
- Experimental Base: Serves as a foundation for ongoing experimentation with various finetuning techniques (LoRA, QLoRA; DPO, ORPO) and dataset iterations.
Training Details
The model was fine-tuned using QLoRA (4-bit, rank 64, alpha 128) on a single Nvidia A6000 Ada GPU for 1 epoch. The Omnibus-1-PL dataset, comprising 75,000 synthetically generated Polish instructions, was central to its training. The development utilized the Unsloth tool for efficient finetuning.
Good For
- Polish-centric AI applications: Ideal for use cases requiring natural and accurate Polish language generation and understanding.
- Research and experimentation: A valuable tool for researchers exploring finetuning techniques and dataset impact on LLM performance in Polish.
- Unrestricted conversational AI: Suitable for applications where an uncensored model is desired for broader interaction.
Limitations
- Dataset Quality: The Omnibus-1-PL dataset is acknowledged as not yet ideal and is under continuous improvement.
- Licensing: The model's use is non-commercial due to the synthetic dataset generation using GPT-3.5 and GPT-4, in addition to adhering to the Llama 3 license.
Top 3 parameter combinations used by Featherless users for this model. Click a tab to see each config.