shleeeee/mistral-ko-OpenOrca-wiki-v1
The shleeeee/mistral-ko-OpenOrca-wiki-v1 is a fine-tuned Mistral-7B-v0.1 model developed by shleeeee (Seunghyeon Lee) and oopsung (Sungwoo Park). This model is specifically optimized for Korean language tasks, leveraging fine-tuning on 2000 ko-OpenOrca datasets. It is designed for general text generation and understanding in Korean, building upon the Mistral architecture.
Loading preview...
Model Overview
shleeeee/mistral-ko-OpenOrca-wiki-v1 is a specialized language model developed by shleeeee (Seunghyeon Lee) and oopsung (Sungwoo Park). It is a fine-tuned variant of the Mistral-7B-v0.1 architecture, specifically adapted for Korean language processing.
Key Characteristics
- Base Model: Mistral-7B-v0.1.
- Language Focus: Optimized for Korean language tasks.
- Fine-tuning Data: Trained on 2000 samples from the
ko-OpenOrcadataset. - Training Details: Fine-tuned with 2 epochs and a
train_batchsize of 4. - LoRA Target Modules:
q_proj,k_proj,v_proj,o_proj,gate_projwere targeted during LoRA fine-tuning. - Prompt Format: Utilizes the standard Mistral prompt template:
<s>[INST]{['instruction']}[/INST]{['output']}</s>.
Usage
This model can be easily integrated into applications using the Hugging Face transformers library for text generation tasks. It is suitable for various Korean natural language processing applications where a Mistral-based model with Korean language proficiency is required.
Limitations
Evaluation details are currently pending, so performance metrics and specific strengths/weaknesses are yet to be fully documented.