DBCMLAB/Llama-3-instruction-constructionsafety-layertuning

Model Overview

DBCMLAB/Llama-3-instruction-constructionsafety-layertuning is an 8 billion parameter instruction-tuned model, building upon the beomi/Llama-3-KoEn-8B-Instruct-preview base. This model has been specifically fine-tuned to act as a friendly and detailed expert in construction safety, primarily for Korean-language interactions.

Key Capabilities

• Specialized Knowledge: Excels in providing comprehensive answers related to construction safety guidelines.
• Korean Language Proficiency: Optimized for detailed and friendly responses in Korean, leveraging training data from the Korea Occupational Safety and Health Agency (KOSHA).
• Instruction Following: Designed to follow instructions for generating informative and helpful safety advice.

Training Details

The model was fine-tuned using approximately 11,000 QA datasets and raw data pertaining to construction safety. The training involved full parameter tuning on 2xA100 GPUs (80GB). A unique aspect of its training involved replacing layers 0, 30, and 31 with parameters from the base model after initial fine-tuning, as a precautionary measure against errors from raw data training.

Use Cases

This model is ideal for applications requiring an AI assistant or expert system focused on providing accurate and detailed information on construction safety in Korean. It can be used for educational purposes, safety consultations, or as a knowledge base for construction professionals.