Phonsiri/Thai-Legal-Gemma-4B-CPT

Thai Legal Gemma 4B — CPT Checkpoint

This model, Thai-Legal-Gemma-4B-CPT, is a google/gemma-4-E4B base model undergoing Continued Pre-Training (CPT) on an extensive Thai legal dataset. Its primary purpose is to build a robust legal foundation model for the Thai language, focusing on understanding the structure and terminology of Thai law. It is currently an in-progress CPT checkpoint and has not yet undergone instruction fine-tuning or alignment, meaning it is not production-ready for high-accuracy applications.

Key Characteristics & Training Details

• Base Model: google/gemma-4-E4B
• Training Method: Full Continued Pre-Training (CPT) with a Next-Token Prediction objective.
• Context Length: 8,192 tokens.
• Precision: bfloat16.
• Hardware: Trained on NVIDIA H200 GPUs.
• Dataset Composition: The training data is heavily weighted towards Thai legal documents (70%), including content from the Royal Gazette (ราชกิจจานุเบกษา), ThaiLaw, and WangchanX-Legal-ThaiCCL-RAG. It also incorporates general Thai language (15%), English (10%), code (3%), and English legal texts (2%).

Limitations and Warnings

• Not Production Ready: This is an in-progress CPT checkpoint and is not yet suitable for immediate deployment in systems requiring high accuracy.
• No Legal Advice: The model is not intended to provide official legal advice and should not replace consultation with legal professionals.
• Potential for Hallucination: During this CPT phase, the model may generate legally inaccurate or erroneous text.
• No Safety Alignment: It has not undergone safety alignment processes, making it unsuitable for immediate production use.