OdiaGenAI/odiagenAI-bengali-base-model-v1
The OdiaGenAI/odiagenAI-bengali-base-model-v1 is a 7 billion parameter language model based on the Llama-7b architecture, developed by OdiaGenAI. It has been fine-tuned with a 252k Bengali instruction set, enabling strong Bengali instruction understanding and response generation capabilities. This model is specifically optimized for processing and generating text in Bengali, making it suitable for applications requiring high-quality Bengali language interaction.
Loading preview...
Model Overview
The OdiaGenAI/odiagenAI-bengali-base-model-v1 is a 7 billion parameter language model built upon the Llama-7b architecture. Developed by OdiaGenAI, this model has undergone fine-tuning using a substantial 252,000-entry Bengali instruction set. This specialized training, derived from translated open-source data, significantly enhances its ability to comprehend Bengali instructions and generate relevant responses.
Key Capabilities
- Bengali Instruction Understanding: Excels at interpreting and processing instructions provided in Bengali.
- Bengali Response Generation: Capable of producing coherent and contextually appropriate text in Bengali.
- Llama-7b Foundation: Benefits from the robust architecture of the Llama-7b base model.
Training Details
The model was fine-tuned with specific hyperparameters including a batch size of 128, a learning rate of 3e-4, and 5 epochs. It utilized a cutoff length of 256 and incorporated LoRA with an 'r' value of 16, targeting 'q_proj', 'k_proj', 'v_proj', and 'o_proj' modules. Further details on the Bengali data generation process and training can be found in the OdiaGenAI GitHub repository.
Ideal Use Cases
This model is particularly well-suited for applications requiring robust natural language processing in Bengali, such as chatbots, content generation, or translation services targeting the Bengali-speaking audience. Its specialized training makes it a strong candidate for tasks where accurate and contextually relevant Bengali output is crucial.