MerziaAdamjee/codellama2-finetuned-sqldata
MerziaAdamjee/codellama2-finetuned-sqldata is a fine-tuned CodeLlama-7b-Instruct-hf model, developed by MerziaAdamjee. This model is based on the CodeLlama architecture and has been fine-tuned for 100 steps with a learning rate of 0.0002. Its specific capabilities and primary use case are not detailed in the provided information, as it was trained on an unknown dataset.
Loading preview...
Overview
This model, codellama2-finetuned-sqldata, is a specialized version of the codellama/CodeLlama-7b-Instruct-hf base model. It has undergone a fine-tuning process, though the specific dataset used for this training is currently unknown.
Training Details
The fine-tuning procedure involved 100 training steps with a learning rate of 0.0002. Key hyperparameters included a train_batch_size of 8, eval_batch_size of 8, and a gradient_accumulation_steps of 4, resulting in a total_train_batch_size of 32. The optimizer used was Adam with default betas and epsilon, and a cosine learning rate scheduler was employed. The training was conducted using Transformers 4.34.0.dev0, Pytorch 2.0.1+cu118, Datasets 2.14.5, and Tokenizers 0.14.0.
Intended Uses & Limitations
Due to the lack of detailed information regarding the training data and specific fine-tuning objectives, the precise intended uses and potential limitations of this model are not yet defined. Users should exercise caution and conduct further evaluation to determine its suitability for specific tasks, particularly those involving SQL data, given the model's name.