Overview

This model, codellama2-finetuned-sqldata, is a specialized version of the codellama/CodeLlama-7b-Instruct-hf base model. It has undergone a fine-tuning process, though the specific dataset used for this training is currently unknown.

Training Details

The fine-tuning procedure involved 100 training steps with a learning rate of 0.0002. Key hyperparameters included a train_batch_size of 8, eval_batch_size of 8, and a gradient_accumulation_steps of 4, resulting in a total_train_batch_size of 32. The optimizer used was Adam with default betas and epsilon, and a cosine learning rate scheduler was employed. The training was conducted using Transformers 4.34.0.dev0, Pytorch 2.0.1+cu118, Datasets 2.14.5, and Tokenizers 0.14.0.

Intended Uses & Limitations

Due to the lack of detailed information regarding the training data and specific fine-tuning objectives, the precise intended uses and potential limitations of this model are not yet defined. Users should exercise caution and conduct further evaluation to determine its suitability for specific tasks, particularly those involving SQL data, given the model's name.