MerziaAdamjee/codellama2-finetuned-spiderdata
MerziaAdamjee/codellama2-finetuned-spiderdata is a fine-tuned CodeLlama-7b-Instruct-hf model, developed by MerziaAdamjee. This model is based on the CodeLlama architecture and has been fine-tuned on the None dataset. It is designed for specific tasks related to the fine-tuning data, offering specialized performance for its intended applications.
Loading preview...
Overview
MerziaAdamjee/codellama2-finetuned-spiderdata is a specialized language model derived from the codellama/CodeLlama-7b-Instruct-hf base model. It has undergone a fine-tuning process on the None dataset, indicating an optimization for tasks relevant to that specific data distribution. The model leverages the robust CodeLlama architecture, known for its capabilities in code-related tasks.
Training Details
The fine-tuning process involved a learning rate of 0.0002, with a train_batch_size and eval_batch_size of 8. A gradient accumulation of 4 steps resulted in a total training batch size of 32. The Adam optimizer with default betas and epsilon was used, alongside a cosine learning rate scheduler over 1 epoch. The training was conducted using Transformers 4.34.0.dev0, Pytorch 2.0.1+cu118, Datasets 2.14.5, and Tokenizers 0.14.0.
Intended Use
While specific intended uses and limitations require more information, the fine-tuning on the None dataset suggests its application in domains where that data is relevant. Users should consider the nature of the fine-tuning data when evaluating its suitability for their specific use cases.