rbelanec/train_cola_42_1776331560
The rbelanec/train_cola_42_1776331560 model is a 1 billion parameter language model fine-tuned by rbelanec. It is based on the meta-llama/Llama-3.2-1B-Instruct architecture and was specifically trained on the CoLA (Corpus of Linguistic Acceptability) dataset. This model is optimized for tasks related to linguistic acceptability judgment, demonstrating a validation loss of 0.1763 on the evaluation set. Its primary application is in evaluating grammatical correctness and natural language understanding within the CoLA framework.
Loading preview...
Model Overview
rbelanec/train_cola_42_1776331560 is a 1 billion parameter model, fine-tuned by rbelanec from the meta-llama/Llama-3.2-1B-Instruct base architecture. This model has been specifically trained on the CoLA (Corpus of Linguistic Acceptability) dataset.
Key Characteristics
- Base Model: Llama-3.2-1B-Instruct
- Fine-tuning Dataset: CoLA (Corpus of Linguistic Acceptability)
- Training Objective: Optimized for linguistic acceptability tasks.
- Performance: Achieved a validation loss of 0.1763 on the evaluation set during training.
- Training Details: Trained for 5 epochs with a learning rate of 5e-06, using AdamW_Torch optimizer and a cosine learning rate scheduler with a 0.1 warmup ratio.
Intended Use Cases
This model is primarily suited for tasks requiring grammaticality judgment and linguistic acceptability assessment, particularly within the context of the CoLA benchmark. It can be used for research and development in natural language understanding where evaluating sentence structure and correctness is crucial.