rbelanec/train_qqp_42_1773765557
The rbelanec/train_qqp_42_1773765557 model is a 1 billion parameter language model fine-tuned by rbelanec. It is based on the meta-llama/Llama-3.2-1B-Instruct architecture and specifically optimized for tasks related to the QQP (Quora Question Pairs) dataset. This model demonstrates a validation loss of 0.1541 on the QQP evaluation set, indicating its specialization in identifying semantic similarity between questions.
Loading preview...
Model Overview
The rbelanec/train_qqp_42_1773765557 model is a specialized 1 billion parameter language model, fine-tuned by rbelanec. It is built upon the meta-llama/Llama-3.2-1B-Instruct architecture, indicating its foundation in a Llama-family instruction-tuned model.
Key Capabilities
- Question Pair Similarity: The model has been specifically fine-tuned on the QQP (Quora Question Pairs) dataset, making it adept at tasks requiring the identification of semantic equivalence between two questions.
- Performance Metrics: During training, it achieved a validation loss of 0.1541, with a total of 137,941,664 input tokens seen across 5 epochs.
Training Details
The training process utilized a learning rate of 5e-05, a batch size of 8, and the ADAMW_TORCH optimizer. A cosine learning rate scheduler with a warmup ratio of 0.1 was employed over 5 epochs. The model was developed using Transformers 4.51.3, Pytorch 2.10.0+cu128, Datasets 4.0.0, and Tokenizers 0.21.4.
Intended Use Cases
This model is primarily suited for applications requiring high accuracy in determining if two questions are semantically the same, such as:
- Duplicate Question Detection: Identifying redundant questions in forums or Q&A platforms.
- Information Retrieval: Improving search relevance by grouping similar queries.
- Customer Support: Routing similar customer inquiries to appropriate resources.