TAUR-dev/rankalign-v6-gemma-2-2b-d0.15-e2-hc-b2d-dbl-all-tco-ln-nv1-ng1-fsx
TAUR-dev/rankalign-v6-gemma-2-2b-d0.15-e2-hc-b2d-dbl-all-tco-ln-nv1-ng1-fsx is a 2.6 billion parameter language model fine-tuned from Google's Gemma-2-2b base model. This model is specifically optimized for hypernym prediction tasks, trained using the rankalign project's methodology with online typicality correction and length normalization. It is designed to excel in identifying hierarchical relationships between concepts, particularly within the 'hypernym-concat-bananas-to-dogs-double-all' task. Its specialized fine-tuning makes it suitable for research and applications requiring precise hypernym generation and validation.
Loading preview...
Model Overview
This model, rankalign-v6-gemma-2-2b-d0.15-e2-hc-b2d-dbl-all-tco-ln-nv1-ng1-fsx, is a fine-tuned checkpoint derived from the google/gemma-2-2b base model, developed as part of the rankalign project. It features 2.6 billion parameters and has been trained for 2 epochs with a delta of 0.15.
Key Training Details
The model's training focused on a specific task identified as hypernym-concat-bananas-to-dogs-double-all. Notable training parameters include:
- Base Model:
google/gemma-2-2b - Version: v6
- Epochs: 2
- Delta: 0.15
- Typicality Correction: Online
- Length Normalization: Enabled
- Preference Loss Weight: 1
- NLL Validator Weight: 1
- NLL Generator Weight: 1
Intended Use Cases
This model is specifically designed for tasks involving hypernym prediction and validation. Its fine-tuning process, which includes online typicality correction and length normalization, suggests an emphasis on generating semantically accurate and contextually appropriate hypernyms. It is particularly well-suited for research and development in semantic hierarchy understanding and knowledge graph construction, especially for the specific hypernym tasks it was trained on, such as those involving 'bananas' to 'dogs' and other listed categories.