TAUR-dev/rankalign-v6-gemma-2-2b-d0.15-e2-hc-b2d-dbl-all-tcs-ln-nv1-ng1-vlo-fsx-lo0.1
The TAUR-dev/rankalign-v6-gemma-2-2b-d0.15-e2-hc-b2d-dbl-all-tcs-ln-nv1-ng1-vlo-fsx-lo0.1 model is a 2.6 billion parameter Gemma-2-2b based language model fine-tuned as part of the rankalign project. This model is specifically trained for hypernym prediction tasks, focusing on identifying broader categories for given concepts. It utilizes a unique training methodology involving preference loss, NLL validation, and length normalization to enhance its performance in these specific linguistic tasks. The model is optimized for precise hierarchical classification within semantic relationships.
Loading preview...
Model Overview
TAUR-dev/rankalign-v6-gemma-2-2b-d0.15-e2-hc-b2d-dbl-all-tcs-ln-nv1-ng1-vlo-fsx-lo0.1 is a 2.6 billion parameter language model built upon the google/gemma-2-2b base architecture. It is a fine-tuned checkpoint from the rankalign project, specifically designed for hypernym prediction.
Key Training Details
This model underwent a specialized training regimen (v6, epoch 2) with a delta of 0.15, focusing on a hypernym-concat-bananas-to-dogs-double-all task. Notable training parameters include:
- Typicality correction: Self-correction mechanism.
- Length normalization: Enabled for improved output consistency.
- Preference loss weight: 1, indicating a strong emphasis on preference learning.
- NLL validator/generator weight: Both set to 1, balancing negative log-likelihood for validation and generation.
- Validator log-odds: Enabled, suggesting a focus on log-odds for validation.
- Force same-x: Enabled.
- Labeled-only ratio: 0.1, indicating a portion of the training used labeled data.
Primary Use Case
This model is specifically tailored for tasks requiring the identification of hypernyms (broader categories) for given terms. Its fine-tuning process makes it particularly suitable for semantic hierarchy understanding and classification within defined linguistic contexts, as demonstrated by its evaluation scripts for various hypernym tasks like 'hypernym-bananas', 'hypernym-dogs', and 'hypernym-cars'.