TAUR-dev/rankalign-v6-gemma-2-2b-d0.15-e2-hc-b2d-dbl-all-tco-ln-p0-nv1-ng1-vlo-fsx
TAUR-dev/rankalign-v6-gemma-2-2b-d0.15-e2-hc-b2d-dbl-all-tco-ln-p0-nv1-ng1-vlo-fsx is a 2.6 billion parameter Gemma-2-2b based model, fine-tuned as part of the rankalign project. This model is specifically optimized for hypernym generation tasks, focusing on identifying broader categories for given concepts. It utilizes a unique training approach involving typicality correction and length normalization, making it suitable for research and applications requiring precise hierarchical semantic understanding.
Loading preview...
Model Overview
This model, TAUR-dev/rankalign-v6-gemma-2-2b-d0.15-e2-hc-b2d-dbl-all-tco-ln-p0-nv1-ng1-vlo-fsx, is a fine-tuned checkpoint derived from the google/gemma-2-2b base model, developed within the rankalign project. It features 2.6 billion parameters and is specifically trained for hypernym generation tasks, aiming to identify superordinate concepts.
Key Training Details
The model underwent a specialized training regimen (v6, epoch 2) with a delta of 0.15. Notable training parameters include:
- Task:
hypernym-concat-bananas-to-dogs-double-all - Typicality correction: Online
- Length normalization: Enabled
- Preference loss weight: 0
- NLL validator/generator weight: 1
- Validator log-odds: Enabled
Use Cases and Evaluation
This model is particularly suited for research and applications involving semantic hierarchy and concept generalization. The README provides detailed evaluation scripts for various hypernym tasks (e.g., hypernym-bananas, hypernym-dogs, hypernym-cars), demonstrating its intended use in assessing its ability to correctly identify hypernyms with zero-shot generation and few-shot discrimination.