TAUR-dev/rankalign-v6-gemma-2-2b-d0.15-e2-hc-b2d-dbl-all-tco-ln-p0-nv1-ng1-fsx
The TAUR-dev/rankalign-v6-gemma-2-2b-d0.15-e2-hc-b2d-dbl-all-tco-ln-p0-nv1-ng1-fsx model is a 2.6 billion parameter Gemma-2-2b based language model fine-tuned as part of the rankalign project. This model is specifically optimized for hypernym-related tasks, focusing on identifying and generating hierarchical relationships between concepts. It utilizes a unique training methodology involving typicality correction and length normalization, making it suitable for research and applications requiring precise semantic hierarchy understanding.
Loading preview...
Model Overview
This model, rankalign-v6-gemma-2-2b-d0.15-e2-hc-b2d-dbl-all-tco-ln-p0-nv1-ng1-fsx, is a fine-tuned checkpoint derived from the google/gemma-2-2b base model, developed within the rankalign project. It features 2.6 billion parameters and has been trained for 2 epochs with a delta of 0.15.
Key Training Details
- Base Model:
google/gemma-2-2b - Task Focus:
hypernym-concat-bananas-to-dogs-double-all, indicating a specialization in hypernym identification and generation across various categories. - Optimization: Incorporates online typicality correction and length normalization during training.
- Loss Weights: Utilizes a preference loss weight of 0, with NLL validator and generator weights set to 1.
Use Cases and Evaluation
This model is primarily designed for research and development in semantic hierarchy and hypernym tasks. The README provides extensive evaluation scripts for various hypernym-related tasks (e.g., hypernym-bananas, hypernym-dogs, hypernym-cars), demonstrating its intended application in discerning and validating hierarchical relationships between terms. It is particularly useful for experiments involving zero-shot generation and few-shot discrimination in hypernym contexts.