TAUR-dev/rankalign-v6-gemma-2-2b-it-d0.15-e2-hc-b2d-dbl-all-p0-nv1-ng1-fsx-sm0.1
TAUR-dev/rankalign-v6-gemma-2-2b-it-d0.15-e2-hc-b2d-dbl-all-p0-nv1-ng1-fsx-sm0.1 is a 2.6 billion parameter instruction-tuned model fine-tuned from Google's Gemma-2-2b-it base model. Developed as part of the rankalign project, this model is specifically optimized for hypernym generation tasks, focusing on identifying broader categories for given concepts. It leverages a unique training methodology involving delta 0.15 and a NLL validator weight of 1, making it suitable for research into semantic hierarchy and linguistic relationships.
Loading preview...
Model Overview
This model, rankalign-v6-gemma-2-2b-it-d0.15-e2-hc-b2d-dbl-all-p0-nv1-ng1-fsx-sm0.1, is a fine-tuned checkpoint derived from Google's gemma-2-2b-it base model. It is part of the rankalign project, which focuses on advanced alignment techniques for language models.
Key Training Details
The model underwent specific fine-tuning for a task identified as hypernym-concat-bananas-to-dogs-double-all. Notable training parameters include:
- Base Model:
google/gemma-2-2b-it - Version: v6
- Epochs: 2
- Delta: 0.15
- Preference Loss Weight: 0
- NLL Validator Weight: 1
- NLL Generator Weight: 1
- Semi-supervised Ratio: 0.1
This configuration suggests a strong emphasis on the NLL validator during training, indicating an optimization for generating semantically correct and coherent outputs, particularly within the domain of hypernym identification. The force-same-x parameter also points to a controlled training environment to ensure consistency.
Use Cases
This model is particularly well-suited for:
- Hypernym Generation: Identifying broader categories or superordinate concepts for given terms.
- Semantic Relationship Research: Exploring and evaluating hierarchical relationships between words.
- Linguistic Analysis: Applications requiring precise understanding and generation of semantic hierarchies.
Developers can reproduce evaluations using provided Python scripts targeting various hypernym tasks such as hypernym-bananas, hypernym-dogs, and hypernym-elephants, among others.