AbeerMostafa/Novelty_Reviewer
AbeerMostafa/Novelty_Reviewer is an 8 billion parameter instruction-tuned causal language model, fine-tuned from Meta Llama 3.1-8B-Instruct. This model is specifically trained on a novelty dataset, leveraging a 32,120 token context length and advanced Liger optimizations for enhanced performance. It is designed for tasks related to novelty review and analysis.
Loading preview...
Overview
AbeerMostafa/Novelty_Reviewer is an 8 billion parameter language model, fine-tuned from the powerful meta-llama/Llama-3.1-8B-Instruct base model. This model has been specialized through training on a unique dataset, Dataset_construction/tokenized_novelty_dataset_5_for_llama/train_full.parquet, indicating a focus on tasks related to identifying or evaluating novelty.
Key Technical Details
- Base Model: Meta Llama 3.1-8B-Instruct
- Parameter Count: 8 billion
- Context Length: Utilizes a substantial sequence length of 32,120 tokens, allowing for processing extensive inputs.
- Training Framework: Built with Axolotl, incorporating advanced Liger optimizations such as
liger_rope,liger_rms_norm,liger_glu_activation, andliger_fused_linear_cross_entropyfor improved efficiency and performance. - Optimization: Trained with a learning rate of 2e-05 using the AdamW optimizer and a cosine learning rate scheduler.
Potential Use Cases
Given its specialized training on a "novelty dataset," this model is likely well-suited for applications requiring:
- Novelty Detection: Identifying unique or new concepts within text.
- Content Review: Assessing the originality or distinctiveness of written material.
- Research Analysis: Aiding in the review of scientific papers or patents for new contributions.
Further details on specific intended uses and limitations would require more information from the model's developers.