ISTA-MLCV/Llama_3.1_8b_single_emb
ISTA-MLCV/Llama_3.1_8b_single_emb is an 8 billion parameter Llama 3.1 model, fine-tuned as a vanilla baseline for the ASIDE (Architectural Separation of Instructions and Data in Language Models) research. This model serves as an unmodified reference point for evaluating architectural modifications in LLMs, specifically focusing on instruction and data separation. With a 32768 token context length, it is designed for research and comparative analysis within the ASIDE framework.
Loading preview...
Model Overview
This model, ISTA-MLCV/Llama_3.1_8b_single_emb, is an 8 billion parameter variant of the Llama 3.1 architecture. It was fine-tuned by ISTA-MLCV as a vanilla (unmodified) baseline for the research presented in the paper "ASIDE: Architectural Separation of Instructions and Data in Language Models." The primary purpose of this model is to provide a standard, unmodified reference point for experiments and evaluations related to architectural changes in language models, particularly those exploring the separation of instructions and data.
Key Characteristics
- Base Model: Built upon the robust Llama 3.1 8B foundation.
- Research Baseline: Serves as a direct comparison for models incorporating architectural modifications, specifically within the ASIDE framework.
- Context Length: Features a substantial context window of 32768 tokens.
- Training: Fine-tuned using the same data and procedure as other ASIDE models, but without any embedding modifications.
Intended Use Cases
- Academic Research: Ideal for researchers studying LLM architectures, instruction following, and data separation.
- Comparative Analysis: Use as a baseline to measure the impact of novel architectural changes or fine-tuning strategies.
- Reproducibility: Provides a stable, unmodified reference for reproducing results from the ASIDE paper.
For detailed usage and integration into research workflows, refer to the official ASIDE Repository.