voidism/SelfCite-8B-CC-SFT
voidism/SelfCite-8B-CC-SFT is an 8 billion parameter instruction-tuned causal language model developed by voidism, based on Llama-3.1-8B-Instruct. This model is an intermediate checkpoint from the SelfCite project, specifically fine-tuned for context attribution in large language models. It is trained on SFT data generated with ContextCite, focusing on self-supervised alignment for improved citation capabilities. This model is designed to enhance the ability of LLMs to attribute information to its source context.
Loading preview...
SelfCite-8B-CC-SFT Overview
This model, voidism/SelfCite-8B-CC-SFT, is an 8 billion parameter instruction-tuned checkpoint derived from meta-llama/Llama-3.1-8B-Instruct. It represents an intermediate stage in the SelfCite project, which focuses on self-supervised alignment for context attribution in large language models.
Key Capabilities & Features
- Context Attribution: Specifically fine-tuned to improve the model's ability to attribute generated content to its source context.
- Self-Supervised Alignment: Utilizes a novel self-supervised approach for alignment, as detailed in the associated SelfCite paper.
- ContextCite Data Training: Trained on 11,185 supervised fine-tuning (SFT) examples generated using the ContextCite method, involving 256 ContextCite calls.
- Intermediate Checkpoint: This version is the model before SimPO preference optimization. For the final SelfCite model after SimPO, refer to
voidism/SelfCite-8B-from-CC.
Use Cases & Considerations
- Research in Attribution: Ideal for researchers exploring methods to enhance LLM citation and source attribution.
- Foundation for Further Alignment: Serves as a strong base for subsequent preference optimization or other alignment techniques aimed at improving factual grounding.
- Understanding SelfCite Methodology: Provides a practical checkpoint for those studying the SelfCite framework and its intermediate SFT stage.
This model is a crucial component for developing LLMs that can more reliably cite their information sources, addressing a significant challenge in generative AI.