voidism/SelfCite-8B
voidism/SelfCite-8B is an 8 billion parameter language model fine-tuned for self-supervised alignment in context attribution, building upon the LongCite-8B architecture. This model specializes in enabling large language models to accurately attribute information to its source within long contexts. With a 32768 token context length, it is designed to improve the reliability and verifiability of LLM outputs by enhancing citation capabilities.
Loading preview...
SelfCite-8B: Self-Supervised Alignment for Context Attribution
SelfCite-8B is an 8 billion parameter language model developed by voidism, based on research from Massachusetts Institute of Technology and Meta AI. It is a reproduction of the SelfCite 8B SimPO fine-tuned model, initialized from the LongCite-8B architecture and trained using official scripts from the SelfCite repository.
Key Capabilities
- Enhanced Context Attribution: SelfCite-8B is specifically fine-tuned to improve the ability of large language models to attribute generated content back to its source within provided contexts.
- Self-Supervised Alignment: The model leverages a self-supervised alignment approach, as detailed in its accompanying paper, to achieve more reliable context attribution.
- Long Context Processing: With a 32768 token context length, it is well-suited for tasks requiring the processing and referencing of information from extensive documents or conversations.
- Improved Verifiability: By focusing on fine-grained citation generation, SelfCite-8B aims to make LLM outputs more verifiable and trustworthy.
Good For
- Applications requiring accurate source attribution in generated text.
- Question-answering systems that need to cite specific passages from long documents.
- Research tools where verifying information origin is crucial.
- Developing LLM-powered agents that can provide evidence for their claims.