SelfCite-8B: Self-Supervised Alignment for Context Attribution

SelfCite-8B is an 8 billion parameter language model developed by voidism, based on research from Massachusetts Institute of Technology and Meta AI. It is a reproduction of the SelfCite 8B SimPO fine-tuned model, initialized from the LongCite-8B architecture and trained using official scripts from the SelfCite repository.

Key Capabilities

• Enhanced Context Attribution: SelfCite-8B is specifically fine-tuned to improve the ability of large language models to attribute generated content back to its source within provided contexts.
• Self-Supervised Alignment: The model leverages a self-supervised alignment approach, as detailed in its accompanying paper, to achieve more reliable context attribution.
• Long Context Processing: With a 32768 token context length, it is well-suited for tasks requiring the processing and referencing of information from extensive documents or conversations.
• Improved Verifiability: By focusing on fine-grained citation generation, SelfCite-8B aims to make LLM outputs more verifiable and trustworthy.

Good For

• Applications requiring accurate source attribution in generated text.
• Question-answering systems that need to cite specific passages from long documents.
• Research tools where verifying information origin is crucial.
• Developing LLM-powered agents that can provide evidence for their claims.