xinyifang/ProductsLlama
ProductsLlama is an 8 billion parameter language model developed by xinyifang, fine-tuned from Llama3.1_8b_Ins_Arxiv_4096 with a 32768-token context length. This model is specifically optimized for node classification on text-attributed graphs, demonstrating strong performance in categorizing product descriptions and similar textual data within graph structures. It leverages a multi-profiling framework for data augmentation, enabling efficient processing of complex graphs with limited computational resources.
Loading preview...
Overview
ProductsLlama is an 8 billion parameter language model developed by xinyifang, specifically designed for node classification on text-attributed graphs (TAGs). Fine-tuned from Llama3.1_8b_Ins_Arxiv_4096, this model addresses challenges like input window size limitations and computational overhead in large-scale graph processing.
Key Capabilities
- Optimized for Text-Attributed Graphs: Excels at classifying nodes within graphs where nodes have associated textual information, such as product descriptions.
- Multi-Profiling Framework: Utilizes a novel data augmentation method involving multiple profiling/summarizing models to increase training data diversity and quantity.
- Efficient Graph Processing: Designed to process complex graphs efficiently, even with limited computational resources, by constructing concise yet informative fine-tuning prompts.
- Strong Performance: Outperforms 11 state-of-the-art baselines, achieving 74.31% accuracy on
ogbn-arxivand 85.15% accuracy onogbn-productsdatasets. - Fast Training: The model was trained 2x faster using Unsloth and Hugging Face's TRL library.
Good For
- Product Categorization: Accurately classifying product descriptions into predefined categories.
- Node Classification: Performing classification tasks on text-attributed graphs in domains like social networks and recommender systems.
- Research in Graph Neural Networks: Serving as a strong baseline or component for further research in LLM applications for graph data.
This model is based on the paper "LLM Profiling and Fine-Tuning with Limited Neighbor Information for Node Classification on Text-Attributed Graphs" (IEEE Xplore).