Model Overview

The curiousily/Llama-3.2-1B-Mental-Health-Sentiment is a 1 billion parameter instruction-tuned model from Meta's Llama 3.2 family, built on an optimized transformer architecture. It is designed for multilingual dialogue and agentic applications, having been trained on up to 9 trillion tokens of publicly available data with a knowledge cutoff of December 2023. The model incorporates knowledge distillation from larger Llama 3.1 models and uses supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) for alignment.

Key Capabilities

• Multilingual Dialogue: Optimized for conversations across officially supported languages including English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai.
• Agentic Applications: Strong performance in tasks like knowledge retrieval, summarization, and query/prompt rewriting.
• Quantization Support: Features like SpinQuant and QLoRA enable efficient deployment on constrained environments, such as mobile devices, significantly improving inference speed and reducing memory footprint.
• Specialized Fine-tuning: This specific model variant is fine-tuned for mental health sentiment analysis, making it particularly adept at understanding nuances in mental health-related text.

Good For

• Mental Health Applications: Analyzing sentiment in mental health contexts, supporting chatbots or assistants in this domain.
• Multilingual Chatbots: Developing assistant-like applications that require robust multilingual dialogue capabilities.
• Resource-Constrained Deployments: Ideal for on-device AI applications due to its 1B parameter size and optimized quantization schemes, offering a balance of performance and efficiency.