ShieldX/manovyadh-1.1B-v1-chat

Overview

ShieldX/manovyadh-1.1B-v1-chat is a 1.1 billion parameter language model developed by ShieldX, specifically fine-tuned for mental health counseling. It is based on the TinyLlama/TinyLlama-1.1B-Chat-v1.0 architecture and aims to provide empathetic and supportive responses to users discussing their feelings and challenges. The model's primary objective is to assist in coping with stress and improving mental health.

Key Capabilities

• Mental Health Counseling: Fine-tuned on a mental health counseling dataset to offer empathetic listening and helpful suggestions.
• Empathetic Interaction: Designed to respond with compassion and support, focusing on user needs and goals.
• Base Model for Further Finetuning: Can serve as a foundational model for additional specialized fine-tuning.

Training Details

The model was fine-tuned from TinyLlama/TinyLlama-1.1B-Chat-v1.0 using the ShieldX/manovyadh-3.5k dataset. Training involved 400 steps with a learning rate of 2.5e-05, a batch size of 2 (accumulated to 8), and an Adam optimizer. The final training loss achieved was 1.8587.

Limitations and Out-of-Scope Uses

It is crucial to note that this model is not intended for production purposes or real-life health applications. It should not be used to generate text for research or academic purposes related to health. Users should be aware of potential biases and fairness issues inherent in language models, which may include disturbing and harmful stereotypes.

When to Use This Model

• As a base model for further experimental fine-tuning in mental health domains.
• For developing demo web or mobile applications to showcase AI-powered empathetic responses.
• For educational or exploratory purposes to understand the application of LLMs in mental health support.