Intellecta is a 1 billion parameter LLaMA 3.2-based causal language model developed by kssrikar4, fine-tuned for general-purpose conversational AI and instruction-following tasks. It leverages a transformer architecture and is optimized for diverse applications including chatbots, virtual assistants, and research in specialized domains like healthcare. The model's training incorporates a variety of instruction and conversational datasets, enhancing its ability to answer questions, summarize text, and generate responses.
Loading preview...
Intellecta: A Fine-Tuned LLaMA 3.2-1B Model
Intellecta is a 1 billion parameter model built upon the meta-llama/Llama-3.2-1B architecture, fine-tuned by kssrikar4. This transformer-based causal language model is designed for natural language understanding and generation, with a specific focus on conversational AI and instruction-following capabilities.
Key Capabilities
- Instruction Following: Excels at tasks such as answering questions, summarizing text, and generating content based on given instructions.
- Conversational AI: Suitable for developing chatbots and virtual assistants, including those for specialized fields like healthcare or education.
- Research and Development: Provides a robust base for further fine-tuning and benchmarking on various downstream tasks.
Training Details
The model was fine-tuned on a diverse set of datasets, including fka/awesome-chatgpt-prompts, BAAI/Infinity-Instruct (3M), allenai/WildChat-1M, lavita/ChatDoctor-HealthCareMagic-100k, zjunlp/Mol-Instructions, and garage-bAInd/Open-Platypus. The training process involved tokenization with padding and truncation, using a maximum input length of 512 tokens. It utilized a learning rate of 0.0001, a batch size of 4 with 4 gradient accumulation steps, and was trained for 4 epochs using mixed-precision (FP16) for efficiency. The fine-tuned model is pushed to the Hugging Face Hub for easy access and deployment.