Intellecta: A Fine-Tuned LLaMA 3.2-1B Model

Intellecta is a 1 billion parameter model built upon the meta-llama/Llama-3.2-1B architecture, fine-tuned by kssrikar4. This transformer-based causal language model is designed for natural language understanding and generation, with a specific focus on conversational AI and instruction-following capabilities.

Key Capabilities

• Instruction Following: Excels at tasks such as answering questions, summarizing text, and generating content based on given instructions.
• Conversational AI: Suitable for developing chatbots and virtual assistants, including those for specialized fields like healthcare or education.
• Research and Development: Provides a robust base for further fine-tuning and benchmarking on various downstream tasks.

Training Details

The model was fine-tuned on a diverse set of datasets, including fka/awesome-chatgpt-prompts, BAAI/Infinity-Instruct (3M), allenai/WildChat-1M, lavita/ChatDoctor-HealthCareMagic-100k, zjunlp/Mol-Instructions, and garage-bAInd/Open-Platypus. The training process involved tokenization with padding and truncation, using a maximum input length of 512 tokens. It utilized a learning rate of 0.0001, a batch size of 4 with 4 gradient accumulation steps, and was trained for 4 epochs using mixed-precision (FP16) for efficiency. The fine-tuned model is pushed to the Hugging Face Hub for easy access and deployment.