aiplanet/effi-13b

aiplanet/effi-13b: Enhanced Reasoning with Chain of Thought

aiplanet/effi-13b is a 13 billion parameter causal decoder-only model developed by AI Planet, built upon the Llama-2-13b-chat-hf architecture. Its primary distinction lies in its fine-tuning on 1.8 million conversations from Chain of Thought (CoT) datasets, which significantly enhances its ability to provide rationales and reasoning for given contexts.

Key Capabilities

• Rationale Generation: Excels at providing detailed reasoning and explanations alongside its responses, a direct benefit of its CoT training.
• Chat/Instruct Model: Ready-to-use for conversational AI and instruction-following tasks.
• Llama-2 Foundation: Leverages the robust base of Llama-2-13b-chat-hf, a strong open-source foundation.

Training Details

The model was fine-tuned using the kaist-ai/CoT-Collection dataset, employing a PEFT (Parameter-Efficient Fine-Tuning) approach with QLoRA for efficient training. This involved specific hyperparameters like lora_alpha=32, r=8, and max_seq_length=2048.

Considerations for Use

• Memory Requirements: Requires substantial memory, approximately 85-100GB, for swift inference.
• Language: Primarily trained on English data, limiting its generalization to other languages.
• Bias and Limitations: Inherits biases from its large-scale web-trained corpora; guardrails and risk assessment are recommended for production use.

Should I use this for my use case?

This model is particularly well-suited for applications requiring not just answers, but also the underlying reasoning or explanation behind those answers. If your use case involves complex problem-solving, educational tools, or any scenario where understanding the 'why' is as important as the 'what', effi-13b's enhanced rationale generation makes it a strong candidate. However, for general-purpose fine-tuning, starting directly from Llama-2-13b-chat-hf might be more appropriate, as effi-13b is already an instruct-tuned model.