Run this model on Google Colab for free:

## Discord-Micae-Hermes-3-3B ![Micae](Micae-Thumb.png) ## Model Description Discord-Micae-Hermes-3-3B is a new finetune on [NousResearch/Hermes-3-Llama-3.2-3B](https://huggingface.co/NousResearch/Hermes-3-Llama-3.2-3B). This model serves as a foundation for ongoing exploration into the capabilities of human-adjacent text generation. - **Training Schedule:** - 17M tokens of 260 thousand single-turn exchanges (STX) – 6 epochs @ 2e-5 - 5.5M tokens of 101 thousand multi-turn chains – 6 epochs @ 2e-5 - Combined dataset – 1 epoch @ 1e-5 - **Fine-Tuning Method:** LoRA merge (α = 32, r = 8, dropout = 0.1) - **LoRA target modules:** q_proj, v_proj - **Scheduler:** Cosine schedule with 220 warmup steps per phase - **Optimizer:** torch.optim.AdamW (betas = (0.9, 0.999), amsgrad = False, eps = 1e-8, weight decay = 0.01) - **Batching:** Effective size of 126 (7 batch size × 18 gradient accumulation steps) - **Timeline and Hardware:** Training took place over 17 days on a GTX 1080 (8GB). ## Dataset The model was fine-tuned on the [mookiezi/Discord-OpenMicae](https://huggingface.co/datasets/mookiezi/Discord-OpenMicae) dataset. ## Intended Use - Conversational AI research - Experimentation with dialogue agents trained on Discord data - Chatbots requiring **casual, human-like tone** ### Limitations - The model inherits potential biases from Discord-style language. - It is **not safety-aligned** for deployment without moderation. - While it does inhert knowledge from Hermes-3-3B it is not intended for factual or sensitive information retrieval. ## Prompting Micae uses the same ChatML prompt format as Hermes 3 and handles context and chat history. ``` <|im_start|>user what do you think about ai?<|im_end|> <|im_start|>assistant i'm not a fan of AI but I can understand why people are excited to use it. it's like the first time they got an electric car, or when they were able to fly in space, that excitement is real<|im_end|> <|im_start|>user where do you think it's heading?<|im_end|> <|im_start|>assistant it will be used for good and bad things just as with other techs<|im_end|> <|im_start|>user what do you do use it for personally?<|im_end|> <|im_start|>assistant nothing much yet, I hope to find some stuff later on.<|im_end|> <|im_start|>user do you like pokemon?<|im_end|> <|im_start|>assistant yes I love Pokémon.<|im_end|> <|im_start|>user what's your favorite pokemon<|im_end|> <|im_start|>assistant garchomp<|im_end|> ``` --- ## Inferfacing A python script for interfacing, optimized for casual, Discord-style conversation, is available at [https://github.com/mookiezi/interface](https://github.com/mookiezi/interface). To use this model run: ``` python interface.py -m mookiezi/Discord-Micae-Hermes-3-3B ``` ![Interface Screenshot](interface-screenshot.png) (Example interface with [mookiezi/Discord-Micae-8B-Preview](https://huggingface.co/mookiezi/Discord-Micae-8B-Preview)) --- ## License See the [Meta LLaMA 3 Community License](https://ai.meta.com/llama/license/) for details. ## How to cite: If you use this model in your work, please cite both **Discord-Micae-Hermes-3-3B** and the base model **Hermes 3**: ```bibtex @misc{discord-micae-hermes3b, title = {Discord-Micae-Hermes-3-3B}, author = {mookiezi}, year = {2025}, url={https://huggingface.co/mookiezi/Discord-Micae-Hermes-3-3B} } @misc{teknium2024hermes3technicalreport, title={Hermes 3 Technical Report}, author={Ryan Teknium and Jeffrey Quesnelle and Chen Guang}, year={2024}, eprint={2408.11857}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2408.11857} } ``` [](https://20000.online/micae) [](https://20000.online/openmicae) [](https://20000.online/discord-dialogues)