Amritemperor123/PoC-demo
Amritemperor123/PoC-demo is an 8 billion parameter Llama-based model, finetuned by Amritemperor123 using LoRA and Unsloth on health disease and treatment datasets. This model was developed as a proof-of-concept for medical-related text generation, leveraging its 8192 token context length. It aims to explore applications in health information processing, though its stability and performance are noted as experimental.
Loading preview...
Model Overview
Amritemperor123/PoC-demo is an 8 billion parameter Llama-based model, finetuned from ruslanmv/Medical-Llama3-8B by Amritemperor123. This model was developed as a proof-of-concept, utilizing LoRA finetuning with Unsloth and Huggingface's TRL library, which enabled 2x faster training.
Finetuning Details
The model was finetuned on various health disease and treatment datasets sourced from Kaggle, comprising approximately one million lines of data. The training was conducted on Google Colab's free tier. It's important to note that the developer explicitly states this model is experimental and might not perform as expected.
Key Characteristics
- Base Model: Llama-based, finetuned from
ruslanmv/Medical-Llama3-8B. - Parameter Count: 8 billion parameters.
- Context Length: 8192 tokens.
- Training Method: LoRA finetuning using Unsloth.
- License: apache-2.0.
Important Considerations
This model is presented as a proof-of-concept and comes with a warning that it "might be corrupted or a failure, nothing risky but probably won't work." Users should approach this model with the understanding that its functionality and reliability are experimental.