donoway/TinyStoriesV2_Llama-3.2-1B-2c9ozw1u
The donoway/TinyStoriesV2_Llama-3.2-1B-2c9ozw1u model is a 1 billion parameter language model, fine-tuned from Meta's Llama-3.2-1B architecture. This model was trained with a context length of 32768 tokens over 100 epochs. Specific details regarding its primary differentiator and intended use cases are not explicitly provided in the available documentation.
Loading preview...
Model Overview
This model, donoway/TinyStoriesV2_Llama-3.2-1B-2c9ozw1u, is a fine-tuned variant of the meta-llama/Llama-3.2-1B architecture. It features 1 billion parameters and was trained with a substantial context length of 32768 tokens.
Training Details
The model underwent 100 training epochs using a learning rate of 5e-05 and an AdamW optimizer. The training process utilized a batch size of 1, with an evaluation batch size of 112. The training was conducted using Transformers 4.51.3, PyTorch 2.6.0+cu124, Datasets 3.5.0, and Tokenizers 0.21.1.
Key Characteristics
- Base Model: Meta's Llama-3.2-1B
- Parameter Count: 1 billion
- Context Length: 32768 tokens
- Training Epochs: 100
Current Limitations
Detailed information regarding the specific dataset used for fine-tuning, the model's intended uses, and its limitations is not available in the provided documentation. Users should exercise caution and conduct further evaluation to determine its suitability for specific applications.