Platyporoni-7B Overview

Platyporoni-7B is a fine-tuned language model created by marcchew, building upon the AIDC-ai-business/Marcoroni-7B base model. The specific dataset used for fine-tuning is not detailed in the available information. The model was trained for one epoch using a learning rate of 8e-06, a train_batch_size of 48, and gradient_accumulation_steps of 2, leading to a total_train_batch_size of 96. The training process utilized an Adam optimizer with specific beta and epsilon values.

Training Performance

During its single training epoch, Platyporoni-7B achieved a final validation loss of 2.7324. The training involved 256 steps, with validation loss progressively decreasing from an initial 2.8691 to its final value. Key hyperparameters included:

• Learning Rate: 8e-06
• Optimizer: Adam (betas=(0.9, 0.999), epsilon=1e-08)
• Epochs: 1
• Batch Size: 48 (train), 6 (eval)

Limitations

Detailed information regarding the model's intended uses, specific capabilities, and limitations is not provided in the current documentation. Users should exercise caution and conduct further evaluation to determine its suitability for particular applications, as the training data and specific optimizations remain unspecified.