Model Overview

AmberYifan/safe-spin-iter0 is a 7 billion parameter language model derived from the alignment-handbook/zephyr-7b-sft-full base model. It has been fine-tuned using the AmberYifan/spin_iter0 and AmberYifan/safe_spin_iter0 datasets, suggesting an emphasis on safety and alignment during its development. The model supports an 8192-token context length.

Key Training Details

• Base Model: alignment-handbook/zephyr-7b-sft-full
• Datasets: Fine-tuned on AmberYifan/spin_iter0 and AmberYifan/safe_spin_iter0.
• Epochs: Trained for 3 epochs.
• Learning Rate: 5e-07.
• Batch Size: A total training batch size of 32 was used across 4 GPUs.

Intended Use Cases

This model is suitable for applications that require a 7B parameter model with characteristics influenced by the Zephyr architecture and further refined for safety and alignment through its specific training datasets. Developers looking for a model with a foundation in instruction-following and an additional layer of safety considerations may find this model relevant.