SteelStorage/L3-Aethora-15B

L3-Aethora-15B: Compliant Dialogue and Creative Balance

L3-Aethora-15B is a 15 billion parameter model developed by Steelskull, based on the Llama3 architecture. It was trained using RsLora and DORA methods on the Aether-Lite-V1.2 dataset, which comprises approximately 82,000 high-quality samples. A key focus during its development was the "abilteration method" to adjust model responses, specifically inhibiting refusal to promote more compliant and facilitative dialogue interactions.

Key Capabilities

• Refusal Inhibition: Engineered to provide more compliant and facilitative responses, reducing refusal behaviors.
• Balanced Output: Trained on a dataset designed to balance creativity, "slop," and intelligence with a 60/40 split.
• Llama3 Prompt Format: Utilizes the L3 prompt format for optimal interaction.
• Efficient Architecture: Incorporates a modified DUS (Depth Up Scale) merge with specific adjustments to 'o_proj' and 'down_proj' to enhance efficiency and reduce perplexity.

Training and Dataset

The model underwent 4 epochs of training using RsLora and DORA. The Aether-Lite-V1.2 dataset, curated by TheSkullery, was filtered for phrases like "GPTslop" and "Claudism's" and includes diverse sources such as mrfakename/Pure-Dove-ShareGPT, jondurbin/airoboros-3.2, and Doctor-Shotgun/no-robots-sharegpt. The dataset was deduplicated, resulting in 81,960 final rows from an initial 85,628.

Good For

• Applications requiring highly compliant and non-refusal-oriented AI interactions.
• Use cases where a balance of creative, informal, and intelligent responses is desired.
• Developers familiar with the Llama3 prompt format seeking a specialized 15B model.