Lambent/arsenic-nemo-unleashed-12B
Lambent/arsenic-nemo-unleashed-12B is a 12 billion parameter causal language model, DPO-tuned from MarinaraSpaghetti/NemoMix-Unleashed-12B. It is optimized for creative writing with an archaic, poetic style, influenced by Gutenberg and toxic DPO datasets. The model supports a context length of 32768 tokens and exhibits a unique quirk of occasionally slipping into Arabic during chat.
Loading preview...
Model Overview
Lambent/arsenic-nemo-unleashed-12B is a 12 billion parameter language model, fine-tuned using Direct Preference Optimization (DPO) on a blend of jondurbin/gutenberg-dpo-v0.1 and unalignment/toxic-dpo-v0.2 datasets. The tuning process aimed to imbue the model with a classic human talent and an 'edge' for writing, resulting in a distinct archaic and poetic style, particularly noticeable in generated poetry.
Key Characteristics
- Base Model: Fine-tuned from MarinaraSpaghetti/NemoMix-Unleashed-12B.
- Tuning Method: DPO-tuned using Axolotl, with specific configurations for
qloraadapter andpaged_adamw_8bitoptimizer. - Context Length: Verified to function effectively up to 32768 tokens.
- Unique Quirk: The model occasionally integrates brief Arabic phrases into its chat responses, a charming side effect of its DPO training.
- Architectural Improvements: Includes quality-of-life enhancements such as proper pad token assignment and an added chat template for improved compatibility.
Intended Use Cases
This model is particularly well-suited for:
- Creative Writing: Generating poetry, prose, or narratives with an archaic or classical literary style.
- Stylistic Exploration: Experimenting with language models that exhibit unique and distinct linguistic quirks.
- Research: Investigating the effects of DPO tuning on stylistic output and the emergence of unexpected linguistic behaviors.