Model Overview

invisietch/L3.1-70Blivion-v0.1-rc1-70B is a 70 billion parameter release candidate model, built upon a merge of NVIDIA's Llama-3.1-Nemotron-70B-Instruct-HF and Sao10K/L3.1-70B-Euryale-v2.2. This model has undergone a subsequent QLoRA training step over two epochs using a mix of public and private datasets, primarily aimed at further decensoring the model and addressing issues arising from the initial merge. It maintains Llama 3.1's long context capabilities with a 16384 sequence length during training.

Key Capabilities & Characteristics

• Optimized for Creative Writing & Roleplay: Specifically designed to excel in generating engaging and immersive content for these applications.
• Reduced Censorship: Significantly less censored than its Nemotron 70B base, with further decensoring achieved through training.
• Llama-3 Instruct Format: Recommended prompting format for optimal performance.
• Long Context: Trained with a 16384 sequence length to preserve long context understanding.

Known Issues (Release Candidate)

• Still somewhat censored, though less than Nemotron 70B.
• May reproduce parts of the system prompt in its output.
• Can shy away from NSFL content, though this can be mitigated with system prompts.

Use Cases

This model is particularly well-suited for:

• Creative Storytelling: Generating detailed and imaginative narratives.
• Interactive Roleplay: Acting as a gamemaster or character in complex roleplay scenarios.
• Content Generation: Producing descriptive and engaging text for various creative projects.