Hugging Face Overview
Model Overview
invisietch/L3.1-70Blivion-v0.1-rc1-70B is a 70 billion parameter release candidate model, built upon a merge of NVIDIA's Llama-3.1-Nemotron-70B-Instruct-HF and Sao10K/L3.1-70B-Euryale-v2.2. This model has undergone a subsequent QLoRA training step over two epochs using a mix of public and private datasets, primarily aimed at further decensoring the model and addressing issues arising from the initial merge. It maintains Llama 3.1's long context capabilities with a 16384 sequence length during training.
Key Capabilities & Characteristics
- Optimized for Creative Writing & Roleplay: Specifically designed to excel in generating engaging and immersive content for these applications.
- Reduced Censorship: Significantly less censored than its Nemotron 70B base, with further decensoring achieved through training.
- Llama-3 Instruct Format: Recommended prompting format for optimal performance.
- Long Context: Trained with a 16384 sequence length to preserve long context understanding.
Known Issues (Release Candidate)
- Still somewhat censored, though less than Nemotron 70B.
- May reproduce parts of the system prompt in its output.
- Can shy away from NSFL content, though this can be mitigated with system prompts.
Use Cases
This model is particularly well-suited for:
- Creative Storytelling: Generating detailed and imaginative narratives.
- Interactive Roleplay: Acting as a gamemaster or character in complex roleplay scenarios.
- Content Generation: Producing descriptive and engaging text for various creative projects.