xero0000/Qwen3.8-27B-Palimpsest

Model Overview

xero0000/Qwen3.8-27B-Palimpsest is an experimental 27 billion parameter BF16 fine-tune of the Qwen3.8-27B base model, developed by xero0000. This model focuses on enhancing capabilities in literary prose, maintaining continuity in long-form text, structured tool use, and position-aware long-context processing. It integrates two small, architecture-aware LoRA stages into the language model, while retaining the base model's vision encoder.

Key Differentiators & Capabilities

• Literary Prose & Continuity: Fine-tuned with datasets like Dxniz/Novelist and nchapman/figaro-creative-writing to excel in generating high-quality, continuous literary prose and maintaining narrative flow over extended passages.
• Structured Tool Use: Incorporates training from microsoft/orca-agentinstruct-1M-v1 and project-generated PACT-Q cases, demonstrating strong instruction following and tool-use capabilities without strict-format regression.
• Position-Aware Long Context: Features a unique position-aware long-context repair stage, enabling effective information preservation and revision across a working context up to 262,144 native tokens. Local evaluations show improved virtual-position exact rows and token accuracy.
• Experimental Fine-tune: Represents a merged fine-tune of two LoRA stages, with the final model not requiring PEFT adapters at inference time.

Intended Use Cases

This model is designed for research and local experimentation in:

• Creative Drafting: Generating long-form creative writing, including fiction and varied voice styles.
• Continuity Editing: Assisting in maintaining coherence and consistency across extended textual outputs.
• Structured Tool Calls: Implementing and testing structured tool use within generative tasks.
• Long-Context Generalization: Exploring and developing applications that require understanding and generating content over very long contexts.