adikuma/mumble-cleanup

Overview

adikuma/mumble-cleanup is a compact 0.5 billion parameter language model, fine-tuned from Qwen/Qwen2.5-0.5B-Instruct. Its primary function is to transform raw, unpunctuated, and disfluent speech-to-text transcripts into clean, readable text with proper capitalization and punctuation. The model was trained using LoRA on a hand-curated synthetic dataset and is designed for efficient CPU inference, with ONNX exports available, including a dynamically quantized int8 variant.

Key Capabilities

This model excels at various transcript cleanup tasks without paraphrasing or adding new content:

• Disfluency Removal: Eliminates filler words (e.g., "um", "uh", "like") and collapses word stutters (e.g., "we we" to "we").
• Punctuation & Capitalization: Restores correct punctuation and capitalization, including proper nouns.
• Correction: Handles homophone correction (e.g., "their" / "there"), apostrophe restoration (e.g., "dont" to "don't"), and false start cleanup.
• Formatting: Splits run-on sentences, formats numbers (e.g., "two thirty" to "2:30"), and recognizes enumeration cues for list formatting.

Use Cases

This model is ideal for applications requiring automated cleanup of speech-to-text output. It is particularly suited for:

• Post-processing ASR transcripts: Enhancing readability and accuracy of dictation from various sources.
• CPU-bound environments: Its ONNX export and int8 quantization make it efficient for deployment on devices with limited computational resources.

Limitations

• English only: The model is currently limited to processing English language transcripts.
• Synthetic Data Training: Performance on real-world ASR output may vary due to differences from the synthetic training data.
• Input Length: Designed for short-to-medium dictation (up to ~512 tokens); longer inputs require chunking.
• Over-correction: May occasionally over-correct genuinely intended sentence fragments.