ryota-komatsu/SylReg-LM-7B
TEXT GENERATIONPricing:Input $0.4 / Cached $0.08 / Output $0.8Concurrent Unit Cost:1Model Size:7.6BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Feb 2, 2026License:cc-by-nc-sa-4.0Architecture:Transformer Open Weights Featherless Exclusive Cold
SylReg-LM-7B by ryota-komatsu is a 7 billion parameter Qwen2ForCausalLM-based language model specifically designed for speech language modeling. It processes syllabic units derived from audio inputs, enabling tasks like unit-to-speech synthesis. This model is fine-tuned from Qwen/Qwen2.5-7B and trained on a diverse dataset totaling over 129,000 hours of speech and text data.
Loading preview...
SylReg-LM-7B Overview
SylReg-LM-7B is a 7 billion parameter language model developed by ryota-komatsu, built upon the Qwen2ForCausalLM architecture and fine-tuned from Qwen/Qwen2.5-7B. Its core functionality revolves around speech language modeling, where it processes and generates sequences of syllabic units derived from audio.
Key Capabilities
- Speech Language Modeling: Designed to understand and generate sequences of syllabic units, enabling advanced speech processing tasks.
- Unit-to-Speech Synthesis: Integrates with an encoder (SylReg-Distill) to convert raw audio into syllabic units and a decoder (SylReg-Decoder) to synthesize speech from these units.
- Extensive Training Data: Trained on a large corpus exceeding 129,000 hours, including datasets like LibriSpeech, Libriheavy, Emilia-Large, People's Speech, VoxPopuli, TinyStories, and Cosmopedia-v2.
- High-Performance Training: Utilized bf16 mixed precision, 15k training steps, and a batch size of 2^21 tokens on 32 NVIDIA H100 GPUs.
Good For
- Researchers and developers working on novel speech synthesis methods.
- Applications requiring language modeling directly on syllabic representations of speech.
- Experiments in speaker-disentangled chunk-wise regression for syllabic tokenization, as detailed in the associated research paper arXiv:2607.04064.