ryota-komatsu/SylReg-LM-7B
SylReg-LM-7B by ryota-komatsu is a 7 billion parameter Qwen2ForCausalLM model, fine-tuned from Qwen/Qwen2.5-7B, designed for speech language modeling. It processes syllabic units derived from audio inputs to generate speech, making it suitable for unit-to-speech synthesis tasks. The model leverages a large dataset totaling over 129,000 hours of speech and text data for its training.
Loading preview...
SylReg-LM-7B: Speech Language Model
SylReg-LM-7B is a 7 billion parameter Qwen2ForCausalLM, developed by ryota-komatsu, specifically designed for speech language modeling. It is fine-tuned from the Qwen/Qwen2.5-7B base model and operates by processing syllabic units to generate speech.
Key Capabilities
- Speech Language Modeling: Takes encoded syllabic units as input and generates sequences of units for speech synthesis.
- Unit-to-Speech Synthesis: Integrates with an encoder (SylReg-Distill) to convert raw audio into syllabic units and a decoder (SylReg-Decoder) to synthesize speech from generated units.
- English Language Support: Primarily focused on English language processing.
Training Details
The model was trained on a diverse and extensive dataset totaling 129,364 hours, including prominent speech datasets like LibriSpeech, Libriheavy, Emilia-Large, People's Speech, and VoxPopuli, alongside text datasets such as TinyStories and Cosmopedia-v2. Training involved 15,000 steps using bf16 mixed precision on 32 NVIDIA H100 GPUs.
Good For
- Researchers and developers working on advanced speech synthesis, particularly unit-based approaches.
- Applications requiring the generation of speech from discrete syllabic representations.
- Experiments in speech language modeling and understanding the relationship between syllabic units and generated speech.