m-a-p/ChatMusician
ChatMusician, developed by m-a-p, is a 7 billion parameter LLaMA2-based language model designed for intrinsic musical understanding and generation. It integrates musical abilities by continual pre-training and fine-tuning on ABC notation, treating music as a second language. This model excels at composing structured, full-length music conditioned on text, chords, melodies, and forms, and demonstrates strong performance on music understanding benchmarks.
Loading preview...
ChatMusician: Music-Integrated LLM
ChatMusician is an open-source 7 billion parameter LLM, built upon LLaMA2, that uniquely integrates intrinsic musical abilities. Unlike other models, it processes and generates music using a pure text tokenizer and ABC notation, eliminating the need for external multi-modal neural structures. This approach allows it to understand and generate music intrinsically, treating it as a second language.
Key Capabilities
- Music Generation: Composes well-structured, full-length music conditioned on various inputs like text, chords, melodies, motifs, and musical forms.
- Music Understanding: Demonstrates strong performance on college-level music understanding tasks, outperforming LLaMA2 and GPT-3.5 on the custom MusicTheoryBench in zero-shot settings.
- Language Preservation: Endowing musical abilities does not degrade general language abilities; it even shows a slight improvement in MMLU scores.
Good For
- Music Composition: Generating musical pieces from textual descriptions, chord progressions, or melodic structures.
- Music Analysis: Investigating and analyzing musical works to understand structural organization or consistent melodic elements.
- Research & Development: Exploring the integration of musical intelligence into large language models and advancing the field of AI music generation.