ChatMusician: Music-Integrated LLM

ChatMusician is an open-source 7 billion parameter LLM, built upon LLaMA2, that uniquely integrates intrinsic musical abilities. Unlike other models, it processes and generates music using a pure text tokenizer and ABC notation, eliminating the need for external multi-modal neural structures. This approach allows it to understand and generate music intrinsically, treating it as a second language.

Key Capabilities

• Music Generation: Composes well-structured, full-length music conditioned on various inputs like text, chords, melodies, motifs, and musical forms.
• Music Understanding: Demonstrates strong performance on college-level music understanding tasks, outperforming LLaMA2 and GPT-3.5 on the custom MusicTheoryBench in zero-shot settings.
• Language Preservation: Endowing musical abilities does not degrade general language abilities; it even shows a slight improvement in MMLU scores.

Good For

• Music Composition: Generating musical pieces from textual descriptions, chord progressions, or melodic structures.
• Music Analysis: Investigating and analyzing musical works to understand structural organization or consistent melodic elements.
• Research & Development: Exploring the integration of musical intelligence into large language models and advancing the field of AI music generation.