Who YuE is for#
Musicians exploring AI-assisted composition
YuE2 generates an editable melody-and-chord plan alongside audio, so musicians can review the symbolic score, modify it by hand or through an agent, and render a revised version. The agentic editing demo shows iterative changes to harmony, lyrics, tempo, and arrangement through a conversation.
Skip if:
Skip if you need a browser-based point-and-click editor. YuE2 is a Python library and command-line tool with no graphical interface.
Researchers studying open source music generation
YuE2 provides public model weights, a staged Python API, and the WildSongBench evaluation dataset for reproducing published benchmark scores. The pipeline supports swapping decoder variants (YuE2-Vae vs YuE2-Vae-legacy) and calling generation stages independently.
Skip if:
Skip if your research requires commercial licensing for model weights. CC BY-NC 4.0 prohibits commercial use of the YuE2 weights.
Developers building cover or transcription pipelines
SheetSage2 transcribes any audio recording into ABC notation with beat, chord, melody, and structure. Paired with YuE2's cover generation mode, developers can build pipelines that accept a source recording and produce a stylized version in a new genre without cover-specific fine-tuning.
Skip if:
Skip if your environment lacks an NVIDIA GPU with at least 24 GB VRAM. The pipeline does not support CPU-only or low-VRAM setups.
AI agents running agentic music editing workflows
The yue2-music skill packages the full pipeline for AI agents: generating songs, transcribing and covering recordings, editing ABC scores, and comparing versions. The skill supports multi-step editing sessions where each version is retained for review and comparison.
Skip if:
Skip if you need a stable production API. YuE2 is a research release and interfaces may change across versions.
The problem it solves#
Music generation tools from commercial platforms produce output you cannot inspect. You submit a prompt, receive audio, and have no access to the underlying composition. If the melody is wrong, the only recourse is to try again with a different prompt. This trial-and-error loop is slow, and the gap between your intent and the output widens when the composition is complex.
Running a music generation model locally has historically required stitching together separate audio AI components with no unified pipeline, and the results lagged behind commercial quality. Most available open models either lacked vocal generation, produced artifacts that required post-processing, or offered no structured intermediate that a musician or agent could inspect and modify.
How it solves it#
Editable symbolic score in ABC notation
YuE2 generates a melody-and-chord plan in ABC notation before rendering audio. You can read the plan, modify it by hand or with an agent, and supply the edited score back to produce a different recording. The `cot='full'` setting enables this by default for new songs; `cot='off'` skips the plan and generates audio directly.
Full-song generation with vocals and accompaniment
Give YuE2 lyrics and a style prompt and it outputs 48 kHz stereo audio with a lead vocal track and instrumental accompaniment. The AR-NAR Mixture-of-Transformers backbone generates semantic tokens autoregressively, then uses flow matching to produce acoustic latents that a VAE decodes into stereo audio.
Zero-shot cover generation from audio recordings
Transcribe any recording with SheetSage2 to get its melody as ABC notation, then supply new lyrics or a style prompt to generate a cover in a different arrangement. On a 948-work evaluation, full-score YuE2 reaches 0.647 CLEWS mAP for source-identity preservation, compared to 0.006 without a score.
Agentic music editing through conversation
An agent can revise the composition by modifying the ABC score, changing harmony, tempo, melody, or arrangement, then re-render the audio. The project demo follows one song through 9 editing steps and 14 versions, moving from Mandarin pop to English jazz with new harmony and a saxophone solo.
State-of-the-art benchmark scores against commercial systems
On WildSongBench (192 prompts, September 2026), YuE2 best-of-8 scores 6.9632 SongBench Avg, the highest among all 17 evaluated settings. This exceeds Suno v5 (6.8721), Suno v5.5 (6.7150), Suno v6 (6.5562), and Mureka 9 (6.9377). Standard single-pass YuE2 scores 6.7316, also above Suno v5.5 and v6.
Strengths and trade-offs#
Strengths
- Editable composition before audio renderingThe symbolic planning step exposes the melody and chord structure as ABC notation before the model produces audio. Unlike Suno or Udio, where generation is opaque, YuE2 lets you change the composition and render a different recording from the revised score. This is the key capability that no tested commercial system offers.
- Top WildSongBench score across all 17 evaluated settingsIn the September 2026 WildSongBench evaluation across 192 prompts, YuE2 best-of-8 scored 6.9632 SongBench Avg, the highest of all 17 settings tested. Standard YuE2 scored 6.7316, above Suno v5.5 (6.7150) and Suno v6 (6.5562). The model runs on publicly available weights with no subscription required.
- Apache 2.0 code with open weights on Hugging FaceYuE2's source code, agent skill, and documentation are Apache 2.0 licensed. Model weights are available on Hugging Face under CC BY-NC 4.0 and can be downloaded and run locally without an API key or subscription. The full pipeline runs on your own infrastructure with no managed service involved.
- SheetSage2 state-of-the-art audio transcription includedSheetSage2, bundled with the YuE2 repository, reaches state of the art on 10 of 13 benchmark metrics across six transcription tasks (beat, downbeat, key, chord, structure, melody). It turns any recording into editable ABC notation that feeds directly into YuE2's cover and editing pipeline.
Trade-offs
- -Requires Linux, Python 3.12, and a 24 GB VRAM NVIDIA GPUThe documented quickstart requires Linux, Python 3.12, and an NVIDIA GPU with BF16 support and at least 24 GB of VRAM. The model produces 48 kHz stereo audio without quantization. Mac, Windows, and CPU-only setups are not covered in the official setup guide, and low-VRAM GPUs are not supported.
- -Model weights are CC BY-NC 4.0, prohibiting commercial useYuE2's code is Apache 2.0 licensed, but the model weights are under CC BY-NC 4.0. You can run YuE2 locally for personal projects, research, and internal non-commercial use, but you cannot use the weights to power a commercial product or service. Organizations planning commercial deployments need to review this license before proceeding.
- -Best-of-8 quality requires 8x compute per generated songThe top benchmark score (6.9632 SongBench Avg) uses best-of-8 selection: eight full songs are generated and ranked before one is chosen. Standard single-pass YuE2 scores 6.7316, noticeably below the benchmark headline. For workflows where GPU time matters, the cost per song is 8x higher to match the reported quality.
YuE vs alternatives#
YuE2 vs Suno
Both generate songs from a text prompt, but they differ in transparency, self-hosting, and commercial licensing.
| Feature | YuE2 | Suno |
|---|---|---|
| Code license | Apache 2.0 | Proprietary |
| Model weights | CC BY-NC 4.0 (non-commercial) | Proprietary |
| Self-hosting | Yes | No |
| Editable score | Yes (ABC notation) | No |
| WildSongBench SongBench Avg | 6.9632 (best-of-8) | 6.8721 (v5) |
YuE2's editable symbolic score is the key differentiator versus Suno. When you generate a song with YuE2, you get ABC notation showing the melody and chords before the model renders audio. You can modify that score and produce a different recording from the revised plan. Suno generates audio directly with no intermediate you can read or edit. On WildSongBench, YuE2 best-of-8 outscores all four tested Suno versions on SongBench Avg.
Suno is the better choice when you need a web-based interface with no hardware requirements, when your use case requires commercial licensing, or when you want a fast hosted API with no server to manage. YuE2 requires a Linux machine with a 24 GB VRAM GPU, and its model weights prohibit commercial use.
YuE2 vs Udio
Udio is a commercial AI music generation service with a managed web interface and API. Like Suno, it generates audio directly from prompts without exposing an editable intermediate composition. YuE2 differs on the same axes: open weights with a CC BY-NC 4.0 model license, self-hosting on Linux with an NVIDIA GPU, and an editable ABC score that can be reviewed and revised before rendering.
Udio is the better choice when you need a no-setup hosted service with commercial licensing and a browser-based editor. YuE2 is the better choice when you need open weights, self-hosted infrastructure, or direct control over the composition before the model renders audio.
Quick start#
Install YuE2 from source on Linux with Python 3.12 and an NVIDIA GPU with at least 24 GB VRAM.
```bash
git clone https://github.com/multimodal-art-projection/YuE.git
cd YuE
python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install .
python examples/generate.py --output outputs/first-song
```What it's built on#
- Languages
- Python
FAQ#
Is YuE2 free to use?
YuE2's source code and agent skill are Apache 2.0 licensed and free for any use, including commercial. The model weights are separately licensed under CC BY-NC 4.0, which permits personal and research use but prohibits commercial applications. You need both code and weights to run the model, so organizations planning commercial products should review the weight license before proceeding.
What hardware does YuE2 require?
YuE2 requires Linux, Python 3.12, and an NVIDIA GPU with BF16 support and at least 24 GB of VRAM. It outputs 48 kHz stereo audio without quantization by default. Model files download from Hugging Face on first run. Mac, Windows, and CPU-only configurations are not covered in the official quickstart.
How does YuE2 score against Suno on quality benchmarks?
On WildSongBench with 192 prompts evaluated in September 2026, YuE2 best-of-8 scored 6.9632 SongBench Avg, above Suno v5 (6.8721), Suno v5.5 (6.7150), and Suno v6 (6.5562). Standard single-pass YuE2 scored 6.7316. The benchmark documentation notes that the small gap between the top means does not establish statistical significance.
Can YuE2 generate zero-shot covers of existing songs?
Yes. SheetSage2, included in the YuE2 repository, transcribes a source recording into ABC notation covering beat, chord, melody, and structure. You supply that score to YuE2 with new lyrics or a target style and it generates a cover. On a 948-work evaluation, full-score YuE2 reached 0.647 CLEWS mAP for source-identity preservation, compared to 0.006 without a score.
What is the difference between cot='full' and cot='off' in YuE2?
With cot='full' (the default for new songs), YuE2 generates an editable melody-and-chord plan in ABC notation before producing audio. You can review or modify the plan and supply it back to render a different recording. With cot='off', YuE2 generates audio directly from lyrics and style, skipping the intermediate score. The plan mode enables inspection and editing; direct generation does not.
Similar open-source tools#
God's Eye View
Real-time global intelligence on a 3D globe
paperclip
Self-hosted AI agent management with org charts and budgets
OpenCode
OpenCode is an open-source AI coding agent that assists developers in
nodus
Local-first desktop app for academic research with Zotero integration
Claude-Red
78 offensive security skills for Claude, drop-in and MIT licensed
worktrunk
Git worktree management for parallel AI agents