YuE

数据集与模型

multimodal-art-projection/YuE

前沿开源音乐生成模型,支持符号化规划与智能体编辑。

项目简介

YuE2 是开源音乐生成基础模型,可根据歌词与风格提示先生成可编辑的旋律与和弦乐谱,再合成为带人声的 48kHz 立体声完整歌曲。支持零样本翻唱、白盒乐谱编辑以及对话式智能体音乐修改,生成质量可与主流商业系统媲美。

README 预览

> Looking for the original YuE? Its code, documentation, and license are preserved on the **[YuE-v1 branch](https://github.com/multimodal-art-projection/YuE/tree/YuE-v1)**.\n\n\n  \n\n\n\n  \n    \n    \n  \n\n\nYuE2: Unifying Symbolic and Audio Music Generation at Frontier Quality\n\nCompose in symbols. Create in sound.\n\n\n  🎧 Demos ·\n  🤗 YuE2 ·\n  🚀 Quick start ·\n  🤖 Agent skill ·\n  📊 Benchmarks ·\n  🤗 MERT2 ·\n  🤗 SheetSage2 ·\n  🤗 WSB ·\n  📦 Release ·\n  \n\n\n**YuE2 brings frontier song quality to music generation with an editable composition.** Give it lyrics and a style prompt: it writes a melody-and-chord plan, then realizes that plan as a complete song with vocals and accompaniment.\n\n- **Frontier quality.** YuE2 is competitive with Suno v5/v6 on WildSongBench. YuE2 (best-of-8) achieves **6.9632 SongBench Avg**, the highest observed mean among all evaluated settings.\n- **White-box music generation through symbolic planning.** Read, play, and change the composition before rendering it. Melody and chords become explicit controls that a person or an agent can inspect and edit.\n- **Zero-shot covers and agentic editing.** Reimagine a transcribed song in a new style, or refine a song through a conversation about its score, arrangement, and lyrics—all with the same generation checkpoint.\n\n[](https://map-yue2.github.io/#model-overview)\n\n*192 WildSongBench prompts. Both YuE2 settings use symbolic planning. Bo8 = best-of-8. The axes are normalized comparison indices; bubble area represents AudioBox production quality. [Scores and evaluation protocol](docs/benchmarks.md). [Vector PDF](assets/frontier-teaser.pdf) · [SVG](assets/frontier-teaser.svg).*\n\n## Hear what you can make\n\n| Create | Cover | Edit with an agent |\n|---|---|---|\n| Lyrics + style → score → full song | Source recording → melody score → a new interpretation | Musical feedback → score, style, or lyric revisions → a new recording |\n| [Listen and inspect the score](https://map-y

常见问题 (1)

故障排除
如何修复 ComfyUI YuE(smthemex 节点)中的 "UnboundLocalError: cannot access local variable raw_output" 错误?

这是 ComfyUI_YuE 包中的一个逻辑 bug:当 Stage 1 生成产生零个片段时(歌词过短、某些 prompt 组合、guidance 被禁用),给 raw_output 赋值的循环永远不会执行,导致该变量在 yue_node.py 中的 ids = raw_output[0].cpu().numpy() 处未定义。

修复 1 — 修补 custom_nodes/ComfyUI_YuE/yue_node.py:在 sampler_main 内部,于生成循环之前初始化该变量:

raw_output = None

并在 ids = raw_output[0]... 这一行之前添加:
if raw_output is None:
    raise RuntimeError("Stage1 produced no output tokens (raw_output is None)")

或者初始化为 raw_output = torch.empty((1, 0), device=device, dtype=torch.long)。这样对于有效的生成,Stage A 即可正常完成。

修复 2 — 如果你接下来遇到 ImportError: TorchCodec is required for load_with_torchcodec:torchaudio ≥ 2.9 默认使用 TorchCodec 来处理 torchaudio.load()。请在与 ComfyUI 相同的 venv 中安装它:

pip install torchcodec

已在 Linux 上验证可用,环境为 ComfyUI 0.9.2、Python 3.12、torch 2.9.1+cu128、torchaudio 2.9.1+cu128、torchcodec 0.9.1。应用这两个修复后,完整的 YuE 流程可以正常运行,音频生成正常。

来源 Issue #148