project // research // generative audio // ml systems

Synthorea / SynthGPT_

A natural-language to u-he Diva synthesizer-preset generator. A prompt like "warm analog bass with a soft attack" is transcribed into a synth-agnostic semantic intent, mapped to an 881-dimensional Diva parameter vector by a family-conditioned mixture-of-experts, and compiled to a loadable .h2p preset. Design principle: learn taste, compile contract.

一个「自然语言 → u-he Diva 合成器预设」生成系统。诸如 "warm analog bass with a soft attack" 的提示词先被转写为与合成器无关的语义意图(semantic intent),再由「按音色族条件化的专家混合模型(MoE)」映射为 881 维 Diva 参数向量,最终编译为可直接加载的 .h2p 预设。核心理念:学习品味,编译契约

PyTorch Transformer Mixture of Experts Semantic IR DSP / u-he Diva Preference Judge
Presentation
Product Site
architecture / demo / results
Role Architect / ML Eng
Status In Development
Explore The Product Site
signal flow // prompt to preset
prompt Layer 1 · Transformer semantic IR vectorize (246-d) Layer 2 · MoE (881-d) compiler + repair .h2p

What I Built

two learned stages inside a deterministic shell

  • • A semantic-IR transformer front-end that maps free text to a validated, synth-agnostic intent, with a deterministic constraint pass that guarantees internal coherence.
  • • A family-conditioned residual mixture-of-experts (6 experts, activity-masked) that predicts the 249-field / 881-dim Diva parameter vector — the mechanism behind a measured typicality gain (family deviation 1.105 → 0.763).
  • • An attribution-aware compiler with prompt-constraint repair contracts, plus a deterministic, admin-calibrated preference judge (78% → 86.4% agreement) extended with an offline audio term via a headless Diva render harness.
  • • 语义 IR 转写前端:将自由文本映射为经校验、与合成器无关的意图,并以确定性约束层保证内部一致性。
  • • 按音色族条件化的残差专家混合模型(6 专家,activity-mask):预测 249 字段 / 881 维 Diva 参数向量——正是「音色典型度」提升(family deviation 1.105 → 0.763)背后的机制。
  • • 归因感知编译器 + 提示约束修复契约;以及经管理员校准的确定性偏好判别器(一致率 78% → 86.4%),并通过无头 Diva 渲染管线扩展出离线音频项。

Tech Stack

core

PyTorch · Transformer + MoE
layer 1 + layer 2 models
Deterministic DSP + Compiler
.h2p contract / repair
Headless Diva (pedalboard)
audio-in-the-loop judge

Explore

product site & source