EssentiaTD: real-time audio analysis inside TouchDesigner

TouchDesigner keeps its audio analysis in the box now. Five new CHOP plugins from Darien Brito wrap the Essentia library, the audio-analysis toolkit that grew out of the Music Technology Group at Pompeu Fabra University in Barcelona. No external software. Raw audio in, feature tables out, at frame rate.

The suite ships five operators. Essentia Spectral reads MFCCs, spectral centroid, flux, rolloff, contrast and complexity. Essentia Tonal reports pitch, key and scale, and there is a confession buried in the docs: a coarse FFT can settle on the wrong key with total confidence, so the operator sizes its own window from the sample rate. Essentia Rhythm watches for onsets and tracks BPM. Essentia Loudness meters to EBU R128, and Essentia Spectrum passes you raw FFT magnitude and phase for GLSL or resynthesis.

Version 2.0 landed on 16 September as a deliberate break. Every analyzer now takes raw audio and runs its own FFT, so no shared spectrum node forces one window size on every algorithm, and the difference is measurable. Feed Rhythm a 4096-point spectrum instead of 1024 and onset detection drops from 0.47 to 0.13 on a 263-onset reference track. Each operator gets the window its algorithm wants.

Darien is a creative coder and audiovisual artist working out of The Hague. The build decisions read like someone who has run these networks in a show: mono by design, with clear instructions for stereo sources; installers that skip the macOS quarantine flag; a warning when the analysis window falls below the per-cook audio chunk. Everything lives in https://github.com/DarienBrito/EssentiaTD, packages are on https://github.com/DarienBrito/EssentiaTD/releases, and the interactive guide at https://darienbrito.github.io/EssentiaTD walks every parameter with examples.

What matters is what the suite removes. Real-time pitch, key and onset tracking used to mean a second application, a network round trip, or a fragile external process. Now it is a CHOP you wire like any other, and a sketch can grow into a show without changing tools. The forge here turns sound into numbers at frame rate.

■ Filed from The Foundry: the Fern, Fire wing of the city.