RNeNcodec

Realtime interactive synthesis with trained audio models

Small model training and generation using Encodec. Plays in real time in Python on a cpu with interactive parameteric control (RT for the web coming soon).

Pouring and "unpouring" water in a metal cup

Synthesis driven by parameters only

Demo visualization 2
The conditioning parameter is the "fill level"

Driving Engine sounds from RPM and Torque

Original

Demo visualization 094

Resynthesis driven by parameters only

Demo visualization 094
The conditioning parameters are RPM and Torque.

Driving Engine sounds from RPM and Torque

Original

Demo visualization 094

Resynthesis driven by parameters only

Demo visualization 094
The conditioning parameters are RPM and Torque. An interesting thing to note here is that RNN predicts Encodec token stacks at 75 frames/s, and the "idling" at the end of this sound has a period of something like 6 cycles/s. Notice how the reconstructed matches the original coding, and also that there are no phase discontinuites between frames.

And some real-time interaction

(The RNeNcodec model is running on an Intel CPU: i7-14700K)
Sound classes are selected by "1 Hot" parameters, and controled by param (real)