It’s telling that Ableton Live, an app which has centred its entire audio engine, interface and workflow around a low-latency and real-time paradigm (the clue is in the name), didn’t introduce spectral effects until version 11 of the program. The in-house effects that did emerge are wonderful and intuitive examples of spectral processing capabilities, coming at a small price in terms of latency. We get to hear from Rob Tubb, a developer at Ableton (and brilliant electronic musician going by the name of Cursor Miner), who was partly responsible for introducing these effects into Live.
What attracted you to working with spectral processing – was there a particular moment or insight that sparked your interest?
I’ve always been fascinated by the timbre of sounds and the spectrum seems like the best definition we have of that. It’s a bit like nanotechnology for sound: unpacking a sound into its ‘atoms’ and being able to build anything up from the pieces again. In practice it’s not that easy, but the dream is still alive! Even though I first heard about Fourier theory thirty years ago, to this day I find it quite magical that you can express any waveform as a sum of sinusoids. It seems impossible that the sound of, say, a bucket of marbles being tipped into a dustbin can be made by adding up the sound of thousands of tuning forks, but nevertheless it always works out.
What is your technical background and how is the relationship between your technical side and your personal creative projects?
I did physics as a first degree way back in 97, so became fairly familiar with waves and Fourier, then after 13 years of programming and making music separately I realised I could combine all these three bits of my experience by doing a Masters degree in DSP. Queen Mary University in London’s Centre for Digital Music had a DSP course specifically tailored for music and that was perfect. When I am making music I am more or less in a completely non-technical mindset but surely this background has influenced me in terms of how I process things – how I do sound design, which kit I buy, even how I think about sound. I do make some devices just for my own music, but very often it can be too much of a distraction to see all the problems with the thing you just built whilst you are trying to rock out.
How does your work as Cursor Miner influence your approach to developing audio tools?
I think if you are going to make music tech it does really help to have a long and varied musical background. You develop an intuition for what musicians want and what kind of things feel inspiring. I think that finding the balance, of when control is important and when unexpected things are welcome, you can only get through experience. Because Cursor Miner and the bands I’ve been in are all quite diverse I like to think I understand not only what a techno musician wants but also the priorities of a guitar player or someone trying to produce a band.
What were the main objectives when developing the spectral audio effects in Ableton Live and what were the key challenges?
Our overall goal was to take some things that had appeared in more ‘obscure’ and maybe difficult to use plugins and make them more accessible and immediate. The biggest challenge I find with STFT (short time Fourier transform) based effects is to avoid what I would call ‘spectral mush’. If you just process spectra without care everything ends up sounding like a badly compressed MP3 from the late 90s. The sinusoidal grains no longer add up nicely and you can hear the bleeps and bloops poking out from the sound.
Were there particular technical or creative capabilities you wanted to ensure were possible with these devices? How do you balance giving users detailed control while maintaining accessibility?
The freezer (in the Spectral Time) for instance should be usable whilst playing a guitar and using a footswitch. When I was using a guitar, I also quickly realised that if you wanted to freeze dense chords you would need a very long FFT length, so we added two extra-long ones to cover this case. There are also some hidden things you can do, for instance if you play with the delay time in the spectral delay, you can scrub through the delay buffer, going backwards and forwards in time. Initially this didn’t sound that good, so we added phase vocoder style time-stretching to the delay so that you could get reasonably good time-stretching by modulating the delay time. It’s the sort of thing that very few people would stumble across, but if you do it is fun!
It was also important to me that Spectral Resonator could do much more than just a vocoder or a comb filter – for instance be tuned to ridiculous low frequencies, have the harmonics spaced strangely – this gives you the opportunity to create weird drum like resonators or even reverb. Early in the design process it’s good to push all the parameters to the extremes just in case there is a cool use case there.
Are there particular creative applications of spectral processing you’d like to see more producers explore?
I think there is a lot of scope for a morphing progression of loops with spectral processing. Often, it’s just cutoff, decay, reverb to build up to the drop but there could be such weirder transformations such as spectral gating or compression (it does take someone to do the DSP for these effects nicely though!). One of my favourite albums is A Shocking Hobby by Speedy J. I don’t’ know exactly what he was doing, but it sounds like some twisted spectral stuff going on in the breakdowns. It’s amazing morphing metallic crunchy weirdness, it’s a shame that nobody (including Speedy J) really ran with that sound!
What are some entry-point applications of spectral processing that you’d recommend to people new to this form of processing?
The Ableton devices are quite accessible of course! Also, a spectral compressor is very fun to use. If you are familiar with a multiband compressor, they are basically the same but with a zillion bands.
What are some creative tricks you’ve discovered while working with spectral processing?
I’m always after ways to make my vocals more robotic, so I’m using Spectral Resonator as a send effect, with MIDI controlling the frequency of the resonator. This gives you a strange synth-like reverb tail. I also use the freezer to freeze the vowel parts of my vocals, so I can turn a sustained part of a vocal into a constant tone, a bit like an old Speak-and-Spell would sound. I also enjoy using a guitar with Spectral Time and setting the freeze to occur when I play a particularly loud note. This way I’m controlling the effect just by playing, rather than worrying about footswitches all over the place.
Do you have any exciting projects lined up for either your musical persona or the software development side of things?
For my music I have embarked on a project that will take a while to gestate – the goal being to play techno influenced repetitive dance riffs on the guitar, using a hexaphonic pick-up and processing each string differently to make polyrythmic, mesmeric, abstract arpeggios. That’s the idea anyway, the catch is that it only sounds good if you can play perfectly evenly and exactly on the grid – and I can’t yet! I basically need to practice way more with a metronome.
What we’re working on at Ableton is still a secret I’m afraid, but I am tinkering with some VST plugins on the side, which I am writing in the Rust language. I have a cross between a sequencer and an arpeggiator with some nice performance-oriented controls. I am also working on a scrubber/beat repeat thing that is intended to blur the line between looping, freezing and shuffling and is also designed around live performance. I’m hoping I get time to finish them!