De-Noising Audio using Spectral Subtraction in MATLAB and Ableton Live

Last time I wrote about audio restoration using simple digital filtering (in MATLAB and Ableton Live). I’ve since received another old Havering recording from Walt. Again from an old cassette tape, this recording is rather noisy. In this post, I explain how I cleaned it up using a more elaborate technique than previously.

Again I used MATLAB for the algorithm development aspects of the process, in combination with Ableton Live for the audio and mix management.

The noise

Here is a clip of the lead-in to the show. The noise is apparent.

Snippet of the raw (noisy) recording

Figures 1 and 2 show the noise spectrum (over the full bandwidth and zoomed-in to the low-frequency zone, respectively) computed via the MATLAB pspectrum function.

Figure 1: Noise spectrum revealing the broadband nature of the background noise in the recording.

Figure 2: Noise spectrum, zoomed-in on the low-frequency regime, revealing the 60 Hz “power hum” plus a distinct peak around 1150 Hz in both channels and a lesser peak around 1700 Hz in the left channel only.

The noise has similar characteristics to the last time: some low-frequency “power hum” (Figure 2) plus a broad-band “tape hiss” over the extent of the audio/music bandwidth (Figure 1). Interestingly, the low-frequency power hum (Figure 2) comprises only the fundamental mode (at approximately 60 Hz) rather than the multiple harmonics observed last time. Also, there is a distinct peak around 1150 Hz in both channels and a lesser peak around 1700 Hz in the left channel only.

Suppressing the “power hum”

As last time, notch filtering was used to suppress the low-frequency peaks from Figure 2. However, rather than using Ableton Live’s notch filtering as I did last time, I used MATLAB. This allowed me to create a suite of filters which could be separately configured for the left and right channels (since as observed in Figure 2, the characteristics of the noise peaks varies between the channels). As a starting point, I used the MultiNotchFilter example “plugin” bundled with the MATLAB Audio Toolbox and extended it to have separate controls for each channel (creating what I call the MultiNotchFilterStereo “plugin”). Figure 3 shows a (partial) screenshot of the plugin configured to suppress the peaks identified in the spectrum from Figure 2.

Figure 3: Screenshot of the MultiNotchFilterStereo plugin (adapted from the MultiNotchFilter plugin bundled with MATLAB) loaded into the MATLAB audioTestBench. The plugin has ten notch filters per channel. Only the first seven of the left channel filter controls are visible in the screenshot (there are similar controls for each of the ten filters per channel). Only three of the notches are being used on the left channel (and only two on the right channel), corresponding to the three noise peaks (at 55 Hz, 1136 Hz, and 1702 Hz) in the left channel (and 55 Hz and 1168 Hz for the right channel).

Here is the result of applying the notch filtering to the original noisy clip:

Result of applying the notch filtering to the snippet of the raw (noisy) recording in order to suppress the low-frequency noise components. Comparing with the raw clip presented earlier, it is clear that the filters have had an audible effect on suppressing some of the components of the noise.

Suppressing the “tape hiss”

Instead of simple filtering used last time, I wanted to try something more sophisticated in an attempt to achieve improved broad-band noise suppression with minimal audible artefacts.

The approach adopted was to adapt the SpectralSubtractor “plugin” bundled with the MATLAB Audio Toolbox, again extended to have separate processing for each channel (creating what I call the SpectralSubtractorStereo “plugin”) since the original plugin catered for mono signals only. Figure 4 shows a screenshot of the plugin configured (by trial-and-error listening experiments) to suppress the broadband noise identified in the spectrum from Figure 2.

Figure 4: Screenshot of the SpectralSubtractorStereo plugin loaded into the MATLAB audioTestBench. The plugin (adapted from the SpectralSubtractor plugin bundled with MATLAB) performs noise reduction by spectral subtraction, applied independently to both channels, but with the same user-configurable parameters configured on both channels.

The algorithm works by subtracting a representation of the noise from the noisy signal in the frequency domain. In this case, the representation of the noise is a simple constant amplitude (band-limited) “white noise” model.

The core of the algorithm is encapsulated in the first line of the following two lines of MATLAB code:

mag_X_out = max (0, abs(X_in)-Mag2Subtract);

X_out = mag_X_out.*exp(li*angle(X_in));

where mag_X_out is the magnitude of the processed spectrum, X_in is the noisy signal spectrum, and Mag2Subtract is the user-selected “noise magnitude” (i.e., configured via the the “Noise Estimate” control in Figure 4). In the second line of code, X_out is the processed spectrum created by reuniting the modified magnitude mag_X_out with the original phase of X_in.

Not shown in this code snippet is the application of the Fast Fourier Transform (FFT) and its inverse — to convert to/from the frequency/time domains — nor have I included the machinery for managing the data buffers, since I wanted to emphasise the crux of the algorithm (rather than the utility code around it) — and moreover, I wanted to demonstrate how compact the MATLAB language is for implementing mathematical expressions applied to complex-valued matrices (such as X_in and X_out).

A schematic illustrating the spectral subtraction technique is shown in Figure 5.

Figure 5: De-noising via the technique of spectral subtraction. The plots are in the frequency domain (i.e., after the FFT computation). Note that these are not actual signal spectra, merely pictorial representations to aid the explanation. Also, just a single-channel (mono) signal is depicted here (in the actual processor, the same algorithm is applied independently to each channel). The number of frequency bins (and hence the frequency resolution for a given sample-rate) is determined by the length of the analysis frame (i.e., the number of samples, per channel, sent to the FFT in each successive computation, performed frame-by-frame over the entire signal duration), adjusted via the “Analysis Frame” control in Figure 4. The “Noisy signal” (blue) in the upper plot corresponds to abs(X_in). The “Noise model” (red) corresponds to Mag2Subtract. The “De-noised signal” (green) in the lower plot corresponds to mag_X_out. It has the value zero whenever the “Noisy signal” is below the level of the “Noise model”. Elsewhere, it has the value given by (abs(X_in) minus Mag2Subtract).

In a sense, the “0” branch in the expression for mag_X_out in the code snippet can be thought of as a frequency-dependent noise gate, whereby for each frequency bin, if the spectral magnitude is below the user-selected threshold (i.e., the “white noise” magnitude), the signal output is cut completely. For the other branch, if the spectral magnitude is above the assumed model noise threshold, then that constant threshold level (representing the “white noise” magnitude) is subtracted from each bin.

The noise threshold is user-adjusted by trial-and-error. Too low, the de-noising is not effective. Too high, and audible artefacts appear in the output as a characteristic “tinkling”. This invariably occurs when frequency-domain audio manipulation is pushed too far. Indeed, it can be used as an effect in itself e.g., vocoders and robotic voices, or in the (well-established) technique of cranking up autotune to the extreme. But for the present purposes of de-noising, the parameters have been adjusted such that maximal noise suppression is achieved with minimal perceivable adverse effects on the output signal. Note that the “Analysis Window” (i.e., the type of windowing used before performing the FFT), the “Analysis Frame” (i.e., the length of the data chunk sent to the FFT), and the “Frame Overlap” are commonly-used in spectral analysis (as described in many references, so not detailed here). Suffice it to say, for present purposes, these parameters were selected by trial-and-error (via subjective listening experiments) to give the best result on the audio file in question.

Here is the result of applying the spectral subtraction to the noisy clip using the settings displayed in Figure 4:

Result of applying the spectral subtraction to the previous clip (i.e., the one with the power hum already removed). Comparing with the original raw clip presented at the start, it is clear that the spectral subtraction algorithm is very effective for suppressing the broad-band noise. There is a little bit of “tinkling” evident in the output, but this is effectively masked by the music (once it starts playing).

“One click” plugin creation

Having built and tested the MultiNotchFilterStereo and the SpectralSubtractorStereo “plugins” entirely within the MATLAB environment, I then converted each of them to VST plugins using the “one click” conversion button provided in the MATLAB Audio Toolbox audioTestBench interface.

Additional tweaks to the mix within Ableton Live

I then loaded the VST plugins into Ableton Live, applied a noise gate in front of them, and some equalisation and dynamic range control downstream, as shown in the screenshot in Figure 6.

Figure 6: End-to-end plugin effects chain implemented in Ableton Live for this de-noising project. The first (“Short Cut” noise gate) and last (“Punchy Dance Master” compressor/limiter/equaliser component) are Ableton built-in plugins used to tweak the mix. The middle two components (“MultiNotchFilterStereo” and “SpecralSubtractorStereo”) are the VST plugins built entirely in MATLAB and are the core of the de-noising solution presented in this article.

This effects chain was applied to the noisy recording of the entire radio show. The resulting cleaned-up audio can be streamed from here.

Conclusions

The spectral subtraction method, using a simple flat “white noise” model, is found to be rather effective in removing broad-band “tape hiss” noise from audio/music recordings. Compared with simple digital filtering (covered in the previous post), the spectral subtraction method is found to be superior (from informal subjective listening trials).

As an enhancement of the technique, it would be interesting to try subtracting a shaped noise spectrum (rather than the simple flat value used here). This could be computed from a noise-only portion of the recording. Likewise, it would be interesting to compare the spectral subtraction approach with alternative techniques such as wavelet-based de-noising, machine-learning/deep-learning based de-noising, and adaptive filtering. All these can be explored via MATLAB.

MATLAB is again found to be a very powerful and convenient environment for prototyping the audio processing algorithms. Moreover, the (remarkable) “one click” creation of VST plugins from entirely within MATLAB makes it trivially simple to bring the algorithms into the Digital Audio Workstation (DAW) universe.

Footnote

You may have noticed this logo in the compiled MATLAB VST plugin screenshots above. There is a history to this. Just over twenty years ago, I worked with a very talented programmer, Pepijn Sitter, from The Netherlands, to create an audio effects processing software product called WaveWarp. We distributed it under the trading name Sounds Logical. It was critically acclaimed, winning an Editor’s Choice Award from Electronic Musician Magazine in 2001.

WaveWarp enabled you to build your own audio effects from a library of modular building blocks. In that sense, it’s architecture resembled Simulink, but was fundamentally much faster (even compared with the compiled version of Simulink deployed via the RealTimeWorkshop) on account of the fact that the WaveWarp audio engine (and each individual module) was written in highly-optimised C code (making extensive use of pointer arithmetic) such that it could process multi-channel audio in real-time, sample-by-sample, on a typical desktop PC of the age. Moreover, it had full multi-rate functionality (via a library of decimators, interpolators, polyphase filterbanks, etc) allowing for elaborate mixed sample-rate designs. It used the FFTW (Fastest Fourier Transform in the West) library for spectral analysis, just as MATLAB does now. The WaveWarp software worked in standalone mode or as a DirectX plugin, and even had a real-time interface to MATLAB (akin to the audioTestBench available in the MATLAB Audio Toolbox today).

Alas, WaveWarp is now long gone. Moreover, I lost track of the source-code years ago, and I don’t have a running version. Also, it has almost completely faded from the internet. I could find only this review on PCRecording.com.

Anyway, given that I find myself delving into the world of audio processing again, I thought it fitting to revive the logo.