Audio Feature Extraction in Julia

Audio911.jl extracts audio features for machine learning: spectrograms on linear, mel, bark and ERB scales, MFCC and GTCC cepstra in every published variant, deltas, spectral and temporal descriptors, chroma, onsets and tempo, harmonic/percussive separation, and the utilities around them. Where MATLAB's Audio Toolbox has a feature, Audio911 reproduces it numerically (the test suite checks against audioFeatureExtractor fixtures); where librosa has one, the coverage table in the docs says whether and how it is covered. The algorithms of audioFlux are ported and checked against audioFlux itself; the inventory page lists every one of them.

Features

Interchangeable pipeline

Every stage is an immutable object built from the previous one, and every time-frequency front end implements the same interface, so an STFT and a wavelet scalogram feed the same mel filterbank, cepstrum and descriptors:

load ─▶ AudioFile ─▶ Frames ─┬─▶ Stft ──┐
                             ├─▶ Cwt ───┤
                             ├─▶ Cqt ───┼─▶ LinSpec ─────────────▶ Spectral*
                             ├─▶ Pwt ───┼─▶ MelSpec / BarkSpec ─▶ Mfcc ─▶ Delta
                             ├─▶ Nsgt ──┼─▶ ErbSpec ─────────────▶ Gtcc ─▶ Delta
                             └─▶ St/Fst ┴─▶ Chroma, Tonnetz, Contrast, Onset, Hpss, ...

Time-frequency front ends

• STFT (Stft): pre-planned real FFT streamed over lazy frames, threaded, power or magnitude spectrum
• Wavelet scalogram (Cwt): Morlet, Morse, bump, Paul, DOG, Mexican hat, Hermitian and Ricker wavelets on a geometric or any filterbank grid, pooled on the same frame grid as the STFT; synchrosqueezing (Wsst, Synsq)
• Discrete wavelets (dwt, wpt, swt and the Dwt, Wpt, Swt front ends) with 51 wavelet families
• Wigner-Ville and Choi-Williams distributions (Wvd, Cwd), empirical mode decomposition and the Hilbert-Huang spectrum (emd, Hht), the empirical wavelet transform (ewt, Ewt)
• Constant-Q / variable-Q transform (Cqt), pseudo wavelet transform (Pwt), S-transform and fast S-transform (St, Fst), non-stationary Gabor transform (Nsgt), ported from audioFlux and checked against it
• Complex coefficients on request (get_complex, get_phase), the inverse STFT (istft) and the reassigned spectrogram (Reassign)

Spectrograms and filterbanks

• LinSpec, MelSpec (HTK or Slaney mel), BarkSpec, ErbSpec (gammatone)
• audioFlux's linspace, erb, octave and logspace scales and window filter styles (hanning, gauss, kaiser, bohman, point, ...) on any filterbank stage
• filterbank design on any frequency grid: auditory_fbank, gammatone_fbank, chroma_fbank, plus the integer-bin banks of ETSI and python_speech_features

Cepstra, in every variant

• Mfcc, Gtcc, Delta, with every axis the literature differs on as a keyword (rectification, floor, DCT scaling, liftering, C0, energy)
• presets: mfcc_matlab, mfcc_htk, mfcc_kaldi, mfcc_librosa, mfcc_etsi, mfcc_psf

Descriptors and features

• eleven MATLAB spectral descriptors plus SpectralBandwidth, Rms, Energy, Zcr, Pitch (NCF, YIN, cepstral, PEF, HPS, LHS, spectral peaks), HarmonicRatio, harmonic_count
• audioFlux's descriptors: energy, RMS, high-frequency content, spectral difference, modified Kullback-Leibler, phase and complex-domain deviations, broadband, novelty, energy-entropy, max/peak/mean/variance
• Chroma (STFT or constant-Q), Tonnetz, SpectralContrast, PolyFeatures
• spectral deconvolution (Deconv), the cepstrogram (Cepstrogram), the energy to zero-crossing ratio (Ezr), xxcc_standard
• OnsetStrength, audioFlux's Novelty onset envelopes, onset_detect, Tempogram, tempo, beat_track
• Hpss harmonic/percussive separation (spectrograms or signals), nmf, viterbi and discrete Hmms, pcen, noisegate, SpectralGate
• unit conversions, dB scaling, A/B/C/D weighting curves, synthesis (synth_f0), silence trimming, LPC, czt, xcorr, convolve, feature_scale
• time_stretch, pitch_shift and the phase_vocoder; band-limited sinc resampling (resample(...; method=:sinc))

Extras

• Built-in audio loading: WAV, FLAC, OGG and MP3 through libsndfile and mpg123; in-memory arrays through AudioFile(x, sr); resampling, mono, normalisation
• Plots recipes for every stage (using Plots; plot(mel)), no Plots dependency
• Float32 end to end and allocation-regression tests

Installation

using Pkg
Pkg.add("Audio911")

Quick Start

using Audio911

# Load an audio file (resampled to 16 kHz, mono, Float32)
audio = load("speech.wav"; sr=16000)

# Frames and STFT
stft = Stft(audio; winsize=512, winstep=256, type=hamming, periodic=true, spectrum=power)

# Mel spectrogram (MATLAB-compatible defaults)
mel = MelSpec(stft; win_norm=true, nbands=32, norm=bandwidth, domain=:linear, scale=htk)

# MFCC and deltas
mfcc  = Mfcc(mel; ncoeffs=13, rect=mlog)
delta = Delta(mfcc)

get_data(mfcc)      # frames × 13 matrix
get_times(mfcc)     # frame centres in seconds

# Same pipeline on a wavelet scalogram
mfcc_w = Mfcc(MelSpec(Cwt(audio; winsize=512, winstep=256); nbands=32); ncoeffs=13)

# A published recipe in one call
k = mfcc_kaldi(audio)

using Plots
plot(mel; freq_scale=:log10)

Learn More

The documentation has a step-by-step tutorial, one page per stage, the pipeline design, the MFCC variants with their references, the librosa/MATLAB coverage table, before/after performance numbers and the full API reference.

Development: test/run.sh test runs the suite, test/run.sh docs builds the site, test/run.sh bench prints the benchmark.

About

Audio911.jl is developed by the ACLAI Lab @ University of Ferrara.

License

MIT License