EE269 - Signal Processing and Quantization for Machine Learning

Lecture Slides

Introduction, signal processing and machine learning systems

Discrete signals, quantization and change of basis

Quantization noise

Non-uniform quantizers, Lloyd-Max optimality and high-rate theory

Dithering and stochastic rounding

Discrete Fourier Transform (DFT)

DFT based spectral descriptors

Distance based signal classification, nearest neighbor classifier and Hilbert spaces

Continuous and Discrete Wavelet Transform

Applications of Wavelets and Short Time Fourier Transform: Signal classification, separation and denoising

Linear systems and eigenvector decomposition

Cepstrum and Mel-frequency Cepstral Coefficients (MFCC)

Bayes classifiers, Bayes risk and signal detection

Stationary signals, autocorrelation, linear and quadratic discriminant analysis

Fisher's linear discriminant, simultaneous diagonalization

Multi-class discriminant, separating hyperplanes and Support Vector Machine (SVM)

Constrained optimization, convex duality and dual SVM

Nonlinear features, kernels and kernel machines

Least-squares regression and autoregressive models

Reproducing Kernel Hilbert Spaces and functional regularization

Adaptive filters, Least Mean Squares algorithm

Neural networks

Deep learning, convolutional networks and spectrograms

Convex optimization for neural networks and overparameterized models

Autoencoders, Robust Principal Component Analysis and nuclear norm

Nonnegative Matrix Factorization, clustering and deep matrix factorizations

Dictionary learning and matching pursuit

Diffusion models for waveform and image generation