Apply convolution to image processing, signal processing, and deep learning

Convolution is a mathematical operation that combines two signals and outputs a third signal.

Assuming we have two functions, \(f(t)\) and \(g(t)\), convolution is an integral that expresses the amount of overlap of one function \(g\) as it is shifted over function \(f\)

Convolution is expressed as:

$$(f*g)(t) \approx^{def} \int_{-\infty}^{\infty}f(\tau)g(t-\tau)dr$$

Graphical representation of convolution.

Depending on the application of convolution, functions may be replaced with signals, images, or other types of data. Convolution and its applications can be implemented in several ways in MATLAB®.

Convolution in Signal Processing

Convolution is used in digital signal processing to study and design linear time-invariant (LTI) systems such as digital filters.

The output signal, \(y[n]\), in LTI systems is the convolution of the input signal, \(x[n]\) and impulse response \(h[n]\) of the system

Convolution for linear time-invariant systems.

Convolution for linear time-invariant systems.

In practice, the convolution theorem is used to design the filter in the frequency domain. The convolution theorem states that the convolution in the time domain equals the multiplication in the frequency domain.

MATLAB functions such as conv and filter allow you to perform convolution and build filters from scratch. Signal Processing Toolbox™ and DSP System Toolbox™ have several functions and Simulink® blocks for direct implementation of digital filters such as designfilt, lowpass, and highpass.

Convolution in Image Processing

In image processing, convolutional filtering can be used to implement algorithms such as edge detection, image sharpening, and image blurring.

This is done by selecting the appropriate kernel (convolution matrix).

Convolution operation with a 3x3 sharpening filter kernel.

Image Processing Toolbox™ has functions such as fspecial and imfilterv to design filters to emphasize certain features or remove other features in images.

Image blurring performed through convolution with an averaging filter.

Image blurring performed through convolution with an averaging filter. See MATLAB example.

Convolution Neural Networks (CNNs)

Convolution plays a key role in convolutional neural networks (CNNs). CNNs are a type of deep network commonly used to analyze images. CNNs eliminate the need for manual feature extraction, which is why they work very well for complex problems like image classification and medical image analysis. CNNs are effective for non-image data analysis such as audio, time-series, and signal data.

CNNs have several layers, the most common of which are convolution, ReLu, and pooling.

Layers in a convolutional neural network (CNN)

Layers in a convolutional neural network (CNN).

Convolution layers act as filters – each layer applies a filter and extracts specific features from the image. These filter values are learned by the network when the network is trained. The initial layers typically extract low-level features while the deeper layers extract high-level features from the data.

For more information on convolution and its applications in signal processing, image processing, deep learning, and other areas, see Signal Processing Toolbox, DSP System Toolbox, Image Processing Toolbox and Deep Learning Toolbox™ for use with MATLAB.

See also: convolutional neural network, edge detection, integral, MATLAB and Simulink for signal processing, MATLAB for image processing and computer vision, MATLAB for deep learning, deep learning for signal processing

Image Processing Onramp

Learn the basics of practical image processing techniques in MATLAB.

Signal Processing Onramp

An interactive introduction to practical signal processing methods for spectral analysis.