# Deep Learning Custom Training Loops

If the `trainingOptions`

function
does not provide the training options that you need for your task, or
custom output layers do not support the loss functions that you need,
then you can define a custom training loop. For networks that cannot be
created using layer graphs, you can define custom networks as a
function. To learn more, see Define Custom Training Loops, Loss Functions, and Networks.

## Functions

## Topics

### Custom Training Loops

**Train Deep Learning Model in MATLAB**

Learn how to training deep learning models in MATLAB^{®}.

**Define Custom Training Loops, Loss Functions, and Networks**

Learn how to define and customize deep learning training loops, loss functions, and networks using automatic differentiation.

**Train Network Using Custom Training Loop**

This example shows how to train a network that classifies handwritten digits with a custom learning rate schedule.

**Specify Training Options in Custom Training Loop**

Learn how to specify common training options in a custom training loop.

**Define Model Gradients Function for Custom Training Loop**

Learn how to define a model gradients function for a custom training loop.

**Update Batch Normalization Statistics in Custom Training Loop**

This example shows how to update the network state in a custom training loop.

**Make Predictions Using dlnetwork Object**

This example shows how to make predictions using a `dlnetwork`

object by splitting data into mini-batches.

**Train Network on Image and Feature Data**

This example shows how to train a network that classifies handwritten digits using both image and feature input data.

**Train Network with Multiple Outputs**

This example shows how to train a deep learning network with multiple outputs that predict both labels and angles of rotations of handwritten digits.

**Classify Videos Using Deep Learning with Custom Training Loop**

This example shows how to create a network for video classification by combining a pretrained image classification model and a sequence classification network.

**Train Image Classification Network Robust to Adversarial Examples**

This example shows how to train a neural network that is robust to adversarial examples using fast gradient sign method (FGSM) adversarial training.

This example shows how to train an augmented neural ordinary differential equation (ODE) network.

**Train Robust Deep Learning Network with Jacobian Regularization**

This example shows how to train a neural network that is robust to adversarial examples using a Jacobian regularization scheme [1].

**Solve Ordinary Differential Equation Using Neural Network**

This example shows how to solve an ordinary differential equation (ODE) using a neural network.

**Assemble Multiple-Output Network for Prediction**

This example shows how to assemble a multiple output network for prediction.

**Run Custom Training Loops on a GPU and in Parallel**

Speed up custom training loops by running on a GPU, in parallel using multiple GPUs, or on a cluster.

### Model Functions

**Train Network Using Model Function**

This example shows how to create and train a deep learning network by using functions rather than a layer graph or a `dlnetwork`

.

**Update Batch Normalization Statistics Using Model Function**

This example shows how to update the network state in a network defined as a function.

**Make Predictions Using Model Function**

This example shows how to make predictions using a model function by splitting data into mini-batches.

**Initialize Learnable Parameters for Model Function**

Learn how to initialize learnable parameters for custom training loops using a model function.

### Automatic Differentiation

**List of Functions with dlarray Support**

View the list of functions that support `dlarray`

objects.

**Automatic Differentiation Background**

Learn how automatic differentiation works.

**Use Automatic Differentiation In Deep Learning Toolbox**

How to use automatic differentiation in deep learning.

### Deep Learning Function Acceleration

**Deep Learning Function Acceleration for Custom Training Loops**

Accelerate model functions and model gradients functions for custom training loops by caching and reusing traces.

**Accelerate Custom Training Loop Functions**

This example shows how to accelerate deep learning custom training loop and prediction functions.

**Check Accelerated Deep Learning Function Outputs**

This example shows how to check that the outputs of accelerated functions match the outputs of the underlying function.

**Evaluate Performance of Accelerated Deep Learning Function**

This example shows how to evaluate the performance gains of using an accelerated function.