Deep reinforcement learning is a branch of machine learning that enables you to implement controllers and decision-making systems for complex systems such as robots and autonomous systems. Deep reinforcement learning lets you implement deep neural networks that can learn complex behaviors by training them with data generated dynamically from simulated or physical systems. Unlike other machine learning techniques, there is no need for predefined training datasets, labeled or unlabeled. Typically, all you need is a simulation model that represents your environment.
Using MATLAB®, Simulink®, and Reinforcement Learning Toolbox™ you can run through the complete workflow for designing and deploying a decision-making system. You can:
- Get started with deep reinforcement learning using examples for simple control systems, autonomous systems, robotics, and scheduling problems
- Quickly switch, evaluate, and compare popular reinforcement learning algorithms with only minor code changes
- Model the environment in MATLAB or Simulink
- Use deep neural networks to define complex deep reinforcement learning policies based on image, video, and sensor data
- Train policies faster by running multiple simulations in parallel using local cores or the cloud
- Deploy deep reinforcement learning policies to embedded devices
Deep Reinforcement Learning Agents
Deep reinforcement learning agents are comprised of a deep neural network policy that maps an input state to an output action, and an algorithm responsible for updating this policy. Deep Q-network (DQN), deep deterministic policy gradient (DDPG), soft actor critic (SAC), and proximal policy optimization (PPO) are popular examples of algorithms. The algorithm updates the policy based on the observations and rewards collected from the environment to maximize the expected long-term reward.
Reinforcement Learning Toolbox helps you create deep reinforcement learning agents programmatically, or interactively with the Reinforcement Learning Designer app. Select from popular algorithms provided out of the box, or implement your own custom algorithm using available templates and examples.
Environment Modeling in MATLAB and Simulink
Training with deep reinforcement learning algorithms is a dynamic process as the agent interacts with the environment around it. For applications such as robotics and autonomous systems, performing this training with actual hardware can be expensive and dangerous. This is why virtual models of the environment that generate data through simulations are greatly preferred for deep reinforcement learning.
You can build a model of your environment in MATLAB and Simulink that describes the system dynamics, how they are affected by actions taken by the agent, and a reward that evaluates the goodness of the action performed. These models can be continuous or discrete in nature and can represent your system at varying levels of fidelity. In addition, you can parallelize simulations to accelerate training. In some cases, you may be able to reuse existing MATLAB and Simulink models of your system for deep reinforcement learning with minimal modifications.
Examples and Reference Applications
Get started with deep reinforcement learning by training policies for simple problems such as balancing an inverted pendulum, navigating a grid-world problem, and balancing a cart-pole system. You can also design systems for adaptive cruise control and lane-keeping assist for autonomous vehicles. Deep reinforcement learning can also be used for robotics applications, such as trajectory planning, and teaching behaviors, such as locomotion.