rlPPOAgent
Description
Proximal policy optimization (PPO) is a model-free, online, on-policy, policy gradient reinforcement learning method. This algorithm alternates between sampling data through environmental interaction and optimizing a clipped surrogate objective function using stochastic gradient descent. The action space can be either discrete or continuous. For continuous action spaces, this agent does not enforce constraints set in the action specification; therefore, if you need to enforce action constraints, you must do so within the environment.
For more information on PPO agents, see Proximal Policy Optimization (PPO) Agents. For more information on the different types of reinforcement learning agents, see Reinforcement Learning Agents.
Creation
Syntax
Description
Create Agent from Observation and Action Specifications
creates a proximal policy optimization (PPO) agent for an environment with the given
observation and action specifications, using default initialization options. The actor
and critic in the agent use default deep neural networks built from the observation
specification agent
= rlPPOAgent(observationInfo
,actionInfo
)observationInfo
and the action specification
actionInfo
. The ObservationInfo
and
ActionInfo
properties of agent
are set to
the observationInfo
and actionInfo
input
arguments, respectively.
creates a PPO agent for an environment with the given observation and action
specifications. The agent uses default networks configured using options specified in
the agent
= rlPPOAgent(observationInfo
,actionInfo
,initOpts
)initOpts
object. Actor-critic agents do not support recurrent
neural networks. For more information on the initialization options, see rlAgentInitializationOptions
.
Create Agent from Actor and Critic
Specify Agent Options
creates a PPO agent and sets the AgentOptions
property to the agent
= rlPPOAgent(___,agentOptions
)agentOptions
input argument. Use this syntax after
any of the input arguments in the previous syntaxes.
Input Arguments
Properties
Object Functions
train | Train reinforcement learning agents within a specified environment |
sim | Simulate trained reinforcement learning agents within specified environment |
getAction | Obtain action from agent, actor, or policy object given environment observations |
getActor | Extract actor from reinforcement learning agent |
setActor | Set actor of reinforcement learning agent |
getCritic | Extract critic from reinforcement learning agent |
setCritic | Set critic of reinforcement learning agent |
generatePolicyFunction | Generate MATLAB function that evaluates policy of an agent or policy object |
Examples
Tips
For continuous action spaces, this agent does not enforce the constraints set by the action specification. In this case, you must enforce action space constraints within the environment.
While tuning the learning rate of the actor network is necessary for PPO agents, it is not necessary for TRPO agents.
Version History
Introduced in R2019b
See Also
Apps
Functions
getAction
|getActor
|getCritic
|getModel
|generatePolicyFunction
|generatePolicyBlock
|getActionInfo
|getObservationInfo
Objects
rlPPOAgentOptions
|rlAgentInitializationOptions
|rlValueFunction
|rlDiscreteCategoricalActor
|rlContinuousGaussianActor
|rlPGAgent
|rlACAgent
|rlTRPOAgent