getActionInfo

Obtain action data specifications from reinforcement learning environment, agent, or experience buffer

Syntax

actInfo = getActionInfo(env)

actInfo = getActionInfo(agent)

actInfo = getActionInfo(buffer)

Description

actInfo = getActionInfo(env) extracts action information from reinforcement learning environment env.

actInfo = getActionInfo(agent) extracts action information from reinforcement learning agent agent.

actInfo = getActionInfo(buffer) extracts action information from experience buffer buffer.

Examples

Extract Action and Observation Information from Reinforcement Learning Environment

Extract action and observation information that you can use to create other environments or agents.

The reinforcement learning environment for this example is the simple longitudinal dynamics for ego car and lead car. The training goal is to make the ego car travel at a set velocity while maintaining a safe distance from lead car by controlling longitudinal acceleration (and braking). This example uses the same vehicle model as the Adaptive Cruise Control System Using Model Predictive Control (Model Predictive Control Toolbox) example.

Open the model and create the reinforcement learning environment.

mdl = 'rlACCMdl';
open_system(mdl);
agentblk = [mdl '/RL Agent'];
% create the observation info
obsInfo = rlNumericSpec([3 1],'LowerLimit',-inf*ones(3,1),'UpperLimit',inf*ones(3,1));
obsInfo.Name = 'observations';
obsInfo.Description = 'information on velocity error and ego velocity';
% action Info
actInfo = rlNumericSpec([1 1],'LowerLimit',-3,'UpperLimit',2);
actInfo.Name = 'acceleration';
% define environment
env = rlSimulinkEnv(mdl,agentblk,obsInfo,actInfo)

env = 
SimulinkEnvWithAgent with properties:

           Model : rlACCMdl
      AgentBlock : rlACCMdl/RL Agent
        ResetFcn : []
  UseFastRestart : on

The reinforcement learning environment env is a SimulinkWithAgent object with the above properties.

Extract the action and observation information from the reinforcement learning environment env.

actInfoExt = getActionInfo(env)

actInfoExt = 
  rlNumericSpec with properties:

     LowerLimit: -3
     UpperLimit: 2
           Name: "acceleration"
    Description: [0x0 string]
      Dimension: [1 1]
       DataType: "double"

obsInfoExt = getObservationInfo(env)

obsInfoExt = 
  rlNumericSpec with properties:

     LowerLimit: [3x1 double]
     UpperLimit: [3x1 double]
           Name: "observations"
    Description: "information on velocity error and ego velocity"
      Dimension: [3 1]
       DataType: "double"

The action information contains acceleration values while the observation information contains the velocity and velocity error values of the ego vehicle.

Input Arguments

env — Reinforcement learning environment

rlFunctionEnv object | SimulinkEnvWithAgent object | rlNeuralNetworkEnvironment object | predefined MATLAB environment object

Reinforcement learning environment from which to extract the action information, specified as one of the following:

• MATLAB^® environment represented as one of the following objects.

  • rlFunctionEnv

  • rlNeuralNetworkEnvironment

  • Predefined MATLAB environment created using rlPredefinedEnv

• Simulink^® environment represented as a SimulinkEnvWithAgent object.

For more information on reinforcement learning environments, see Create MATLAB Reinforcement Learning Environments and Create Simulink Reinforcement Learning Environments.

agent — Reinforcement learning agent

rlQAgent object | rlSARSAAgent object | rlDQNAgent object | rlPGAgent object | rlDDPGAgent object | rlTD3Agent object | rlACAgent object | rlPPOAgent object | rlTRPOAgent object | rlSACAgent object | rlMBPOAgent object

Reinforcement learning agent from which to extract the action information, specified as one of the following objects.

• rlQAgent

• rlSARSAAgent

• rlDQNAgent

• rlDDPGAgent

• rlTD3Agent

• rlPGAgent

• rlACAgent

• rlPPOAgent

• rlTRPOAgent

• rlSACAgent

• rlMBPOAgent

For more information on reinforcement learning agents, see Reinforcement Learning Agents.

buffer — Experience buffer

rlReplayMemory object | rlPrioritizedReplayMemory object

Experience buffer from which to extract the action information, specified as an rlReplayMemory or rlPrioritizedReplayMemory object.

Output Arguments

actInfo — Action data specifications

array of rlNumericSpec objects | array of rlFiniteSetSpec objects

Action data specifications extracted from the reinforcement learning environment, returned as an array of one of the following:

• rlNumericSpec objects

• rlFiniteSetSpec objects

• A mix of rlNumericSpec and rlFiniteSetSpec objects

Version History

Introduced in R2019a