Error in "Soft-Actor critic" agent with "rlNumericSpec" Observations and "rlFiniteSpec" Actions.

12 views (last 30 days)
Hello, I am trying to train SAC agent with "rlNumericSpec" Observations and "rlFiniteSpec" Actions.
This is my code:
ObservationInfo = rlNumericSpec([2 1],...
'LowerLimit',[-inf -inf]',...
'UpperLimit',[inf inf]');
ObservationInfo.Name = "observations";
ObservationInfo.Description = 'aa, bb';
numObservations = ObservationInfo.Dimension(1);
ActionInfo = rlFiniteSetSpec([1 2 3 4 5 6 7 8 9]);
ActionInfo.Name = 'Action';
ActionInfo.Description = 'test';
numActions = ActionInfo.Dimension(1);
net = [
featureInputLayer(ObservationInfo.Dimension(1))
fullyConnectedLayer(16)
reluLayer
fullyConnectedLayer(16)
reluLayer
fullyConnectedLayer(9)
];
net = dlnetwork(net);
critic = rlVectorQValueFunction(net,ObservationInfo,ActionInfo);
net1 = [
featureInputLayer(ObservationInfo.Dimension(1))
fullyConnectedLayer(16)
reluLayer
fullyConnectedLayer(16)
reluLayer
fullyConnectedLayer(9)
];
net1 = dlnetwork(net1);
actor = rlDiscreteCategoricalActor(net1,ObservationInfo,ActionInfo);
agentOptions = rlSACAgentOptions("SampleTime",Ts, ...
"DiscountFactor", 0.001, ...
"ExperienceBufferLength",1e6, ...
"MiniBatchSize",64, ...
"TargetSmoothFactor",0.005 ...
);
agent = rlSACAgent(actor,critic,agentOptions);
However, when I trying to train the model, the error occurs:
Error using rlSACAgent (line 96)
First argument must be an rlDiscreteCategoricalActor object, an rlContinuousGaussianActor object or an observation specification created
using 'rlNumericSpec' or 'rlFiniteSetSpec' objects.
Error in mdl_RF (line 60)
agent = rlSACAgent(actor,critic,agentOptions);
Please help me if you know the solutions.
  1 Comment
Hitesh
Hitesh on 22 Oct 2024
I am not able to reproduce this error using the above snippet of code, Could you share the complete code where you are creating the "env" and passing it during training ?

Sign in to comment.

Answers (1)

Gayathri
Gayathri on 29 Oct 2024
I faced a similar issue in MATLAB R2024a version. But the same code runs fine in MATLAB R2024b version. The solution would be to upgrade to MATLAB R2024b version.
Hope you find this information helpful.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!