Error in "Soft-Actor critic" agent with "rlNumericSpec" Observations and "rlFiniteSpec" Actions.
12 views (last 30 days)
Show older comments
Hello, I am trying to train SAC agent with "rlNumericSpec" Observations and "rlFiniteSpec" Actions.
This is my code:
ObservationInfo = rlNumericSpec([2 1],...
'LowerLimit',[-inf -inf]',...
'UpperLimit',[inf inf]');
ObservationInfo.Name = "observations";
ObservationInfo.Description = 'aa, bb';
numObservations = ObservationInfo.Dimension(1);
ActionInfo = rlFiniteSetSpec([1 2 3 4 5 6 7 8 9]);
ActionInfo.Name = 'Action';
ActionInfo.Description = 'test';
numActions = ActionInfo.Dimension(1);
net = [
featureInputLayer(ObservationInfo.Dimension(1))
fullyConnectedLayer(16)
reluLayer
fullyConnectedLayer(16)
reluLayer
fullyConnectedLayer(9)
];
net = dlnetwork(net);
critic = rlVectorQValueFunction(net,ObservationInfo,ActionInfo);
net1 = [
featureInputLayer(ObservationInfo.Dimension(1))
fullyConnectedLayer(16)
reluLayer
fullyConnectedLayer(16)
reluLayer
fullyConnectedLayer(9)
];
net1 = dlnetwork(net1);
actor = rlDiscreteCategoricalActor(net1,ObservationInfo,ActionInfo);
agentOptions = rlSACAgentOptions("SampleTime",Ts, ...
"DiscountFactor", 0.001, ...
"ExperienceBufferLength",1e6, ...
"MiniBatchSize",64, ...
"TargetSmoothFactor",0.005 ...
);
agent = rlSACAgent(actor,critic,agentOptions);
However, when I trying to train the model, the error occurs:
Error using rlSACAgent (line 96)
First argument must be an rlDiscreteCategoricalActor object, an rlContinuousGaussianActor object or an observation specification created
using 'rlNumericSpec' or 'rlFiniteSetSpec' objects.
Error in mdl_RF (line 60)
agent = rlSACAgent(actor,critic,agentOptions);
Please help me if you know the solutions.
1 Comment
Hitesh
on 22 Oct 2024
I am not able to reproduce this error using the above snippet of code, Could you share the complete code where you are creating the "env" and passing it during training ?
Answers (1)
Gayathri
on 29 Oct 2024
I faced a similar issue in MATLAB R2024a version. But the same code runs fine in MATLAB R2024b version. The solution would be to upgrade to MATLAB R2024b version.
Hope you find this information helpful.
0 Comments
See Also
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!