How to format sequences to store in experience buffer for DRQN?

16 views (last 30 days)
For DRQN (Deep Recurrent Q Learning) in POMDP it is needed to store entire sequences instead of individual transitions in the replay buffer. For the object agent.ExperienceBuffer, how to construct the data? For example, for Observation element i have tried to have a 1x1 cell with inside the numchannel x sequencelength, and also to have a cell array directly numchannel x sequencelength. the idea was to then sample minibatch of sequences instead of minibatch of transitions.
For any trial I get an error
Error using rl.replay.rlReplayMemory/validateExperience
Observation dimensions must match the dimensions specified in the corresponding specifications.
More specifically, when debugging i see that in the first case (1x1 cell) the code crashes at :
for obsCh = 1:numObsChannels
if ~all(size(NewObs{obsCh}) == obj.InternalReplayMemory_.ObservationDimension{obsCh})
error(message('rl:general:errIncorrectObservationDim'));
end
And in the second case at:
if numObsChannels ~= numel(NewObs)
error(message('rl:general:errIncorrectObservationDim'));
end
In MATLAB it is possible to have dqn with recurrent layers, so there is certainly a way to store these sequences somehow.
Thank you,
Imola

Answers (0)

Categories

Find more on Sequence and Numeric Feature Data Workflows in Help Center and File Exchange

Products


Release

R2022b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!