Normalization of observation in RL in DDPG

2 views (last 30 days)
Laxmi
Laxmi on 2 Oct 2024
Answered: Drishti on 3 Oct 2024
I am training a smart train optimization model using reinforcement learning, where distance and velocity are part of the observation space. The distance ranges from 0 to 10,000 meters, while the velocity has significantly smaller values. I'm considering whether to convert the distance to kilometers for scaling or use a different normalization technique, as I’ve done in PyTorch. However, I haven't come across built-in normalization options in MATLAB for this task.

Answers (1)

Drishti
Drishti on 3 Oct 2024
Hi Laxmi,
The Reinforcement Learning toolbox provides the ‘rlNormalizer’ object which configure normalization for input of function approximator object. It ensures that the inputs are on a similar scale when provided to the network.
The ‘rlNormalizer’ has been introduced in MATLAB R2024a and later versions.
You can refer to the MATLAB Documentation of ‘rlNormalizer’ for better understanding:
You can also leverage the ‘unitsratio’ function for the conversion of input from one unit to another.
Refer to the example below:
mPerFoot = unitsratio("meter","feet");
MATLAB provides explicit functions ‘normalize’ and ‘zscore’ to scale the data. The ‘normalize’ function normalizes data within specified range whereas the ‘zscore’ depicts the standardized z-score.
Refer to the implementation below:
distance_normalized = normalize(distance, "range", [0, 1]);
distance_standardized = zscore(distance);
You can also leverage the MATLAB Documentation of ‘normalize’ and ‘zscore’ functions:
I hope this helps in getting started.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!