How to put vector as a element in rlNumericSpec?

Hi, Everyone,

I am trying to create custom MATLAB Environment from template. My observations consist of numerical variable and vector, like

observation = [number, number, vector, vector]

Unfortunately, I find rlNumericSpec only support number as element. I want to know how to deal with this situation.

Answer 1

[Emmanouil Tzorakoleftherakis]

You could do something along the lines of:

ObservationInfo(1) = rlNumericSpec([1 1]);
ObservationInfo(1).Name = 'scalar';
ObservationInfo(2) = rlNumericSpec([3 1]);
ObservationInfo(2).Name = 'vector';

and then use the obsInfo to create your environment

[Yang Chen]

Thx, Emmanouil. I also want to know the situation for variable-length array. For example, we have a variable whose spec can't be confirm at that time.

Var = [];

for i= 1 : n

Var(end)= 1;

end

In that case, how to set the rlNumericSpec?

[Emmanouil Tzorakoleftherakis]

Unfortunately you variable-length observations are not supported right now. If you know what the max length of those arrays could be, one workaround is to create observations of max length, and then as you train you can complete any empty slots with 0s.

[Yang Chen]

Hi, Emmanouil, I know the max length of my variable-length observations. Could you please offer any examples about how to do it? I will appreciate it.

[Emmanouil Tzorakoleftherakis]

Not sure how much I can add without knowing specifics, but assuming your environment is in MATLAB (i.e. not Simulink), when you step through your dynamics you will basically get observations that differ in length. In that function, I would assign each observation to a specific slot. So, for example, obs(1) is position, obs(2) is velocity, obs(3) is acceleration. If in certain situations you don't have, e.g. velocity measurements, then set obs(2)=0 and the other two values would be equal to your measurements. If you don't have acceleration, obs(3)=0 and so on

[Yang Chen]

Thx, Emmanouil, understood.