Advice to best analyze a balance task

I am conducting a research study where subjects are sitting in a simulated wave motion cabin on top of a 6 DOF device to try to get them sick. With this I am having the subjects complete some cognitive tasks during the ride. One of the tasks is having the subjects hold a board and try to keep it level to earth. Subjects are given the example that is a glass of water is sitting on top of the board you do not want it to spill. There is an IMU placed on the board gathering their inputs. I am trying to figure out what the best way is to analyze how well the subjects did on keeping the board level. I tried RMS error and it was really messy. Any advice would be helpful. I attached a figure of the subject IMU inputs as well as reference IMU that is capturing the movement of the simulator.

[Image Analyst]

What is an "IMU"? I'm guessing that your signal is for a glass that changes all 6 degrees of rotation, like the angle of the glass changes and the location of the glass also changes. What would be the signal if the subject kept the water perfectly level (no angular rotation) and in the same place (no translation)? All zeros? How about perfectly level (unrotated) but not in the same (x,y,z) location?

[Austin Bollinger]

IMU is an inertial measurement unit which measures degrees and accelerations. The task is just a wooden board that has this IMU on top. There is no visual reference to the subjects. If they kept this board completely level their values would stay at the zero axis for pitch and roll. If they didn't move the board at all the subject IMU values would follow the reference. The subjects should tilt the board in the opposite direction of the movement to get a level value.

[Jeff Miller]

Do you want an instantaneous accuracy measurement or one integrated across the whole 200 sec?

Looking at the Ref & Subject pitch lines in the figure you plotted, and just looking at the first vertical slice (i.e., before Time 140 sec), it looks like the subject pitch changes in the same direction as the ref but with smaller amplitude. Does this indicate that the subject partially succeeded in compensating for the roll motion of the cabin in that time slice?

What was messy about RMS error?

[Austin Bollinger]

Preferbely one integrated across the whole 200 sec as I am having the subjects do this task multiple times and trying to find any change in ability.

With the smaller amplitude that would indicate that the subject partially succeeded in compensating.

I just took an average RMS error across the whole 200 seconds or so. Some subjects it worked okay and others not as much. The ones that did fairly well on the task the RMS works out, but the ones that did poorly it wasn't working very well.

[Image Analyst]

The RMS is a single number. Exactly what about that number tells you whether it (the RMS metric) "worked out" or "wasn't working very well"?

[Austin Bollinger]

It works in a sense, but I just don't think RMS is the way to go.

Bad: Roll 6.4533 Pitch 13.0291

Good: Roll 2.6608 Pitch 3.2351

Here is two of the subjects. The subject that did the task very well had a low RMS acorss the trial, which is what I am looking for, but the subject that did bad had a higher RMS than even the reference values. The deviation from 0. It works in a sense that it does tell me this subject did awful at the task.

Answer 1

[Image Analyst]

If you don't like rms then maybe use mad - the mean or median deviation of the absolute value of the signal. It's like RMS but less influenced by extreme values.

[Austin Bollinger]

Thanks! I'll take a look at that.