Deep learning Toolbox - LSTM Training

4 views (last 30 days)
PB75
PB75 on 2 Sep 2022
Edited: PB75 on 7 Sep 2022
Hi All,
I am building an LSTM ROM to integrate into my Simscape model, which is using training and test data captured in ANSYS Chemkin software.
I have had multiple issues with pre-processing the data, which I am working through, that main reason being is ANSYS only uses a variable step solver.
I am basing my code on the LSTM ROM example. The validation of the LSTM model (before we integrate into the full Simscape model) will have to be a simple model using the "Stateful Predict" block, and then using the test and train data captured in the workspace to comapre to the LSTM model, using "From Workspace" array block to get the signals into Simulink.
I have been able to run the code, and the training progress seems that the model has minimised the RMSE and loss, however, when I try to run my simple validaton model the results show something else!, see below.
As I am unsure as to how to proceed with debugging, I am unsure whether it is the pre-processed data (which may need resampling), the LSTM network and parameter selection or the validation model that is the main cause.
My question is, can I now use the Deep Network Designer app to help debugging. Now using the code generated the concatenated data is now in a 10x1 cell, can I now save this as a datastore and use the app to train the network?
I can show the training progress, data structure and validation output below.
Training Progress:
Validation Model:
Validation Results: Test data top row, LSTM data bottom row.
Any suggestions on how to proceed would be great, as stuck as to how to proceed.
Thanks in advance,
Patrick

Answers (1)

Arkadiy Turevskiy
Arkadiy Turevskiy on 2 Sep 2022
Hi Patrick,
Two suggestions:
  1. Try to test your LSTM network in MATLAB first. Does it match the validation data. If it does, then the issue is with a Simulink model.
  2. If your validation data in Simulink does not start at time 0, you need to reset the state of LSTM in State and Predict block by putting this block into a resettable subsystem and triggering it before your data starts. Ie. if your data starts at t-0.2, then do a step from 0 to 1 at 0.1 sec and feed that into reset port of resettable susbystem to reset the state.
Arkadiy
  3 Comments
PB75
PB75 on 7 Sep 2022
Edited: PB75 on 7 Sep 2022
Hi Ben,
Thanks for the extra guidance, so I have made a little progress with testing the network in a Simulink model, I will show the state predict block in my main model below and the ouput. My error seems to be the order of the inputs into the mux and sequenceinput. I do have an output from the state predict block now, but the results are poorer than those of the predict function?
Additionally, I was able to use the interp1 command and was able to resize the raw ANSYS data and effectively resample it, Resampling. So now I have all the data at a fixed time step of 2.0E-05 sec prior to post-processing, and now with a downsample factor of 5, which gives a reasonable step size of 1.0E-04 sec for the training and test data I think?
All my sample times in the state predict subsystem triggered block mirrors the sample rate used in the training and test of 1.0E-04 sec. Also I am using a fixed step solver.
I can test the network with the predict function, as show below. The output for the first output state is good, but the output of the remaining 3 states gets poor, which may mean I need to norm all the data prior to preparing the data. I have done a post for this, hope someone can help with my code errors for finding the maximum value and normalising the cell containing the data Normalising a cell. The fidelity of the predict function and state predict outputs seem vastly different, even with the same step sizes of 1.0E-04 sec.
Predict Function Ouput:
Simulink Model Output:
The top signal is the trigger signal, signals 2 are the model total volume and the triggered LSTM volume, signals 3, 4, 5 and 6 are the network ouput.
Simulink Model - sample rates set to 1.0E-04 sec as per LSTM training,and a fixed rate solver (ODE4)
Any suggestions on how to match the predict function output and state predict output would be great, and how to improve the network results in general.
Thanks again,
Patrick

Sign in to comment.

Categories

Find more on Deep Learning with Time Series and Sequence Data in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!