Deep Neural LSTM Network Issues
1 view (last 30 days)
I am training a Deep Neural Net with a regression layer in the end. I have 20 inputs and a sequence of output with 10 steps. I tried using both LSTM and BiLISTM layers with something like 100 to 200 hidden units. I have also included two fully connected layers with some 100 hidden nodes or so. But no matter what I do and change learning rate etc.. the progress comes to a plateau very quickly like in a couple of epochs and remains so for the rest of the training. Tried changing learning rate, min batch size, Number of hidden units, adding relu and not adding relu etc etc.. you name it. But I can not improve the accracy to more than about 0.9 (validation MSE). Is there anything else I can try to improve?