I am struggling a lot in my NN project

I have 20 inputs and one output. I am trying to train using the Deepnetwork work flow. I tried several combinations of hidden layers with varying number of nodes in those layers. I tried adam and sgdm with various combinations of min batchsize and learning rate (0.1, 0.01, 0.001, 0.0001 ) etc. I am getting the same type of results. Here is a sample I am attaching. The train and val start together for a while then val. does not improve and the network training stops. The results are no good as predictions have huge errors. Not sure what else I can try to improve so any suggestions are welcome and I am willing to try.

Some background. Of the 20, 19 are material related properties and 20 is fatigue load. Output is Number of cylces to failure. I am trying to predict given a set of paramters the SN curve (stress vs Cycles). I have close to 6000 or so SN curves as data to train.

Answer 1

[prabhat kumar sharma]

Hello Pappu,

I understand you are facing issue with the generalization of model due to which your final output is almost similar.

You can try the below mentioned strategies to improve your model's performace.

1. Data Processing and Feature Engineering

• Normalization/Standardization: Ensure that all input features are normalized or standardized to have a mean of 0 and a standard deviation of 1.
• Feature Selection: Analyze the importance of each feature and remove features that might not be contributing to the predictive power of the model.
• Feature Engineering: Create new features or transform existing ones to better capture the underlying patterns in the data.

2. Network Architecture

• Simplifying the Model (If your model is too complex)
• Complexifying the Model ( If your model is too simple already)
• Different Activation Functions (Eg: ReLU, ELU, Leaky ReLU etc.)

3. Regularization Techniques:

• Introduction of dropout layers to prevent overfitting.
• L1/L2 Regularization. Regularization - MATLAB & Simulink (mathworks.com)

4. Consider training multiple models and using ensemble techniques, such as averaging or voting, to improve predictions.

5. Select a loss function that aligns with your specific problem, ensuring it is suitable for the task at hand.

I belive this will help you to solve yor problem.