Extracting mathematical equation from a deep neural network?
14 views (last 30 days)
Show older comments
Dear all ,
I have built a deep neural network consist of two hidden layers (6 neurons in the first layer and 4 in the second layer), (11 inputs), and (1 output). Below is the code:
net=newff(Norm_inputs,Norm_Outputs,[6 4],{'tansig' 'tansig' 'tansig'}); %% Neural Network with 12 inputs and four Neurons in the Hidden layers
net.divideParam.trainRatio=80; %% 70% of the data are used for training
net.divideParam.testRatio=20; %% 15% of the data are used for testing
net.divideParam.valRatio=0; %% 15% of the data are used for the validation
net.trainParam.lr=0.01; %% Learning rate
net.trainParam.min_grad=1e-20;
net.trainParam.goal=1e-30;
net.trainParam.epochs=1000;
net = train(net,Norm_inputs,Norm_Outputs);
where Norm_inputs are the normalized inputs and Norm_Outputs are the normalized outputs:
Below is the output of my network:
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/1225322/image.jpeg)
To test the accuracy of network I am using the following:
A = net([0.613497 1 1 0.660657 0.675336 0.150442 0.343584 0.162069 0.655799 0.574284 9.83E-05 ]')
I am comparing the output of A with my raw data to check the performance..
I tried to extract the mathematical form so I can avoid opening matlab everytime to see the results..
I did the following in the command window to exctract the weights and biases:
x = [0.2 0 0.048095238 0.814509123 1 0.292035398 0.028111384 0 0.749206349 0.35239154 0.0013]' % The input vector as a column
b1 = net.b{1}; b2 = net.b{2}; b3 = net.b{3}; % Biases
W_ITH = net.IW{1,1}; W_H1TH2 = net.LW{2,1}; W_H2TO = net.LW{3,2}; % Weights
Then I used the following equation to see the output:
a1 = tansig(b1+W_ITH*x) % Layer 1
a2 = tansig(b2+W_H1TH2*y1) % Layer 2
a3 = tansig(b3+W_H2TO*h2) % Layer 3
Output = tansig(a3) % Output layer
However, if I compared the output with what I got from A = net([ ]'), the results will be different so I think I am doing something wrong in the last line.. To understand what is going on I used the following function:
genFunction(net)
and this is what I got:
function [Y,Xf,Af] = neural_function(X,~,~)
%NEURAL_FUNCTION neural network simulation function.
%
% Auto-generated by MATLAB, 09-Dec-2022 09:21:01.
%
% [Y] = neural_function(X,~,~) takes these arguments:
%
% X = 1xTS cell, 1 inputs over TS timesteps
% Each X{1,ts} = 11xQ matrix, input #1 at timestep ts.
%
% and returns:
% Y = 1xTS cell of 1 outputs over TS timesteps.
% Each Y{1,ts} = 1xQ matrix, output #1 at timestep ts.
%
% where Q is number of samples (or series) and TS is the number of timesteps.
%#ok<*RPMT0>
% ===== NEURAL NETWORK CONSTANTS =====
% Input 1
x1_step1.xoffset = [0;0;0;0;0;0;0;0;0;0;0];
x1_step1.gain = [2;2;2;2;2;2;2;2;2;2;2];
x1_step1.ymin = -1;
% Layer 1
b1 = [-1.6619087105318077757;-0.38036635353628145406;0.070456977515701249559;-1.8091719441920439682;0.84024035176083067267;-2.4082013915970765794];
IW1_1 = [0.17151362799105107637 0.43991566279934013473 -0.94498059911126330856 -0.42006296125946129827 -0.27862563546688645655 -0.83527543485057420547 0.39025369258432240915 -0.59597775914195516567 0.030901804283541799145 -1.5751433471974514156 2.2185153817891616335;1.4170772331681724676 0.35412432286137413007 0.87818875125559181516 -0.17600030163403754258 0.26322893962013838021 -0.84054484142076568709 -0.11892676428554814494 -1.1826941118532154906 1.1218133611751428536 -0.66767731298128440987 3.1106872504963392068;-0.26723479009346129409 0.81357312787374558294 0.72789078646866034461 -0.50921265559455153316 -0.39103833955865030525 -0.62173294964878533797 0.68528117173730473954 -0.34828465558276422431 -0.16312788337813632911 0.21050282043176274382 0.44335360877887930453;0.14487115798130367761 -0.15055640733584960134 0.27543617993750263429 -0.59709575795996938652 -0.75216339591064962367 1.2416027483157188183 1.1298395428727894263 0.84984116490102246111 -0.10259937135856361801 -0.49136865338803564773 -3.4508552900537483588;0.21061050817627047227 -1.0259134414309891703 -0.47307696880542565721 -0.10211219235696178398 -0.97576324395602820339 0.85652978231966880873 0.57414719896924193421 1.0580352911678043704 -0.11113538128158330365 2.0123285342463415049 -1.0707627999428350751;-0.31632362570748778774 1.498261146701386215 1.8694622275477512652 0.54377418209887562472 -0.16850753331158352388 -0.024211365158402883141 0.50363354864679121814 0.053259098864533692408 0.048581620707959571881 -1.3625415675598020471 -4.8017707994150420348];
% Layer 2
b2 = [-2.155229374705729839;-0.46393350865168381247;-0.250561899861700077;-2.030526644413503945];
LW2_1 = [-0.074382195279472429483 3.2640182588225155058 0.17760021246183130272 -3.9560253984244440062 -0.66153193018127776082 0.27963632610539712608;0.44281065487010662363 0.73006863933605237182 -0.59246756174612402734 0.81153976385101223023 -1.876640649290156837 -0.50558075563543158903;-0.95688266706868496669 -1.7303968831455176502 0.27663573250824052829 1.9643907072858886043 -0.46236471788163485686 -0.60703199794076179163;-2.0165635473477441231 -0.63275446155609926802 -0.28943668952945711403 0.3566114961582590448 2.9589004495388779858 4.2633945339043108902];
% Layer 3
b3 = -0.66635787301535986948;
LW3_2 = [3.617152099907068763 1.7855472593354049238 1.8237105913499436216 -3.6510555660572112657];
% Output 1
y1_step1.ymin = -1;
y1_step1.gain = 2;
y1_step1.xoffset = 0;
% ===== SIMULATION ========
% Format Input Arguments
isCellX = iscell(X);
if ~isCellX
X = {X};
end
% Dimensions
TS = size(X,2); % timesteps
if ~isempty(X)
Q = size(X{1},2); % samples/series
else
Q = 0;
end
% Allocate Outputs
Y = cell(1,TS);
% Time loop
for ts=1:TS
% Input 1
Xp1 = mapminmax_apply(X{1,ts},x1_step1);
% Layer 1
a1 = tansig_apply(repmat(b1,1,Q) + IW1_1*Xp1);
% Layer 2
a2 = tansig_apply(repmat(b2,1,Q) + LW2_1*a1);
% Layer 3
a3 = tansig_apply(repmat(b3,1,Q) + LW3_2*a2);
% Output 1
Y{1,ts} = mapminmax_reverse(a3,y1_step1);
end
% Final Delay States
Xf = cell(1,0);
Af = cell(3,0);
% Format Output Arguments
if ~isCellX
Y = cell2mat(Y);
end
end
% ===== MODULE FUNCTIONS ========
% Map Minimum and Maximum Input Processing Function
function y = mapminmax_apply(x,settings)
y = bsxfun(@minus,x,settings.xoffset);
y = bsxfun(@times,y,settings.gain);
y = bsxfun(@plus,y,settings.ymin);
end
% Sigmoid Symmetric Transfer Function
function a = tansig_apply(n,~)
a = 2 ./ (1 + exp(-2*n)) - 1;
end
% Map Minimum and Maximum Output Reverse-Processing Function
function x = mapminmax_reverse(y,settings)
x = bsxfun(@minus,y,settings.ymin);
x = bsxfun(@rdivide,x,settings.gain);
x = bsxfun(@plus,x,settings.xoffset);
end
So everything is similar except the last lines:
% Output 1
Y{1,ts} = mapminmax_reverse(a3,y1_step1);
So I am doing something wrong in last step(i.e., the following line is wrong):
Output = tansig(a3) % Output layer
Any help!
0 Comments
Answers (1)
Udit06
on 1 Sep 2023
Hi Mahmoud,
The code generated using “genFunction” indicates that MATLAB internally applies “minmax” processing to the input and output data. For more details you can refer the following MATLAB answer: https://www.mathworks.com/matlabcentral/answers/385206-why-is-predicting-the-output-of-a-trained-neural-network-by-using-net-sim-different-than-manually#answer_307446.
I hope this helps.
0 Comments
See Also
Categories
Find more on Define Shallow Neural Network Architectures in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!