error in neural ODE simulation

I am discretizing DDE to ODE and constructing Neural ODE for this discretized ODE. But when I run this neural ODE using dlode45 in result of this I am getting complex dlarray and if i want to further proceed for loss function I get errors.

phi=@(x) cos(x); % Initial history function
g=@(x,y,par) par(1)*y/(1+y^par(3))-par(2)*x; % Mackey-Glass equation
par=[4;2;9.65]; % Parameters for the equation
tau=1; % Delay
M=30; % Number of discretization points
T=20; %
xsol=DDE_neural(g,par,tau,phi,M,T);
function xsol=DDE_neural(g,par,tau,phi,M,T)
theta=linspace(-tau,0,M+1)';
X0(:,1)=phi(theta);   
nx = 1; 
hiddenSize = 5;
NDDE = struct;
NDDE.fc1 = struct;
sz = [hiddenSize nx];
NDDE.fc1.Weights = initializeGlorot(sz);
NDDE.fc1.Bias = initializeZeros([sz(1) 1]);
NDDE.fc2 = struct;
sz = [hiddenSize hiddenSize];
NDDE.fc2.Weights = initializeGlorot(sz);
NDDE.fc2.Bias = initializeZeros([sz(1) 1]);
NDDE.fc3 = struct;
sz = [nx hiddenSize];
NDDE.fc3.Weights = initializeGlorot(sz);
xsol=dlode45(@DDEtoODE_euler,[0,T],dlarray(X0(:,1)),NDDE,DataFormat="CB");
    function Y=DDEtoODE_euler(t, X_n, NDDE)
        h=tau/M;
        D = zeros(M, M + 1);
        D(1:end-1, 1:end-2) = diag((1/(2*h)) * ones(M-1, 1));
        D(1:end-1, 2:end-1) = D(1:end-1, 2:end-1) + diag(zeros(M-1, 1));
        D(1:end-1, 3:end) = D(1:end-1, 3:end) + diag((-1/(2*h)) * ones(M-1, 1));
        D(end, end-1:end) = (1/h) * [1,-1];
        Y(2:M+1,1)=D*X_n;
        Y(1,1) = NDDE.fc3.Weights*tanh(NDDE.fc2.Weights*tanh(NDDE.fc1.Weights*g(X_n(1),X_n(M+1),par)+NDDE.fc1.Bias)+NDDE.fc2.Bias);
    end
    function bias = initializeZeros(sz)
        bias = zeros(sz,'single');
        bias = dlarray(bias);
    end
    function weights = initializeGlorot(sz)
        Z = 2*rand(sz,'single') - 1;
        bound = sqrt(6 / (sz(2)+ sz(1)));
        weights = bound * Z;
        weights = dlarray(weights);
    end
end

here I have written only the part of neural ODE simulation

[Torsten]

So you want to write your own delay ode solver because you work with dlarrays ? I wouldn't dare to do so.

Answer 1

[UDAYA PEDDIRAJU]

Hi Muhammad,

Based on the code you provided, it seems that you are getting a complex “dlarray” when you run the neural ODE using “dlode45”. This could be due to a variety of reasons, such as incorrect input arguments, incorrect implementation of the neural ODE, or incorrect use of the “dlode45” function.

The occurrence of complex numbers in your neural ODE simulation might be due to the Mackey-Glass equation's term “y^par(3)” when y is negative and par(3) is not an integer. To resolve this, you can try:

1. Ensure that the variable “y” remains positive if the Mackey-Glass equation in your model requires it.
2. Check for numerical stability in the “DDEtoODE_euler” function and consider using a smaller integration step size or a different ODE solver if necessary.
3. Verify that all initial conditions and parameters lead to real-valued solutions throughout the simulation.
4. Debug the code to pinpoint where the complex values originate and adjust the model or numerical method accordingly.

You can refer to the following documentation for more details:

• dlarray: https://www.mathworks.com/help/deeplearning/ref/dlarray.dlode45.html.
• Neural ODE layer: https://www.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.neuralodelayer.html.
• Nueral ODE: https://www.mathworks.com/help/deeplearning/ug/dynamical-system-modeling-using-neural-ode.html