Single-/Doubleprecision in custom CNN-Layer

Hi everyone,

I have written a custom layer for a CNN in Matlab to double the dimension of an input (2x2-matrix becomes 4x4-matrix). For verification, I used the function checkLayer() in Matlab and I received an error I cannot get rid of concerning single-/doubleprecision.

Here my layer:

classdef myUpsamplingLayer < nnet.layer.Layer

    properties
        % empty
    end

    properties (Learnable)
        % empty
    end
    
    methods
        function layer = myUpsamplingLayer(name)
            layer.Name = name;
        end
        
        function Z = predict(layer, X)
            Z = zeros(size(X,1)*2,size(X,2)*2,size(X,3),size(X,4));
            Idx = (1:2:2*size(X,1));
           
            Z(Idx,Idx,:,:) = X;     % Only purpose:
            Z(Idx+1,Idx,:,:) = X;   % To double the dimensions of the 
            Z(Idx,Idx+1,:,:) = X;   % input-data
            Z(Idx+1,Idx+1,:,:) = X;
            
        end

        function [dLdX] = backward(layer, X, Z, dLdZ, memory)
            dLdX = zeros(size(X,1),size(X,2),size(X,3),size(X,4));
            Idx = (1:2:2*size(X,1));
            dLdX = dLdX + dLdZ(Idx,Idx,:,:) +dLdZ(Idx+1,Idx,:,:) + ...
                dLdZ(Idx,Idx+1,:,:) +dLdZ(Idx+1,Idx+1,:,:);
        end
    end
end

And here the error I get when checking my layer:

>> layer = myUpsamplingLayer("Herbert")

layer =

myUpsamplingLayer with properties:

Name: 'Herbert'

Show all properties

>> checkLayer(layer,[8,8,1,10])

Skipping multi-observation tests. To enable tests with multiple observations, specify the 'ObservationDimension' parameter in checkLayer.

For layers used in convolutional neural networks, set 'ObservationDimension' to be 4.

For layers used in recurrent neural networks, set 'ObservationDimension' to be 2.

Skipping GPU tests. No compatible GPU device found.

Running nnet.checklayer.TestCase

....

================================================================================

Verification failed in nnet.checklayer.TestCase/predictIsConsistentInType(Precision=single,Device=cpu).

----------------

Test Diagnostic:

----------------

Incorrect type of 'Z' for 'predict'.

---------------------

Framework Diagnostic:

---------------------

Actual Type:

double

Expected Type:

single

------------------

Stack Information:

------------------

In C:\Program Files\MATLAB\R2018b\toolbox\nnet\cnn\+nnet\+checklayer\TestCase.m (TestCase.predictIsConsistentInType) at 131

================================================================================

...... .

Done nnet.checklayer.TestCase

__________

Failure Summary:

Name Failed Incomplete Reason(s)

==============================================================================================================================

nnet.checklayer.TestCase/predictIsConsistentInType(Precision=single,Device=cpu) X Failed by verification.

Test Summary:

10 Passed, 1 Failed, 0 Incomplete, 10 Skipped.

Time elapsed: 0.99925 seconds.

Actually if I force Z to be of single precision, I get the opposite error. Then it expects Z to be double and also the derivative of the loss is incorrect due to rounding errors. Can someone tell me how this precision has to be handled? And also if you notice any construction-errors in my layer or have suggestions for improvement, please tell me :D

[Jack Xiao]

The same case, have you sovled it?

Answer 1

[youhao yu]

Z = zeros(size(X,1)*2,size(X,2)*2,size(X,3),size(X,4),'like',X);