Generate Cobertura Coverage Reports for Generated C Code in Equivalence Tests

When you run C equivalence tests using software-in-the-loop (SIL) verification with Embedded Coder^®, you can collect code coverage information for the generated C code and generate a Cobertura XML code coverage report to view the coverage results. Collect the coverage by adding an instance of the matlabtest.coder.plugins.GeneratedCodeCoveragePlugin class to the test runner. As the tests run, the plugin collects information about the parts of the source code that the tests executed. You can then generate the report by using the generateCoberturaReport method.

For more information about generated C/C++ equivalence tests, see Generate C/C++ Code and Test for Equivalence and Collect Coverage for Generated C/C++ Code in Equivalence Tests.

You can collect these types of coverage:

• Statement

• Function

• Decision

• Condition

• Modified condition/decision (MC/DC)

For more information, see Types of Coverage for Generated C/C++ Code in Equivalence Tests.

Note

You can only collect coverage for C equivalence tests on Windows^® and Linux^® platforms.

Write SIL Equivalence Tests

You can collect coverage only for C code generated in an equivalence test that builds a LIB target. To ensure that continuous integration platforms can access the generated files, preserve the generated files in the equivalence test by using the PreserveInFolder name-value argument in the build method. For more information about writing equivalence tests, see Generate C/C++ Code and Test for Equivalence.

Suppose that you want to generate code for a function called myMath, which allows a user to provide two numeric inputs and an operation string to indicate whether to add or subtract the inputs:

function y = myMath(a,b,operation) %#codegen
if operation == "add"
    y = a+b;
elseif operation == "subtract"
    y = b-a;
else
    y = [];
end
end

This SIL equivalence test uses class properties and class-level setup to generate a static library and specifies that the operation argument can be a variable-sized string. The equivalence test preserves the generated files in a folder called myArtifacts and uses parameterization to execute and verify the generated C code twice, with different inputs each time.

classdef tMyMathSIL < matlabtest.coder.TestCase
    properties
        buildResults;
    end

    methods (TestClassSetup)        
        function generateCode(testCase)           
            operationSize = coder.typeof("add");
            operationSize.StringLength = inf;
            buildInputs = {1,2,operationSize};
            path = "myArtifacts";
            testCase.buildResults = build(testCase,"myMath", ...
                Inputs=buildInputs,Configuration="lib", ...
                PreserveInFolder=path);
        end        
    end

    properties (TestParameter)
        runInputs = {{1,2,"add"},{1,2,"subtract"}};
    end
    
    methods(Test)
        function testSILvsMATLAB(testCase,runInputs)
            executionResults = execute(testCase, ...
                testCase.buildResults,Inputs=runInputs);
            verifyExecutionMatchesMATLAB(testCase,executionResults);
        end        
    end
end

Collect Coverage

To run equivalence tests with SIL verification and collect coverage:

1. In the test runner script, define the coverage format by creating an instance of matlab.unittest.plugins.codecoverage.CoverageResult.

2. Create a plugin for the generated C code by creating an instance of matlabtest.coder.plugins.GeneratedCodeCoveragePlugin. Specify the desired coverage type for the plugin or use the default settings, which specify statement and function coverage.

3. Create a test runner by using the testrunner function.

4. Add the code coverage plugin to the test runner by using the addPlugin method.

5. Create a test suite for the equivalence tests by using the testsuite function.

6. Run the tests by using the run method.

This example code creates a test runner with a plugin that programmatically accesses the coverage information for all types of code coverage.

import matlab.unittest.plugins.codecoverage.CoverageResult
import matlabtest.coder.plugins.GeneratedCodeCoveragePlugin

format = CoverageResult;
plugin = GeneratedCodeCoveragePlugin(Producing=format, ...
    MetricLevel="mcdc");
runner = testrunner("textoutput");
addPlugin(runner,plugin);

This example code creates a test suite for the tMyMathSIL equivalence test class and runs the tests. All tests pass.

suite = testsuite("tMyMathSIL.m");
run(runner,suite);

Running tMyMathSIL
..
Done tMyMathSIL
__________

Generate Cobertura XML Report

To generate the Cobertura XML code coverage report, pass the Result property of the matlab.unittest.plugins.codecoverage.CoverageResult object to the generateCoberturaReport method. For more information about the Cobertura XML report format, see matlab.unittest.plugins.codecoverage.CoberturaFormat.

This example code generates an Cobertura XML code coverage report called myCoberturaCoverageReport for the coverage results collected from the test in tMyMathSIL.m and stores the report in a folder called myArtifacts.

covResult = format.Result;
path = fullfile(pwd,"myArtifacts","myCoberturaCoverageReport.xml");
generateCoberturaReport(covResult,path);