Create Custom Plugin

This example shows how to create a custom plugin that counts the number of passing and failing assertions when the TestRunner is running a test suite. The plugin prints a brief summary at the end of the testing. To extend the TestRunner, the plugin subclasses and overrides select methods of the matlab.unittest.plugins.TestRunnerPlugin class.

Create AssertionCountingPlugin Class

In a file in your current folder, create the custom plugin class AssertionCountingPlugin, which inherits from the TestRunnerPlugin class. For the complete code for AssertionCountingPlugin, see AssertionCountingPlugin Class Definition Summary.

To keep track of the number of passing and failing assertions, define two read-only properties, NumPassingAssertions and NumFailingAssertions, within a properties block.

    properties (SetAccess = private)
        NumPassingAssertions
        NumFailingAssertions
    end

Extend Running of TestSuite

Implement the runTestSuite method in a methods block with protected access.

    methods (Access = protected)
        function runTestSuite(plugin, pluginData)
            suiteSize = numel(pluginData.TestSuite);
            fprintf('## Running a total of %d tests\n', suiteSize)

            plugin.NumPassingAssertions = 0;
            plugin.NumFailingAssertions = 0;

            runTestSuite@matlab.unittest.plugins.TestRunnerPlugin(...
                plugin, pluginData);

            fprintf('## Done running tests\n')
            plugin.printAssertionSummary()
        end
    end

The testing framework evaluates this method one time. It displays information about the total number of tests, initializes the properties used by the plugin to generate text output, and invokes the superclass method. After the framework completes evaluating the superclass method, the runTestSuite method displays the assertion count summary by calling the helper method printAssertionSummary (see Define Helper Methods).

Extend Creation of Shared Test Fixtures and TestCase Instances

Add listeners to AssertionPassed and AssertionFailed events to count the assertions. To add these listeners, extend the methods used by the testing framework to create the test content. The test content includes TestCase instances for each Test element, class-level TestCase instances for the TestClassSetup and TestClassTeardown methods, and Fixture instances used when a TestCase class has the SharedTestFixtures attribute.

Invoke the corresponding superclass method when you override the creation methods. The creation methods return the content that the testing framework creates for each of their respective contexts. When implementing one of these methods using the incrementPassingAssertionsCount and incrementFailingAssertionsCount helper methods, add the listeners required by the plugin to the returned Fixture or TestCase instance.

Add these creation methods to a methods block with protected access.

    methods (Access = protected)
        function fixture = createSharedTestFixture(plugin, pluginData)
            fixture = createSharedTestFixture@...
                matlab.unittest.plugins.TestRunnerPlugin(plugin, pluginData);

            fixture.addlistener('AssertionPassed', ...
                @(~,~)plugin.incrementPassingAssertionsCount);
            fixture.addlistener('AssertionFailed', ...
                @(~,~)plugin.incrementFailingAssertionsCount);
        end

        function testCase = createTestClassInstance(plugin, pluginData)
            testCase = createTestClassInstance@...
                matlab.unittest.plugins.TestRunnerPlugin(plugin, pluginData);

            testCase.addlistener('AssertionPassed', ...
                @(~,~)plugin.incrementPassingAssertionsCount);
            testCase.addlistener('AssertionFailed', ...
                @(~,~)plugin.incrementFailingAssertionsCount);
        end

        function testCase = createTestMethodInstance(plugin, pluginData)
            testCase = createTestMethodInstance@...
                matlab.unittest.plugins.TestRunnerPlugin(plugin, pluginData);

            testCase.addlistener('AssertionPassed', ...
                @(~,~)plugin.incrementPassingAssertionsCount);
            testCase.addlistener('AssertionFailed', ...
                @(~,~)plugin.incrementFailingAssertionsCount);
        end
    end

Extend Running of Single Test Suite Element

Extend runTest to display the name of each test at run time. Include this method in a methods block with protected access. Like all plugin methods, the runTest method requires you to invoke the corresponding superclass method.

    methods (Access = protected)
        function runTest(plugin, pluginData)
            fprintf('### Running test: %s\n', pluginData.Name)

            runTest@matlab.unittest.plugins.TestRunnerPlugin(...
                plugin, pluginData);
        end
    end

Define Helper Methods

In a methods block with private access, define three helper methods. These methods increment the number of passing or failing assertions, and print the assertion count summary.

    methods (Access = private)
        function incrementPassingAssertionsCount(plugin)
            plugin.NumPassingAssertions = plugin.NumPassingAssertions + 1;
        end

        function incrementFailingAssertionsCount(plugin)
            plugin.NumFailingAssertions = plugin.NumFailingAssertions + 1;
        end

        function printAssertionSummary(plugin)
            fprintf('%s\n', repmat('_', 1, 30))
            fprintf('Total Assertions: %d\n', ...
                plugin.NumPassingAssertions + plugin.NumFailingAssertions)
            fprintf('\t%d Passed, %d Failed\n', ...
                plugin.NumPassingAssertions, plugin.NumFailingAssertions)
        end
    end

AssertionCountingPlugin Class Definition Summary

The following code provides the complete contents of AssertionCountingPlugin.

classdef AssertionCountingPlugin < ...
        matlab.unittest.plugins.TestRunnerPlugin
    
    properties (SetAccess = private)
        NumPassingAssertions
        NumFailingAssertions
    end
    
    methods (Access = protected)
        function runTestSuite(plugin, pluginData)
            suiteSize = numel(pluginData.TestSuite);
            fprintf('## Running a total of %d tests\n', suiteSize)
            
            plugin.NumPassingAssertions = 0;
            plugin.NumFailingAssertions = 0;
            
            runTestSuite@matlab.unittest.plugins.TestRunnerPlugin(...
                plugin, pluginData);
            
            fprintf('## Done running tests\n')
            plugin.printAssertionSummary()
        end
        
        function fixture = createSharedTestFixture(plugin, pluginData)
            fixture = createSharedTestFixture@...
                matlab.unittest.plugins.TestRunnerPlugin(plugin, pluginData);
            
            fixture.addlistener('AssertionPassed', ...
                @(~,~)plugin.incrementPassingAssertionsCount);
            fixture.addlistener('AssertionFailed', ...
                @(~,~)plugin.incrementFailingAssertionsCount);
        end
        
        function testCase = createTestClassInstance(plugin, pluginData)
            testCase = createTestClassInstance@...
                matlab.unittest.plugins.TestRunnerPlugin(plugin, pluginData);
            
            testCase.addlistener('AssertionPassed', ...
                @(~,~)plugin.incrementPassingAssertionsCount);
            testCase.addlistener('AssertionFailed', ...
                @(~,~)plugin.incrementFailingAssertionsCount);
        end
        
        function testCase = createTestMethodInstance(plugin, pluginData)
            testCase = createTestMethodInstance@...
                matlab.unittest.plugins.TestRunnerPlugin(plugin, pluginData);
            
            testCase.addlistener('AssertionPassed', ...
                @(~,~)plugin.incrementPassingAssertionsCount);
            testCase.addlistener('AssertionFailed', ...
                @(~,~)plugin.incrementFailingAssertionsCount);
        end
        
        function runTest(plugin, pluginData)
            fprintf('### Running test: %s\n', pluginData.Name)
            
            runTest@matlab.unittest.plugins.TestRunnerPlugin(...
                plugin, pluginData);
        end 
    end
    
    methods (Access = private)
        function incrementPassingAssertionsCount(plugin)
            plugin.NumPassingAssertions = plugin.NumPassingAssertions + 1;
        end
        
        function incrementFailingAssertionsCount(plugin)
            plugin.NumFailingAssertions = plugin.NumFailingAssertions + 1;
        end
        
        function printAssertionSummary(plugin)
            fprintf('%s\n', repmat('_', 1, 30))
            fprintf('Total Assertions: %d\n', ...
                plugin.NumPassingAssertions + plugin.NumFailingAssertions)
            fprintf('\t%d Passed, %d Failed\n', ...
                plugin.NumPassingAssertions, plugin.NumFailingAssertions)
        end
    end
end

Create Example Test Class

In your current folder, create a file named ExampleTest.m containing the following test class.

classdef ExampleTest < matlab.unittest.TestCase
    methods(Test)
        function testOne(testCase)  % Test fails
            testCase.assertEqual(5, 4)
        end
        function testTwo(testCase)  % Test passes
            testCase.verifyEqual(5, 5)
        end
        function testThree(testCase)  % Test passes
            testCase.assertEqual(7*2, 14)
        end
    end
end

Add Plugin to TestRunner and Run Tests

At the command prompt, create a test suite from the ExampleTest class.

import matlab.unittest.TestSuite
import matlab.unittest.TestRunner

suite = TestSuite.fromClass(?ExampleTest);

Create a TestRunner instance with no plugins. This code creates a silent runner and gives you control over the installed plugins.

runner = TestRunner.withNoPlugins;

Run the tests.

result = runner.run(suite);

Add AssertionCountingPlugin to the runner and run the tests.

runner.addPlugin(AssertionCountingPlugin)
result = runner.run(suite);

## Running a total of 3 tests
### Running test: ExampleTest/testOne
### Running test: ExampleTest/testTwo
### Running test: ExampleTest/testThree
## Done running tests
______________________________
Total Assertions: 2
	1 Passed, 1 Failed