runInParallel

Class: matlab.unittest.TestRunner
Package: matlab.unittest

Run all tests in TestSuite array in parallel

Syntax

result = runInParallel(runner,suite)

Description

result = runInParallel(runner,suite) divides the specified test suite into groups and uses the specified test runner to run each group on the current parallel pool. The method then returns the results in an array of TestResult objects.

When tests run in parallel, test suite portions run independently on MATLAB^® workers. For example, if your test class has a TestClassSetup method, the method runs locally on each worker. Workers use the information in their corresponding TestSuite elements to run the tests. Each TestSuite element provides the worker with information on the Test method, test class, and shared test fixtures to run for the suite element.

Note

The runInParallel method requires Parallel Computing Toolbox™. The testing framework might vary the order and number of groups or which tests it includes in each group.

Input Arguments

runner — Test runner for parallel test groups

matlab.unittest.TestRunner instance

Test runner for parallel test groups, specified as a matlab.unittest.TestRunner instance.

Consider your test runner configuration before running tests in parallel. Since the runInParallel method runs separate groups of tests on different workers, some plugins, such as StopOnFailuresPlugin, are not well suited for parallelization. The testing framework supports running tests in parallel with a custom plugin, provided that the plugin subclasses the Parallelizable interface.

suite — Set of tests to run in parallel

matlab.unittest.Test array

Set of tests to run in parallel, specified as a matlab.unittest.Test array.

Examples

Run Tests in Parallel

Create the following parameterized test in a file in your current working folder.

classdef TestRand < matlab.unittest.TestCase    
    properties (TestParameter)
        dim1 = createDimensionSizes;
        dim2 = createDimensionSizes;
        dim3 = createDimensionSizes;
        type = {'single','double'};
    end
    
    methods (Test)
        function testRepeatable(testCase,dim1,dim2,dim3)
            state = rng;
            firstRun = rand(dim1,dim2,dim3);
            rng(state)
            secondRun = rand(dim1,dim2,dim3);
            testCase.verifyEqual(firstRun,secondRun);
        end
        function testClass(testCase,dim1,dim2,type)
            testCase.verifyClass(rand(dim1,dim2,type),type)
        end
    end
end
 
function sizes = createDimensionSizes
% Create logarithmically spaced sizes up to 100
sizes = num2cell(round(logspace(0,2,10)));
end

At the command prompt, create a suite from TestRand.m and a test runner that displays text in the Command Window.

suite = matlab.unittest.TestSuite.fromClass(?TestRand);
runner = matlab.unittest.TestRunner.withTextOutput();

The suite contains 1200 test elements.

Run the test suite in parallel.

result = runInParallel(runner,suite)

Split tests into 12 groups and running them on 4 workers.
----------------
Finished Group 2
----------------
Running TestRand
..........
..........
..........
..........
..........
..........
..........
..........
..........
..........
.........
Done TestRand
__________


----------------
Finished Group 4
----------------
Running TestRand
..........
..........
..........
..........
..........
..........
..........
..........
..........
..........
.....
Done TestRand
__________


----------------
Finished Group 3
----------------
Running TestRand
..........
..........
..........
..........
..........
..........
..........
..........
..........
..........
.......
Done TestRand
__________


----------------
Finished Group 1
----------------
Running TestRand
..........
..........
..........
..........
..........
..........
..........
..........
..........
..........
..........
..
Done TestRand
__________


----------------
Finished Group 7
----------------
Running TestRand
..........
..........
..........
..........
..........
..........
..........
..........
..........
.........
Done TestRand
__________


----------------
Finished Group 5
----------------
Running TestRand
..........
..........
..........
..........
..........
..........
..........
..........
..........
..........
...
Done TestRand
__________


----------------
Finished Group 6
----------------
Running TestRand
..........
..........
..........
..........
..........
..........
..........
..........
..........
..........
.
Done TestRand
__________


----------------
Finished Group 8
----------------
Running TestRand
..........
..........
..........
..........
..........
..........
..........
..........
..........
.......
Done TestRand
__________


-----------------
Finished Group 11
-----------------
Running TestRand
..........
..........
..........
..........
..........
..........
..........
..........
..........
.
Done TestRand
__________


-----------------
Finished Group 12
-----------------
Running TestRand
..........
..........
..........
..........
..........
..........
..........
..........
........
Done TestRand
__________


-----------------
Finished Group 10
-----------------
Running TestRand
..........
..........
..........
..........
..........
..........
..........
..........
..........
...
Done TestRand
__________


----------------
Finished Group 9
----------------
Running TestRand
..........
..........
..........
..........
..........
..........
..........
..........
..........
.....
Done TestRand
__________



result = 

  1200x1 TestResult array with properties:

    Name
    Passed
    Failed
    Incomplete
    Duration
    Details

Totals:
   1200 Passed, 0 Failed, 0 Incomplete.
   11.4023 seconds testing time.

Tips

• When you select a test suite to run in parallel, consider possible resource contention. For example, if your test fixtures access global resources, such as a database or a shared file on the same network, the parallel sessions could conflict with each other. In such cases, consider using a prebuilt shared test fixture.

• Starting in R2020b, you can create standalone applications that support running tests in parallel (requires MATLAB Compiler™ and Parallel Computing Toolbox). Use the directive %#function parallel.Pool in your code so that MATLAB Compiler can locate and package all of the components required for running tests in parallel. For more information, see Compile MATLAB Unit Tests.

• Starting in R2021a, you can run tests on a thread-based pool (requires Parallel Computing Toolbox) by starting a parallel pool of thread workers and then calling the runInParallel method.

  Tests to run with runInParallel on a thread-based pool are subject to these restrictions:

  • Your test and source code must use only the functionality supported by thread workers. For more information about the limitations of a thread-based environment, see Choose Between Thread-Based and Process-Based Environments (Parallel Computing Toolbox).

  • Test suites created using matlab.unittest.TestSuite.fromFile, matlab.unittest.TestSuite.fromFolder, or matlab.unittest.TestSuite.fromProject are not supported on a thread-based pool.

  • Storing test artifacts is not supported on a thread-based pool.

  • Simulink^® is not supported in a thread-based environment. Therefore, tests authored using Simulink Test™ cannot run on a thread-based pool.