numpartitions

Return estimate for reasonable number of partitions for parallel processing

Syntax

n = numpartitions(ADS)

n = numpartitions(ADS,pool)

Description

n = numpartitions(ADS) returns the default number of partitions for the datastore, ADS. The default number of partitions is the total number of files.

n = numpartitions(ADS,pool) returns a reasonable number of partitions to parallelize ADS over the parallel pool, based on the total number of files and the number of workers in pool. To parallelize datastore access, you must have Parallel Computing Toolbox™ installed.

Examples

Estimate Reasonable Number of Partitions for Audio Datastore

numpartitions returns a reasonable number of partitions for an audio datastore. You can use numpartitions as input to the partition function.

Specify the file path to the audio samples included with Audio Toolbox™. Create an audio datastore that points to the specified folder.

folder = fullfile(matlabroot,'toolbox','audio','samples');

ADS = audioDatastore(folder);

Use numpartitions to estimate a reasonable number of partitions for the audio datastore, ADS. By default, numpartitions returns the number of files the audio datastore points to.

n = numpartitions(ADS)

n = 34

Number of Partitions for Parallel Datastore Access

Partition a datastore to facilitate parallel access over the available parallel pool of workers.

Specify the file path to the audio samples included with Audio Toolbox™. Create an audio datastore that points to the specified folder.

folder = fullfile(matlabroot,'toolbox','audio','samples');
ADS = audioDatastore(folder);

Return an estimate for a reasonable number of partitions for parallel processing, given the current parallel pool.

pool = gcp;
n = numpartitions(ADS,pool);

Partition the audio datastore and read the data in each part.

parfor ii = 1:n
    subds = partition(ADS,n,ii);
    while hasdata(subds)
        data = read(subds);
    end
end

Input Arguments

ADS — Audio datastore

audioDatastore object

Specify ADS as an audioDatastore object.

pool — Parallel pool

parallel pool object

Parallel pool object.

Output Arguments

n — Number of partitions

positive integer

Number of partitions to parallelize datastore access over.