Error with"Helpe​rGenerateS​peechDenoi​singFeatur​es"

To use Deep learning to denoise.

It told me that i use Tall array in wrong way. Now , i am afraid that there is nothing i can do about it .

I was trying out the Denoise Speech Using Deep Learning Networks example with some modifications. I just set the noise and dataset as local path, and deleted the very first part. Because i didn't need to play the sound that i had already known and i thought it was not as important as the next step.

the part that i changed is here

%changed
noisedataset = "D:\matlab_network\zh-CN\cv-corpus-5.1-2020-06-22\zh-CN\nosie\common_voice_zh-CN_18527375.mp3";
[noise,fs] = audioread(noisedataset);

dataset = "D:\matlab_network\zh-CN\cv-corpus-5.1-2020-06-22\zh-CN";
adsTrain = audioDatastore(fullfile(dataset,"clips"),IncludeSubfolders=true);

speedupExample = true;
if speedupExample
    adsTrain = shuffle(adsTrain);
    adsTrain = subset(adsTrain,1:1000);
end
----------------------------------------------------------------------
%STFT
windowLength = 256;
win = hamming(windowLength,"periodic");
.......

Where the error was reported

%tall数组
%-------error
reset(adsTrain)
T = tall(adsTrain);

[targets,predictors] = cellfun(@(x)HelperGenerateSpeechDenoisingFeatures(x,noise,src),T,UniformOutput=false);
[targets,predictors] = gather(targets,predictors);

%----- error
predictors = cat(3,predictors{:});
noisyMean = mean(predictors(:));
noisyStd = std(predictors(:));
predictors(:) = (predictors(:) - noisyMean)/noisyStd;

targets = cat(2,targets{:});
cleanMean = mean(targets(:));
cleanStd = std(targets(:));
targets(:) = (targets(:) - cleanMean)/cleanStd;

predictors = reshape(predictors,size(predictors,1),size(predictors,2),1,size(predictors,3));
targets = reshape(targets,1,1,size(targets,1),size(targets,2));

However, i met the problem like this. I get this error all the time and i am stuck in this problem for a long time. I get no idea to deal with it. I was wondering if anyone could give me a hand? Thank you in advance.

错误使用 tall
无法根据默认并行集群创建 mapreduce 执行环境。

出错 restart (第 93 行)
T = tall(adsTrain);

原因:
    错误使用 gcp
    Parallel pool failed to start with the following error. For more detailed information, validate
    the profile 'Processes' in the Cluster Profile Manager.
        错误使用 parallel.internal.pool.AbstractInteractiveClient>iThrowWithCause
        Failed to initialize the interactive session.
            错误使用 parallel.internal.pool.AbstractInteractiveClient>iThrowIfBadParallelJobStatus
            The interactive communicating job failed with no message.
 
   

Answer 1

[Brian Hemmat]

Based on the error message, it looks like a problem with the parallel setup. Did you try validating the profile "Processes" in the "Cluster Profile Manager"?

Another option is to bypass the usage of tall. The following modification can help:

Replace this code:

T = tall(adsTrain);
[targets,predictors] = cellfun(@(x)HelperGenerateSpeechDenoisingFeatures(x,noise,src),T,UniformOutput=false);
[targets,predictors] = gather(targets,predictors);

With this:

tadsTrain = transform(adsTrain,@(x)HelperGenerateSpeechDenoisingFeatures(x,noise,src));
allresults = readall(tadsTrain); % Possibly use the UseParallel flag to speed up--probably not necessary if you're patient
targets = allresults(:,1);
predictors = allresults(:,2);

You must also make the following change to the supporting function, HelperGenerateSpeechDenoisingFeatures.m. This change might require other modifications in the example--I haven't didn't run it all the way through.

function out = HelperGenerateSpeechDenoisingFeatures(audio,noise,src) %<<<<<<< Modify the signature to return "out"
% HelperGenerateSpeechDenoisingFeatures: Get target and predictor STFT
% signals for speech denoising.
% audio: Input audio signal
% noise: Input noise signal
% src:   Sample rate converter

% Copyright 2018 The MathWorks, Inc.

WindowLength = 256;
win          = hamming(WindowLength,'periodic');
Overlap      = round(0.75 * WindowLength);
FFTLength    = WindowLength;
NumFeatures  = FFTLength/2 + 1;
NumSegments  = 8;

D            = 48/8; % Decimation factor
L            = floor( numel(audio)/D);
audio        = audio(1:D*L);

audio = src(audio);
reset(src)

randind      = randi(numel(noise) - numel(audio) , [1 1]);
noiseSegment = noise(randind : randind + numel(audio) - 1);

noisePower   = sum(noiseSegment.^2);
cleanPower   = sum(audio.^2);
noiseSegment = noiseSegment .* sqrt(cleanPower/noisePower);
noisyAudio   = audio + noiseSegment;

cleanSTFT = stft(audio, 'Window',win, 'OverlapLength', Overlap, 'FFTLength',FFTLength);
cleanSTFT = abs(cleanSTFT(NumFeatures-1:end,:));
noisySTFT = stft(noisyAudio, 'Window',win, 'OverlapLength', Overlap, 'FFTLength',FFTLength);
noisySTFT = abs(noisySTFT(NumFeatures-1:end,:));

noisySTFTAugmented = [noisySTFT(:,1:NumSegments-1) noisySTFT];
 
STFTSegments = zeros(NumFeatures, NumSegments , size(noisySTFTAugmented,2) - NumSegments + 1);
for index = 1 : size(noisySTFTAugmented,2) - NumSegments + 1
    STFTSegments(:,:,index) = noisySTFTAugmented(:,index:index+NumSegments-1);
end

targets    = cleanSTFT;
predictors = STFTSegments;
out = {targets,predictors}; % <<<<<<<<<  Add this line. It combines the targets and predictors into a single output

[L]

Firstly, I would like to thank you for your assistance.

Question.Is 'Cluster Profile Manager' meant to accelerate computing using chips from multiple computers?

But i am a student in school and I only have only one computer. It's ok for me to wait for the result without acceleration. So is it any other ways to achieve this denoising funtion.

Why i ask the question. I've just started to use this software recently, it's stange for me and it's my first time to use tall arrary. I have no other method to achieve it. Could you provide any other ways to achieve this? (I've already tried FIR and IIR but it does not wok well )

The purpose. The reason for using this code is that I want to implement this denoising functionality into an FPGA chip, and then use it for competition.

The try in source document. I add the script file which contains "HelperGenerateSpeechDenoisingFeatures" function and then i can use this function. But get a new problem: "Too many output parameters." I've tried changing the name of the parameters but it does not work. I think the global parameters definition conflicts with parameters definition in some function.

错误使用 tall/cellfun
输出参数太多。
了解更多有关执行 GATHER 时遇到的错误的信息。

出错 rechange (第 94 行)
[target,predictor] = cellfun(@(x)HelperGenerateSpeechDenoisingFeatures(x,noise,src),T,UniformOutput=false);

出错 tall/gather (第 50 行)
[varargout{:}, readFailureSummary] = iGather(varargin{:});

出错 rechange (第 95 行)
[target,predictor] = gather(target,predictor);

The new problem after changing. It told me that "Invalid transformation function defined on data store."

i will offer the source document to you

数据存储上定义的变换函数无效。

错误的原因是:

未定义与 'dsp.SampleRateConverter' 类型的输入参数相对应的函数 'HelperGenerateSpeechDenoisingFeatures'。

出错 rechange>@(x)HelperGenerateSpeechDenoisingFeatures(x,noise,src) (第 98 行)
tadsTrain = transform(adsTrain,@(x)HelperGenerateSpeechDenoisingFeatures(x,noise,src));

出错 matlab.io.datastore.TransformedDatastore/applyTransforms (第 723 行)
                            data = ds.Transforms{ii}(data);

出错 matlab.io.datastore.TransformedDatastore/read (第 235 行)
            [data, info] = ds.applyTransforms(data, info);

出错 matlab.io.datastore.TransformedDatastore/readall (第 300 行)
                data{end+1} = read(copyds); %#ok<AGROW>

出错 rechange (第 99 行)
allresults = readall(tadsTrain); % Possibly use the UseParallel flag to speed up--probably not necessary if you're patient