How to re-bin the 2D data?
Show older comments
Hi everyone,
I've 2D data, first column (X) is the time and second column (Y) is the corresponding data. It is easy to re-bin the X data as the data values are linear, but could not figure out the way to re-bin the Y data as it is exponentially decaying. Attached screenshot is a piece of data. How can I re-bin the Y data so that I can pull out Y values for each re-binned X data accurately?
X = data(4:214, 1);
Xmax = max(X); Xmin = min(X);
N = 106; % no. of bins I want, this will give me bin size of 0.1 (say)
dy = (Xmax - Xmin)/ (N-1); % N-1 will be clear in the next line
Xedges = Xmin - dy/2 : dy : Xmax + dy/2;
Xedges = Xedges'; % change row to column matrix (transpose)
% re-bin Y data so that I can pull out Y values for each re-binned X column
Y = data(4:214, 2);
Ymax = max(Y); Y1min = min(Y);
N = 106; % no. of bins we want
.... % not sure how to proceed ahead
Accepted Answer
You can use Y = discretize(X,edges) to bin the X data and use the bin index to group the Y data into bin categories. You probably cannot assume that there will be an equal number of data points in each bin so the grouped y-data must be stored in a cell array.
Here's a demo using your variable names.
[bins, Xedges] = discretize(X,N); % Xedges are not used here
Y = data(4:214, 2);
yBinned = arrayfun(@(i)Y(bins==i),unique(bins),'UniformOutput',false);
yBinned is an n-by-1 cell array where yBinned{j} are the Y values in bin number j whose edges are defined by Xedges([j,j+1]).
10 Comments
Could you tell me what the following two outputs are?
ver() % specifically, I'm looking for your matlab version (eg: r2019b)
and
which discretize -all
Futhremore, if you either attach a mat file containing the "data" variable or paste the content of that variable as a comment in such a way that I can very easily copy it into my command window, I could apply my idea to your data.
The 2nd output to discretize() was added in r2016b so I'm guessing your version of Matlab is prior to that. The 2nd input will likely need changed, too, since prior to r2016b you had to provide the edges. In that case, you can use your Xedges variable like this
bins = discretize(X,Xedges);
blues
on 30 Dec 2019
"...seems working but it gives the output in differnet form."
Looking at the r2016a documentation, the output should be the same form
"How to modify this line such that I can get a mean value?"
Instead of modifying that line, it will likely be useful to keep those grouped data. You can compute the mean of each group (each element of the cell array) using
yMean = cellfun(@mean,yBinned);
If your data has NaNs that you'd like to ignore,
yMean = cellfun(@(x)mean(x(~isnan(x))),yBinned);
blues
on 30 Dec 2019
"Now code is working and obtained the data values. "
Great!
" Do you see the problem in the following lines of code? "
Hmmm, it's not immediately obvious to me just by looking at the code. I could probably see the difference fairly quickly if I had your data so I could run each version and look at the outputs.
If you can't attach the data and you'd rather troubleshoot it on your own, I'd start by comparing the binEdge values between the two methods; if they match I'd then compare the bin index values.
There isn't enough information available in your attachment for me to compare the two versions.
For example, I don't know what your Xedges values are. Also, there's only 211 rows of data in your attachment but your code references row 214.
If you attach 1) a mat file containin the data needed for me to run the code and 2) the two sections of code you're using; I can look into the differences.
blues
on 30 Dec 2019
Adam Danz
on 31 Dec 2019
"You can get Xedges from the code that I attached before"
Actually, I couldn't, because it extended to row 214 but your data only had 211 rows. I didn't know anything about headers etc until your previous comment.
"I don't have a mat file."
It would take you less than 1 minute to create one using the save() function. By sharing a mat file, we know we're using the exact same data. But when you give the csv file, it requires me to read in the data and there are multiple ways to do that so we could end up with slightly different values. Also, it takes more time for us volunteers to read-in data. The idea is to make it easy for us to help you and to make sure we're all looking at the exact same thing.
I copied your data from the csv file into the command window and named the variable "data". Then I ran the following two sections of code. The first section produces the variable yMean and the second section produces the variable yMean2. Then I compare the values.
X = data(1:211, 1);
Xmax = max(X); Xmin = min(X);
N = 106; % no. of bins we want
dy = (Xmax - Xmin)/ (N-1); % N-1 will be clear in the next line
Xedges = Xmin - dy/2 : dy : Xmax + dy/2;
Xedges = Xedges';
% VERSION 1
bins = discretize(X,Xedges);
Y = data(1:211, 2);
yBinned = arrayfun(@(i)Y(bins==i),unique(bins),'UniformOutput',false);
yMean = cellfun(@(x)mean(x(~isnan(x))),yBinned);
% VERSION 2
[n,bins2] = histc(X,Xedges);
yMean2 = accumarray(bins2,Y,[],@mean);
% COMPARE VERSIONS
isequal(bins,bins2) % = TRUE; so they are the same
isequal(yMean,yMean2) % = TRUE; so they are the same
As you can see, the two sections produce the same outputs. If you are getting different values it could be due to any of the following reasons
- The inputs in your code are different between the two versions.
- Your versions don't match my versions.
- I'm using r2019b and you're using r2016a. I doubt this is the problem.
More Answers (0)
Categories
Find more on Logical in Help Center and File Exchange
