Tall Array Data Fetch Optimization
2 views (last 30 days)
Show older comments
Hi everyone,
I am working on a Tall Data array backed by a Data Store, the array looks like follows;
My question is, when I try to fetch/mine data using the Data_Point as in the following (tdsDB is the Tall Array and dataPointIndxFirst and dataPointIndxLast parameters are set accordingly):
app.xxyydata = gather(app.tdsDB(app.tdsDB.Data_Point>=dataPointIndxFirst &...
app.tdsDB.Data_Point<=dataPointIndxLast,:));
but this takes several minutes to compute, but Data_Point is unique indexing value which increase along the data rows. And the Data_Point values start from 97137237, not from 1.
Compute time is around 5 mins even the parameters are set like: dataPointIndxFirst = 97137237 and dataPointIndxLast = 97137238 (very next point of dataPointIndxFirst).
My guess is MatLab searches through the whole Tall Array, which is not the optimal way. Is there a way that I can tell MatLab this Data_Point is arranged in an increasing order and acts like (or can be treated as) an index?
Any help to optimize compute time is appreciated....many thanks in advance.
0 Comments
Answers (0)
See Also
Categories
Find more on Tall Arrays in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!