How to handle 1 GB - 2GB .txt files efficiently in matlab, I used textscan. Is it efficient?

2 views (last 30 days)
Please let me know with any alternative methods to proceed with.
thanks

Answers (3)

Walter Roberson
Walter Roberson on 22 Jan 2014
What do you need to do with the text file? Is it a text file that consists just of numbers, or does it have additional information? Do you want to read in all of the numbers in it or just some columns?
How often will you need to process any given text file? Sometimes pre-processing it outside of MATLAB to make it easier for MATLAB to read can be time-efficient, and the relative efficiency goes up the more often you re-use the same file.
  2 Comments
Aditya
Aditya on 22 Jan 2014
text file consists of all kinds of data like char,numbers,special chars. and I deal with retrieval of data in the text file. Help me with this context.
Walter Roberson
Walter Roberson on 22 Jan 2014
Could you give an extract of the file and indicate what you would want retrieved from it?
Also please respond about how often you will need to process any given file.

Sign in to comment.


Aditya
Aditya on 22 Jan 2014
Edited: Aditya on 22 Jan 2014
POINT X1 X2
POINT -124.63-78.433 .... ....
above is how file exists and I need to extract required POINTS.
  1 Comment
Walter Roberson
Walter Roberson on 22 Jan 2014
Are there lines that start with anything other than POINT ? Is the first line of the file the only one that says
POINT X1 X2
or does that occur every second line? Or does it appear after a group of numeric POINT objects, with those values to be grouped together (e.g., a face of an object) ? Do the numbers in "X1 X2" vary and if so are they significant or can they be ignored?
The numeric values you show all run together, and have variable number of decimal places. Are all of the values negative so there will always be a "-" between the values? If not then do the positive values have "+" or a space before them?

Sign in to comment.


Aditya
Aditya on 22 Jan 2014
I just want to search for the respective point and retrieve the data. Only once the file is used and is stored. I did using textscan - cellarrays.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!