Import data with filters

% How can I import some data in a table (import with filters)

Name | Age

Hugo|30

Paco|40

Luis |50

Gus|60

% I need import in a table only person with age >= 50

Answer 1

[Avni Agrawal]

Hi @julios,

I understand that you are trying to import data into a table in MATLAB and apply a filter to only include rows where the age is greater than or equal to 50, you can follow these steps. Assume you have your data in a text file named `data.txt`.

Here's how you can achieve this:

• Use `readtable` to import the data into a table.
• Use logical indexing to filter the rows where the age is greater than or equal to 50.

Here's a sample MATLAB script to do this:

% Read the data from the file
data = readtable('data.txt', 'Delimiter', ',');

% Apply the filter for age >= 50
filteredData = data(data.Age >= 50, :);

% Display the filtered table
disp(filteredData);

      Name      Age
    ________    ___

    {'Luis'}    50 
    {'Gus' }    60 

• readtable('data.txt', 'Delimiter', ',')`: This function reads the data from a CSV file and creates a table. The `Delimiter` parameter specifies that the data is comma-separated.
• data(data.Age >= 50, :)`: This line uses logical indexing to filter the rows. `data.Age >= 50` creates a logical array where each element is `true` if the condition is met and `false` otherwise. The `:` selects all columns for the filtered rows.
• Make sure the data file (`data.txt`) is in the current working directory or provide the full path to the file. Adjust the file name and path as necessary to match your data source.

I hope this helps!

[julios]

Thank you very much, but is it possible to import only specific information? in other words, import only the rows that pass the filter; in this caso import only

{'Luis'} 50

{'Gus' } 60

the problem is, my data have many rows that is no neceasary and with the steps shared, first need import all the rows and in the second step erase information innecesary. With this process the time to read is considerably

With this process the time to read is considerably and i need optimice time to process

[Walter Roberson]

In the special case that the constant data to be matched against happened to be in the leading column: it just might be possible to convince textscan() to do appropriate filtering. It would not be easy... you might have to define everything to be ignored as CommentDelimiter for example.

Otherwise, you would need to fopen() the file, and loop fgetl() and matching the text on the line to see if the line is one you want to keep, and record it if so. This process would be noticably slower than textscan() of everything and later discarding what you do not want. You would use the fgetl() loop for more complicated input formats, or for the case where you just cannot afford the memory burden of reading everything followed by filtering.

[julios]

Thank you all for your asistance, I understand at this moment is more easy read all information, in a second step erase all that I don't need it. Hope in the future we have this capability only for eficient lecture of the large files. Have a great weekend

[Walter Roberson]

If you have a regular table of data, and it is a large file, and you need different sub-sections of it at different times, then you could consider switching to using a database.

[Eric Sofen]

If your data are in a Parquet file, you can filter without importing all the data using rowfilter. That said, rowfilter can also be used on a table imported using readtable, but you do end up reading the whole file because with something like a CSV there isn't a good way to preemptively know where the age >=50 values exist.