Sorting List of Filenames according to Windows

I have a list of files inside a directory and I would like to sort them with Matlab like Windows sorts the files.

For example, this is a list of files sorted according to Windows:

• a_.txt
• a1.txt
• a2.txt

While this is the list sorted according to the Matlab sort function:

list = {'a_.txt'; 'a1.txt'; 'a2.txt'};

sort(list)

• a1.txt
• a2.txt
• a_.txt

How can I sort the files in Matlab like Windows?

[Rik]

You could also define your own sorting rules, which can then try to implement. (e.g. [a-zA-Z] comes after special symbols and numbers should be sorted such that 8 is before 10)

[Stephen23]

@Rik: note that the "special symbols" are not sorted into character code order by Windows Explorer (which is presumably what the OP is referring to by the term "Windows"). This is the order that I get using single-character folder names in Win10:

>> +'''-!#$%&(),;@[]^_`{}~+=' % Folder names copied by hand using the order shown by File Explorer.
ans =
    39    45    33    35    36    37    38    40    41    44    59    64    91    93    94    95    96   123   125   126    43    61

And this is just a subset of the ASCII characters... what about all of the non-alphabetic Unicode "special symbols", where is this order defined?

[Rik]

I didn't mean to suggest that was a complete description of how Windows Explorer sorts file names. I didn't even mean to suggest it was an incomplete description. I meant it as an example of how you could describe a sort order.

Answer 1

[Walter Roberson]

Windows explicitly does not define a sorting order for files. It leaves it up to the filesystems -- so the sort order for FAT32 might be different than NTFS for example.

I have not come across any formal definition of the sort order for NTFS.

I find a claim that NTFS captures some case-mapping information from the locale of the user at the time the disk is formatted, https://devblogs.microsoft.com/oldnewthing/20050617-10/?p=35293 and uses that, and otherwise sorts by UNICODE code point. MATLAB sorts strictly by UNICODE code point.

The implication is that for the same list of files on two different NTFS drives, the sort order could be different.

Some people do encounter _ sorting before letters, but other people do not encounter that. https://stackoverflow.com/questions/29734737/findnextfile-order-ntfs

[Simone Croci]

Thank you for your answer!

Is it possible to read the file order from the Window filesystem in Matlab?

[Walter Roberson]

I am not clear what you are asking for?

MATLAB does not change the order from what it is handed by the operating system. The operating system does not change the order from what it is given by the file system. According to the above discussion, NTFS is not consistent from disk to disk about what order is used, but (supposedly) is consistent for any one disk, with the order (apparently) determined at the time the drive is formatted. In short: MATLAB is giving back whatever is on disk, but the order on disk could be different for different disks.

If you are asking to read the tables that NTFS uses to determine what sorting order to use for any one drive, there is certainly no Mathworks-provided call to do that task. Possibly it would be possible by using a .NET assembly to query the drive file system. That is not a task I have ever run across anyone happening to do, but perhaps information about it could be found online.

[Simone Croci]

My goal is to get the list of files ordered according to the file system. I have tried the Matlab function dir but it returns the same list of sort({'a_.txt'; 'a1.txt'; 'a2.txt'}), which is different to the list of the file system.

Is it possible to read the list of files ordered according to the file system in Matlab?

[Walter Roberson]

Is it possible to read the list of files ordered according to the file system in Matlab?

As I explained, MATLAB already does that, because it does not change the order it gets from the operating system (which in turn does not change the order it gets from the file system.)

But then you are applying sort() to the result, which is giving you a different order than what the filesystem has. If you skip the sort() then you would be working with the order returned by the filesystem.

[Simone Croci]

Probably, I wasn't clear enough. Assuming I have some files in a directory, how can I read them in Matlab and obtain a list of files sorted like the Windows file explorer sorts them when I click on the "Name" column?

I have tried with the MATLAB function dir('c:/folder'), but I don't get the same sorted list of files.

[Stephen23]

"...how can I read them in Matlab and obtain a list of files sorted like the Windows file explorer sorts them..."

As Walter Roberson explained, MATLAB returns the files in the order proved by the file system. The Windows File Explorer is NOT the file system, these are two totally different things. The file system has no idea how some random application (e.g. Windows File Explorer) sorts filenames.

The order provided by Windows File Explorer is not documented (as far as I am aware). If you find its order documented somewhere, paste the link here and someone can probably help you to sort files in the same way.

[Simone Croci]

Thank you for your reply... I was confusing the Windows File Explorer with the file system...

[Walter Roberson]

The Windows Explorer sort order is complicated. Are you sure that is what you need? For example are you sure you would want the order

1. "0000000"
2. "0"
3. "00001"

??

https://stackoverflow.com/questions/4290541/what-is-the-first-character-in-the-sort-order-used-by-windows-explorer

Note too the parts of the response that say specifically that

1. It depends what locale the user has set; and
2. It is subject to change between releases (and people say it has changed a few times.)

[Paul]

Walter,

One of your comments says:

" MATLAB sorts strictly by UNICODE code point. "

and another says:

"MATLAB does not change the order from what it is handed by the operating system."

Are these statements saying the same thing? Or am I misinterpreting the context of one or both?

I too was very recently interested in the return order from dir and found nothing in the doc that stated what that order is. So I resorted to converting dir.name to a string array and sorting on that, which sorts according to Unicode. Not what the OP wants, but worked for me.

Also, FWIW the the MWST says in this very old thread that dir returns files in ASCII dictionary order.

[Walter Roberson]

dir() does not apply any sorting to what it receives from the operating system. sort() is strictly by UNICODE Code Point.

It is easy to disprove that dir() sorts by ASCII dictionary order:

>> !rm *; touch aB Ba
>> dinfo = dir('*'); {dinfo.name}
ans =
  1×4 cell array
    {'.'}    {'..'}    {'Ba'}    {'aB'}

ASCII dictionary order would require that a sort before B . ASCII dictionary order is "upper case letter" followed by "lower case version of letter".

Furthermore, since ASCII ends at char(127), ASCII dictionary order would imply that there is no sort order for anything beyond char(127).

>> !rm *; touch α σ δ φ γ η
>> dinfo = dir('*'); {dinfo.name}
ans =
  1×8 cell array
    {'.'}    {'..'}    {'α'}    {'γ'}    {'δ'}    {'η'}    {'σ'}    {'φ'}
>> cellfun(@double,ans,'uniform',0)
ans =
  1×8 cell array
    {[46]}    {1×2 double}    {[945]}    {[947]}    {[948]}    {[951]}    {[963]}    {[966]}

Those are obviously sorted by Unicode code point; ASCII sort order would leave them out of order.

(There is a flaw in that last argument of mine; can you see it?)

[Paul]

Thank you for the clarification. I misread your first comment as a statement on how dir sorts its output.

Not even going to try to find a flaw in your argument ;).

[Simone Croci]

Walter Roberson,

Yes, I am trying to sort some files in a folder according to the Windows file explorer. The reason is that I have an application that processes the files inside a folder according to the order of the Windows file explorer, and I have a second application that needs to process these files in the same order but has no access to the Windows file explorer... for the second application, I need to provide as input the sorted list of files...

[Rik]

The order provided by Windows File Explorer is not documented (as far as I am aware). If you find its order documented somewhere, paste the link here and someone can probably help you to sort files in the same way.

[Walter Roberson]

How does the application sort the files according to Windows File Explorer order? That order is not the same as the application would be told when it asks the operating system for the directory of files.

[Walter Roberson]

Rik:

In part:

https://getadmx.com/?Category=Windows_10_2016&Policy=Microsoft.Policies.WindowsExplorer::NoStrCmpLogical

See https://superuser.com/questions/415629/sort-order-in-windows-explorer

Implementation is apparently https://docs.microsoft.com/en-ca/windows/win32/api/shlwapi/nf-shlwapi-strcmplogicalw?redirectedfrom=MSDN

As described in https://stackoverflow.com/questions/4290541/what-is-the-first-character-in-the-sort-order-used-by-windows-explorer

[Stephen23]

Knowing the character order is still not sufficient: Windows File Explorer also does some kind of number matching as well, and sorts taking into account the number values. But what counts as a number? Decimal comma or point or both? What effect does the locale have on this? Positive/negative sign? What precision? Is there any other special matching of multiple characters? So many questions...

Answer 2

[U Arda Demiro]

Apparently someone at Matlab fixed this problem, but only in a way that causes other problems. Here is the problem I am having now. Suppose you have four files as follows:

prog1.m

prog1_.m

prog1_5digit.m

prog1_7digit.m

The above order would be how Windows sorts them. Matlab instead currently sorts them as follows:

prog1_.m

prog1_5digit.m

prog1_7digit.m

prog1.m

Fairly annoying.

(Version R2017b, this message: October 7 2020.)

[Walter Roberson]

I have not seen evidence that MATLAB sorts the names it gets back from the file system calls.

Anyhow, the way to get what you are asking for would appear to be to write some mex to use https://docs.microsoft.com/en-ca/windows/win32/api/shlwapi/nf-shlwapi-strcmplogicalw to sort the names.

However, the links I have presented show that the sort order is not constant and can vary according to the user's settings. The order used by your application can be different.

I would recommend repairing the application that depends upon a fragile sort order.