Exclude System or Hidden Folders from Mac Dir List
Show older comments
Here is the attributes for the System Volume Information directory on a Mac disk.
How can I get a Dir listing of all folders on a Mac drive but exclude hidden or system folders.
Note (see 2) ... System Attribute for a System file is Nan... So, if I cannot use the attribute function how can I establish (reliably) what are system or hidden files.
AVF_INFO and 'System...' are hidden directories . DCIM and PRIVATE are 'normal' dirs I expect to find in a dir list.
Note 2. I have already eliminated lots of directories which I also needed to exclude from my Dir list.
MyDirList = HD_FullList([HD_FullList.isdir]); %remove files from list
MyDirList=MyDirList(~ismember({MyDirList.name},{'.','..'}));% remove . & ..
MyDirList=MyDirList(~ismember({MyDirList.name},{'$RECYCLE.BIN'}));% remove recycle bin
MyDirList=MyDirList(~ismember({MyDirList.name},{'.Trashes'}));% remove trash dirs
MyDirList( startsWith({MyDirList.name}, '.') ) = []; %exclude hidden files ie starting with .
And excuse me for venting a little rant here.... Why is it so ridulously difficult to get dir lists of files and directories on Mac and Win. This is excruciatingly painful... and seriously error prone. This might be my last MatLab project.
25 Comments
I'll join in a little of the rant -- I've never understood and always thought (and submitted enhancment requests multiple times years and years ago before finally giving it up as lost cause) dir() should honor the underlying OS switches to give control to the user; ideal of "platform-independent" be d---d. If the user has to deal with multiple platforms, that's part of their need/duties to deal with; give the user the tools to do so.
In general, if I do need more (or less) than what the normal dir() will retrieve; I'll just pass the command to the underlying OS and parse the result instead.
I "know nuthink!" per Sgt. Schultz about the Mac so can't comment further on specifics for it...
Matt O'Brien
on 22 Apr 2023
Moved: Walter Roberson
on 22 Apr 2023
Walter Roberson
on 22 Apr 2023
What is a "system" file or folder on MacOS ?
Matt O'Brien
on 22 Apr 2023
Walter Roberson
on 22 Apr 2023
Notice the DCIM directory. That is a memory card for a digital camera. The standards say that camera memory cards will be formatted by one of DOS FAT file systems. "system volume information" is not generated by MacOS, it is there as part of the DOS file system. MacOS does not offer "system" files or directories.
If you are expecting MATLAB to populate Windows file system attributes for a Windows file system mounted on a Mac, I am not sure that is a reasonable thing to expect.
Matt O'Brien
on 22 Apr 2023
You are correct .... these are SD cards, which conform to DCIM standards, used globally by the camera / video industry, which includes Fat and ExFat standards. Millions of users around the world are using these cards in Macs and Windows machines, directly in a Mac SD reader or via backups of these cards taken in the field. It is an outdated standard, I strongly would be in favour of this standard been upgraded, but we can only use what we have.
There are large number of MatLab users reading their image files from such cards.
The following screen grab is from a Mac screen.
Note that Mac treats the System / Hidden files differently (faded font in the Yellow boxes) ... so the Mac OS knows the difference.
I am not asking that MatLab should recognise and deal with Windows file attributes ... but MatLab should recognise and allow an app to distinguish between these directory / file types.
If MatLab are not prepared to do that then it reinforces in my mind that MatLab is not a place I want to invest time and money.
I will accept a workaround... but so far I have not got one.
I have posted a support case to MatLab. Lets see what MatLab say...
I am currently hard coding my app to deal with this .... but that is guaranteed to break down at some point in the future .. so it would be immoral for me to share my apps as a result.
I do appreciate that you have taken the time and effort to respons and I am grateful for that.
What does the MacOS system 'ls' return for those files? Unless it returns some attribute or has some switch to differentiate them, then you're pretty-much hosed independently of MATLAB.
The MATLAB dir() function on Windows does return hidden/system files whereas the normal OS CMD dir does not; the "/a:hs" switches are negated by default for those two attributes whereas TMW has chosen to set them for their internal dir() function. That has never made any sense to me; particularly when they won't pass a switch and then don't return the attributes field to let one screen by. Of course, Walter just reminded me a day or so ago that even the .Folder property is a moderately recent addition as it came up in another Q? of a poster using R2013x and running into that problem.
I don't know what MacOS does, but I would think the workaround has to be to use the OS-supplied functions and that you'll run into the same issue on whatever platform you develop if it doesn't identify those as eventually it gets passed down to the underlying OS APIs that are fundamental.
As noted, I began a crusade some 30 years ago by submitting enhancement requests; after 10 years of no action, finally just gave up that quest as hopeless. But I still fail to see why TMW is so apparently klewless on this front...
dpb
on 22 Apr 2023
"I will accept a workaround... but so far I have not got one."
The obvious workaround is to use the underlying OS equivalent -- as noted above, however, unless it can tell the difference, then you're pretty-much out of luck no matter what; the differences in the above screen image would seem to imply there is some attribute that is distinguishable or else the mounted file system has installed its own extension into the UI as well and is bypassing the standard OS API.
I never used a Mac and have never done anything with the digital camera stuff even on Windows, so I've not direct knowledge about what/how it is done, but anything you do inside or outside of MATLAB would seem to have similar issues no matter what programming language you choose.
Creating and Handling Invisible Files
In macOS, you make a file invisible to the user by prepending its name with a period character. Adding this character signals to the Finder and other display-related interfaces that they should treat the file as invisible under normal conditions. You should use this capability sparingly, though, and only in very specific cases. For example, the process for localizing a custom directory name involves placing hidden files with the localized names at the top-level of the directory. You should never use this technique to hide user documents or other data your app creates.
Even if you mark a file as invisible, there are still ways for the user to see it. Users can always view the true file system contents using the Terminal app. Unlike the Finder, the command line in the Terminal window displays the actual names of items in the file system and not their display names or localized names. You can also show hidden files in the Open and Save panels (using the setShowsHiddenFiles: method) in cases where you want the user to be able to select them for viewing or editing.
Regardless of whether a file contains a leading period, your app’s code can always see hidden files. Methods that enumerate the contents of directories generally include hidden files in the enumeration. The only exceptions to this rule is the meta directories . and .. that represent the current and parent directories. Therefore, any code that enumerates the contents of a directory should be prepared to handle hidden files and deal with them appropriately, usually by just ignoring them.
From the above, I presume that probably the particular files of interest may begin with the dot in the filename for hidden files; not sure about system files although I presume they would also be hidden.
Good luck, but looks to me like you'll have similar issues no matter what you use to write the application in; doesn't appear this is anything that will be unique to MATLAB although I still agree they could supply more regarding what switches are set/not and let the user control). The ls man page has the list of switches; looks as though it should be feasible to return the limited list desired and/or use the annotation of the type. The big issue with unix-like filenames is that such abominations as \n are allowable legal characters in a filename so you can't rely 100% on parsing a text listing since it uses \n to separate records -- and one of those could just have been part of a filename--Catch 22!
Matt O'Brien
on 22 Apr 2023
Walter Roberson
on 22 Apr 2023
Hiding dot files is a convention, not something enforced by the OS.
MacOS HFS+ and APFS file systems support "flags" ( https://www.maketecheasier.com/use-file-flags-modify-file-behavior-macos/ ) that can be set with chflags https://ss64.com/osx/chflags.html -- but not queried with that command. At the moment all I see for querying the flags is ls -lO (lower-case L, capital Oh), but I would imagine there are other ways to get the flags. Ah, I see they can be read with stat see https://unix.stackexchange.com/questions/640422/api-syscall-to-read-or-list-bsd-macos-file-flags
The supported flags do include hidden but not system
It would not be unreasonable for a MacOS filesystem layer reading a Windows system to examine the windows attributes and fill in the hidden attribute of the MacOS stat() call, but MacOS does not have a system attribute in stat(), so in order to code the system attribute MacOS would have to go even further afield, such as synthesizing an extended attribute https://ss64.com/osx/xattr.html .
I would suggest that even if MacOS went to that trouble, that expecting MATLAB's dir() call to examine that kind of information would be much the same as expecting MATLAB's dir() call to fill out the USERREAD USERWRITE USEREXECUTE fields based upon interpreting Access Control List https://www.techrepublic.com/article/introduction-to-os-x-access-control-lists-acls/ . For example some might say that for a file, USEREXECUTE should be false if the file has com.apple.quarintine associated with it. But other people would say that the results of dir() should reflect only the basic attributes associated with the file -- and on MacOS, going to look in the flags for hidden or (hypothetically) looking in extended attributes to see if some kind of system attribute has been synthesized is not basic attributes associated with a file.
Question: is there a standard for how third party filesystem drivers such as FUSE (for Linux ext* filesystems) or Paragon's NTFS drivers should import non-MACoS filesystem flags and attributes and access control lists? A standard that Mathworks could follow so that it always gets the flags in dir() "right" ?
dpb
on 22 Apr 2023
From what I saw, looks like #5 above is going to be an issue regardless how you attack it; that you're working in MATLAB seems to be irrelevant to the issues you're running into.
If you can't find some other code hacks to handle such on someplace like GitHub or StackOverflow, you may have to resort to native code via mex or somesuch route. However, it seems unlikely this hasn't been solved before as much of a platform for image processing is Mac.
One other link I just found that might be of some use <ImageCaptureCore> on the Apple developer site...I've not looked into it at all that may be that toolset you're looking for, I don't know.
Matt O'Brien
on 22 Apr 2023
dpb
on 22 Apr 2023
OBTW...just a thought -- since apparently the mounted DICOM file system is DOS-based, you may be in luck in that the \n conundrum can't bite you for these folders because it isn't allowable under that file system.
My only experience w/ Linux was 30 years ago now with a Foxboro IA control system upgrade to Unit 9 at the TVA Kingston Fossil Steam Plant; supported/converted a realtime heat rate calculation monitoring system from VAX to run on then NT4; it was virtually all Fortran with some low-level C for the communications. I remember even then "issues" dealing with the file system on the Silicon Graphics machine it was built around...from VMS, the difference was jarring to say the least! I never had reason to deal with anything more in that line and can't say as I feel I've missed anything... :)
Anyways, good luck! Looks like you'll need it, but I don't think your problems are really MATLAB-related, but idiosyncracies of the OS.
Matt O'Brien
on 22 Apr 2023
Edited: Matt O'Brien
on 22 Apr 2023
We're not that far apart, then although I predate you by probably about 10 years or so...first mainframe was a Philco 2000; the class scientific computing machine of its era -- no disk or durm, but 27(!) 7-track tape drives. Fortran II and assembler for it running the proprietary versions of the nuclear design codes from Bettis for a commercial nuclear power vendor. The Philco was already in the process of being retired when I arrived but it took over two years to get the CDC 6600 installed and able to run all the needed codes before could pull the plug on the old Philco. Other than the commercial timeshare still in Philly, I think it well may have been the last one in a production environment. Fairly shortly after it was replaced by a Cyber 76...after only a couple years with the latter, I moved on to the consulting gig and the other end of embedded/standalone systems, mostly around the M6809E running Forth. Quite a number of various specific-purpose "instruments" built with this configuration from loose-parts monitoring to neutron-noise systems looking for things like inner liner vibration/movement, etc., ...
Walter Roberson
on 23 Apr 2023
Note that Mac treats the System / Hidden files differently (faded font in the Yellow boxes) ... so the Mac OS knows the difference.
At most MacOS knows about hidden, not about system.
'System Volume Information' is a directory included when I use Dir on a Mac. I assume this is both a Sytem file and perhaps hidden file.
System Volume Information is a Windows artifact; on Windows it is system folder (possibly hidden too, not sure right now) used to contain restore points.
Are you expecting that the MacOS built-in FAT filesystem drivers will convert the DOS system attribute into some kind of flag or attribute or extended attribute that MATLAB should know to check? If so, then what are you expecting that MacOS already converts the Windows system attribute into, that MATLAB is being negligent in checking? Or are you expecting that MATLAB should have its own FAT filesystem drivers, that it should detect that it is a Windows filesystem and should do its own block-level reading of the files and interpret the binary attributes so that it can fill in the system attribute for the dir() call ? Or are you saying that Mathworks has been irresponsible in not already having (successfully) pushed Apple to convert the DOS FAT hidden attribute into something that MATLAB could have gotten ahold of for the purposes of filling out the dir() information ?
Matt O'Brien
on 23 Apr 2023
Walter Roberson
on 23 Apr 2023
Why is it so ridulously difficult to get dir lists of files and directories on Mac and Win. This is excruciatingly painful... and seriously error prone. This might be my last MatLab project.
Use the ** directory specification, such as
dinfo = dir('/Volumes/Card52312/**');
This will return everything from Card52312 and later, all in one list. dinfo.folder will give the names of the containing folders and dinfo.name will give you the names inside the folder, and dinfo.isdir will tell you whether the entry is a directory or not.
Is dir() a flexible function that will allow you to filter the returned information based on various criteria? No, it is not. It is closer to the unix stat() call, which returns a list of whatever is there, expecting you to do any post-processing that might be appropriate.
Is MATLAB being especially negligent in not returning information that other programming languages makes readily available? No: C++ (2017) std::filesystem does not provide any facility for searching directories for folders/files with particular attributes, and the set of permissions supported by C++ does not include system or hidden or any analogues; https://en.cppreference.com/w/cpp/filesystem/perms
People who need filtered file information on Unix-type systems code the search themselves, or find a pre-written library that will do the search for them -- or use the standard Unix utility find https://ss64.com/osx/find.html . For example,
find . -flags +hidden -print
./DCIM/114NIKON/._DSCN2398.JPG
./DCIM/114NIKON/._DSCN2399.JPG
./DCIM/114NIKON/._DSCN2406.JPG
./NIKON001.DSC
./._.Trashes
./.Trashes
./.fseventsd
./System Volume Information
find . -type d -flags +hidden -print
./.Trashes
./.fseventsd
./System Volume Information
... which still does not permit you to look for system
(Side note: Using ** or using the find utility might potentially cause MacOS to "materialize" dataless files and directories; See https://github.com/fish-shell/fish-shell/issues/8399 )
dpb
on 25 Apr 2023
"Is dir() a flexible function that will allow you to filter the returned information based on various criteria? No, it is not. ..."
That it is what it is, is certainly true, but that it doesn't at least pass down the available equivalent OS common switches isn't much in keeping with the "rapid development" paradigm that's supposed to be MATLAB's forte and reason to exist...that one can parse the results oneself is also certainly true, but one can also rewrite a sort algorithm from scratch every time, too; but why should one have to do so?
Walter Roberson
on 25 Apr 2023
Edited: Walter Roberson
on 25 Apr 2023
There may be room for a findobj() type of operation... but dir() was never designed to be that. I would have a hard time coming up with any computer language that had those kind of operations built in.
Matt O'Brien
on 25 Apr 2023
Edited: Matt O'Brien
on 25 Apr 2023
I did a very quick Google search for options to identify hidden files in Swift / Xcode (ie native App on Mac).
On my first search I found the following .... I highlighted the most interesting sections of this example.
I cannot be sure this will work... but it looks promisiong.
At this stage, I am not spending any more time on this. I will put my use of MatLab on hold... but expect I will abandon MatLab completely.
I will also warn my colleagues and interested parties in relation to this major black hole when trying to process image data from Media cards. My purpose in writing the app was to be able to help others manage high volume image processing. As I know how to manage this stuff myself I see no point in providing an app which is doomed to failure in due course. I can hard code my app if I wish to deal with cameras I use myself to test if I have identifed and coded for all system and hidden files.
I am going to take a rest from this.
My next step will be to see if I can find a Mac Utility which will identify hidden / system files, but I have spent way too much time on this (and learning MatLab) already.
Thanks to Walter for his time and efforts.
Regards.
I yet fail to see why/how this is actually related to MATLAB per se, though.
Can one call SWIFT from C? If so, then a mex gateway should be doable...
But, if you use/move the app to SWIFT for Apple, what do you do for everything else -- would take multiple apps to support alternate platforms; afaict SWIFT doesn't port to Windoes...
@Walter Roberson -- dir() has to eventually devolve to the underlying OS filesystem interaction API calls, anyway, nothing other than using the same switches as does the CMD(*) "dir" or Linux "ls" would be all I'm asking for.
(*) Is TMW ever going to recognize powershell, I wonder? Although it's dreadfully arcane, it makes a lot more features available, it seems. It's always been another frustration that they ignore the COMSPEC environment variable so if use another default command processor, one doesn't get it instead but CMD; one has to launce the other shell explicitly first.
Matt O'Brien
on 25 Apr 2023
Edited: Matt O'Brien
on 25 Apr 2023
I don't have a non-Windoes OS to illustrate with, but the format should suffice to see syntax.
>> [~,s]=system('dir /a:h /b c:\*.sys')
s =
'hiberfil.sys
pagefile.sys
swapfile.sys
'
>>
For a variable string use something like (be sure to be careful to keep embedded spaces where needed between command switches and the filename, etc., ...). For example,
cmd=strcat('dir /a:h /b ',fn);
looks just right, doesn't it? But, it has the nasty behavior that it does an internal strtrim() on the arguments before doing the catenation so you end up with the filename string being directly appended to the last command switch and Boom! the call fails and you'll spend a long time looking to figure out why... :( To make the behavior even more obtuse, if you use a cellstr or string class, then it doesn't trim--go figure.
>> fn=fullfile('c:','*.sys');
>> cmd=['dir /a:h /b ' fn];
>> [~,s]=system(cmd)
s =
'hiberfil.sys
pagefile.sys
swapfile.sys
'
>>
NOTA BENE: the above is one string with embedded \n -- remember the caveat that hopefully nobody was nasty enough to have used one in an actual filename on your system...that's a nasty.
>> split(s,newline)
ans =
4×1 cell array
{'hiberfil.sys'}
{'pagefile.sys'}
{'swapfile.sys'}
{0×0 char }
>>
That's with MS CMD processor that MATLAB uses "out-of-the-box" -- another aberration is that TMW ignores the COMSPEC environment variable so one has to go thru gyrations to get another shell. I use the JPSoft (shameless plug for super product; not related, just 20+ year user) replacement which has more functionality included a "/f" switch besides "/b" to remove the header/trailer info; it returns the fully-qualified name for all.
@Walter Roberson is bound to come along before long and can/will give a specific example, undoubtedly! :)
Answers (5)
Matt O'Brien
on 25 Apr 2023
Thanks for the syntax.
I got the following to work ... in principle ...
-a includes hiddenitems
-R recursive mode ...
clc
MyLsString= strcat('ls -a -R /Volumes/MoB_WD_5TB/DeeAprilTravel2023');
[~,s]=system(MyLsString);
MyList = splitlines(s);
MyLen = length(MyList);
z=0;
But I am not sure I have the skills or trust my efforts to untangle the output.
Splitlines.
I used split lines to get lines raather than a single long text string. This seems to work ... but not sure I can trust the output....
More troubling....
Data Returned.
No doubt this data can be unscrambled... but I would not trust myself to do this.
Also ... I see lots of funny scenarios.
eg. Line 7 DCIM is a very important sub directory ... but it is on the same line as 'fseventsd-uuid'
Maybe this is a splitline issue.
So ... my summary ... Good to know how to issue system commands from Matlab .... but looks scary to try and convert this into useful data to resolve my issue.
4 Comments
dpb
on 25 Apr 2023
Well, you get what the OS tools will give you...first thing I'd do would be to compare to the output format at the command line to see how it formats the output; as noted before there are "issues" with OS'es that allow such abominations as a \n as an allowable character in a filename -- or, perhaps some of the problems may be there are other nonprinting characters embedded in those -- only more spelunking will let you know what it is that is in some of those filename strings for certain.
Good luck; it reminds me why I an always glad despite its warts in other areas to not have to have dealt with the --ix file system.
From the stuff you posted before, it would look to me still like the more sure path would be to use a mex gateway into the SWIFT OS calling stuff you found above, although I'd definitely to some more exhaustive looking at the details of what ls returns and the actual returned formatted output. I've been know to use output redirection and capture a file in the past, too...what does the output of
ls -a -R /Volumes/MoB_WD_5TB/DeeAprilTravel2023 >lsOut.txt
look like? Is it unwrapped differently?
BTW in
MyLsString= strcat('ls -a -R /Volumes/MoB_WD_5TB/DeeAprilTravel2023');
the strcat function is totally superfluous...but, you probably already knew that...
More potentially problematical is
MyLen = length(MyList);
length is dangerous function; it is defined as max(size,x)) so it can potentially return either number of rows or columns. In this case, splitlines will always return a cell array so it works, but as a defensive coding technique, avoid using length and ask for the specific size direction of interest. It's a remnant of very first implementations but would not have been defined as such later...
You cannot use ls together with text splitting reliably determine filenames in any situation in which there might be spaces in valid file names. You instead need to use the '-1' (digit one) option, such as
!ls -1Ra ..
The -a flag and the -1 flag are not supported on Windows.
It is not clear to me why you would want to be using -a to get all files when your complaint is about weakness in not having hidden files automatically filtered out.
In my opinion, if you are going to use OS system-specific code then you should be using find for this task
My next step will be to see if I can find a Mac Utility which will identify hidden / system files,
NO you will not find any such utility. Mac has no understanding of system files. Such a utility would have to be something like Paragon Software's NTFS drivers for Mac that treated the device as a file system.
Matt O'Brien
on 25 Apr 2023
Matt O'Brien
on 26 Apr 2023
All good points... good tip re 'length' .... use of strcat was to allow me play with different combinations of parameters... in this case no parameters ... so looked superfluous....
Here is a screen grab of result of -1Ra ... have not figured out yet how to to try and structure the output...
This is the result of just hovering over the s value....
I am going to take a rest on this for a while... I am not going to try and convert the text into a structured dataset.... I can see some clues how to do that ... but not keen to follow that path...
I will keep you posted on any developments...
I sent you a PM ... just fyi...
Regards...
Matt O'Brien
Matt O'Brien
on 8 May 2023
Update. Explore finding Hidden and Visible files/folders using Apple native tools.
I have spent considerable time exploring the Apple native apps such as Swift/Xcode and AppleScript.
My objective is two fold.
- Explore, using native Apple apps, if I can separate visible and hidden dirs using a script or app
- Recce the possibility of aborting MatLab and reverting to Swift/Xcode and build my apps on Mac / Ipad /Iphone.
I built the very basic and crude Apple script below.... It opens a folder selection dialog, where I select an SD card in a connected card reader.
It has 2 basic loops, both just at the top folder level (ie non recursive). The first looks for hidden folders and the second loop looks for visible folders. In each case it lists (logs) the folders found. The output is extremely crude... will make it pretty later.
Here is the results. It is a screen grab from the AppleScript IDE. Clearly, I can select and identify hidden folders and/or visible folders.
My Key Point.
I had to learn Apple developing and scripting tools from scratch. If I can do this .... then it must be seriously possible for MatLab to add a feature to the current Dir function or create a new function (to preserve compatibility / avoid bugs) which will enable the basic ability to be able to list visible / hidden files/folders with simplefunction option.
I am horrified that such an important feature in such an important function is missing and that I have to go to this level of effort to resolve in a reliable way.
In AppleScript I can identify visible / hidden folders with a single clear statement.
"set MyList to folders of the_folder whose visible is false"
I am enclosing the script I used (it is crude) just for your info.
My Next Action.
I plan to refine this script in due course, get it working for all subfolders.... and then begin to migrate my field apps from MaLab to work on native Apple code. A winter project. This will allow me to use my MacAir very efficiently in the field for ingesting high volume media files and manage the digital assets accumulated. I may even be able to get my app to work on iPhone or iPads, especially with USB C iPhones in the pipeline. That is all nice to have. What is critical for me now is I have a solution which is fit for purpose and not dependent on hard coding to cater for the infinite variety of hidden files or folders stored by different camera companies when saving images / video to digital media.
I will leave this post here for a week or so. If no other solution found by then I plan to mark this post as the answer to my query.
-- Get all files in a selected folder
-- Explore visible / hidden folders
log "Hidden"
-- Get user folder
set the_path to choose folder with prompt "Choose your folder..."
my file_to_folder(the_path)
on file_to_folder(the_folder)
tell application "System Events"
-- Get all folders = hidden
set MyList to folders of the_folder whose visible is false
repeat with MyItem in MyList
-- Remove extension of the file name
set the_file_ext to name extension of MyItem
set the_name to name of MyItem as string
log the_name
end repeat
end tell
log " "
log "Visible"
tell application "System Events"
-- Get all folders = visible
set MyList to folders of the_folder whose visible is true
repeat with MyItem in MyList
-- Remove extension of the file name
set the_file_ext to name extension of MyItem
set the_name to name of MyItem as string
log the_name
end repeat
end tell
end file_to_folder
-- Ending dialog
display dialog ("It's done!") buttons {"Perfect!"}
10 Comments
Walter Roberson
on 8 May 2023
I already showed in https://www.mathworks.com/matlabcentral/answers/1950983-exclude-system-or-hidden-folders-from-mac-dir-list#comment_2717723 the (shell-level) command
find . -flags +hidden -print
that finds and lists all files and directories relative to the current directory that have the filesystem flag hidden set. One item is listed per line. No need to learn AppleScript or the like -- just use of one of the most common unix utilities.
Walter Roberson
on 8 May 2023
function dinfo = dir_no_hidden(dirpath)
if ~exist(dirpath, 'dir')
dinfo = [];
return;
end
dinfo = dir( fullfile(dirpath, '**') );
cmd = sprintf("find '%s' -flags +hidden -print", dirpath);
[status, msg] = system(cmd);
if status == 0
hidden_files = regexp(msg, '\r?\n+', 'split');
dinfo_fullnames = fullfile({dinfo.folder}, {dinfo.name});
need_to_exclude = ismember(dinfo_fullnames, hidden_files);
dinfo(need_to_exclude) = [];
end
end
Matt O'Brien
on 8 May 2023
Walter Roberson
on 8 May 2023
Figuring out files are marked system is not easy; figuring out which files are marked hidden is much easier.
Your Swift / Applescript code is working with hidden and not system
dpb
on 8 May 2023
@Walter Roberson, could the " ... One item is listed per line." be a potential problem for the allowed (I think on Apple?) aberrent file containg an embedded \n in that it can't be definitively proven the split is correct?
This may not be an issue for the OP if the SD cards are using DOS naming conventions so isn't possible, just a question that I saw raised during the previous looking I did earlier.
Matt O'Brien
on 8 May 2023
dpb
on 8 May 2023
I still agree with the initial point that TMW should have either incorporated the common OS switches in the existing dir() function LONG ere now (like in the beginning) or have produced alternate functionality. It is, after all, the user-friendly rapid-development environment that is what it's all about; leaving this to the user to deal with is, simply put, "just plain rude!" as one of my granddaughters is wont to opine...it's doable, yes, but why can't we simply just use "/h" or "/-h" in some form other than calling the OS and parsing it ourselves?
Back years ago, I had done for Windows; unfortunately, it was another of the utilities that got left on the corporate machine when retired to the farm and forgot to offload that stuff before leaving...what I really miss is the mex extension that took a Fortran FORMAT string and variables then passed on the the Fortran i/o library --a MUCH nicer solution than writing those ugly C formatting strings with no repeat counts, etc., ...without paying consulting client, it hasn't been enough of an issue to have bothered to regenerate although at the moment I can't even get a Fortran compiler to link mex; not having the "high-priced spread" of the Intel compiler. Since they support gcc, there's no reason the Fortran compiler shouldn't be supported as well, but I've been unable to solve the name-mangling issues to link. Anyways, sorry for the side rant; we old fogeys tend to wander off...
Matt O'Brien
on 8 May 2023
Paul Martin
on 15 Jan 2026
/ fundimental / fundamental / [that's a droll infelicity, coming from a chap who worked on pipelines ... ]
Matt O'Brien
on 15 Jan 2026
Matt O'Brien
on 8 May 2023
0 votes
I am keen to supply feedback asap .... realised I needed to run the script in a Mac environment...
I copied the struct output from your script above into a table and saved to Excel.
With the test data I currently have on the card / card reader ... I have approx 700 lines of files/folders...
My initial dive into this is encouraging... with a little twist...
The folders, listed in these screen grabs, I am keen to exclude .... It looks like the script has successfully excluded these folders but left the files (or some of them) from within these folders. I only found these 3 examples.
I need to do a more thorough job of checking the test data and maybe testing a few other scenarios (eg different cameras). I may be missing some outliers in the other 700 rows, but initial inspection is looking very promising.
No need to try and tweak the script at this point. I will revert when I am comfortable I have digested the results.
I may need to get the list of 'hidden directories' and then specifically exclude / ignore the content of this directories.
(Ps1. Just FYI. there were dot and double dot folders included in the list ... which I am comfortable getting rid of.)
(Ps2. I will try to acquire a better grasp of the system 'find' command).
Matt O'Brien
on 9 May 2023
0 votes
Matt O'Brien
on 9 May 2023
0 votes
Categories
Find more on File Operations in Help Center and File Exchange
on 21 Apr 2023
on 15 Jan 2026
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!
