Why do I receive an "Error using dir" message when running Experiment Manager?
21 views (last 30 days)
Show older comments
I'm new to the Experiment Manager App, and have been using it help perform hyperparameter optimization (learning rate, number of epochs, number of nodes, and L2 regularization) for a deep autoencoder network that I'm training. I am executing all trial iterations in parallel with 4 workers. The App worked well at first (ran start to finish without any issues) but for some reason when I try to either (A) increase the max number of epochs (past 1000) or (B) increase the number of trials, the App inexplicably returns the error message "Error using dir: Name must be a text scalar" followed by "Error in genpath (line 38): files=dir(d)"
What's unusual is that this error is never returned right away. The App will run through at least 4 trials and has even run up to 36 trials (out of the 60 I attempted to run) before this error message is returned. If I reduce the max number of epochs and/or number of trials, it works fine again. This leads me to believe that it is not an error with my setup function.
Has anyone experienced anything similar to this, have any idea what could be going wrong, or have any idea how to fix it? Any help would be greatly appreciated.
0 Comments
Answers (1)
Sivsankar
on 20 Aug 2024
Hi Jaclyn,
The error message “Error using dir” and “Error in genpath” seems to be related to the ‘dir’ function which is used to list files and directories. The message "Name must be a text scalar" indicates that the function is being called with an argument that isn't a valid string or character vector. So, ensure that any directory paths or filenames are valid strings.
But since this problem only occurs with higher number of epochs or trials, it suggests that the problem might be related to resource constrains. So, you must ensure to configure your parallel pool workers accordingly that fits to your resources. For instance, the experiment might create temporary files that might exceed system limits when the number of trials is increased, and this may have caused the issue. Also try to make sure that your paths used are not excessively long as well.
I believe this may be the reason why you are encountering this issue. Hope the helps
Thanks,
0 Comments
See Also
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!