HDP 3.1.1 and MATLAB R2019a Compatability
2 views (last 30 days)
Show older comments
Brian Coghlan
on 8 Aug 2019
Commented: Onomitra Ghosh
on 2 Oct 2019
I have a cluster of linux machines running HDP 3.1.1. I see in all the HDFS documentation that only Hadoop versions 1.x and 2.x are supported. Will I have compatability issues due to using a 3.x version of Hadoop? When I try to read HDFS files from the cluster from a edge Windows machine, I continually get an error when using datastore (see below).
Error using datastore (line 120)
Cannot determine the Hadoop version. Verify that HADOOP_PREFIX, HADOOP_HOME, or the MATLAB_HADOOP_INSTALL environment variable is set to the root of your Hadoop installation folder.
The Windows machine I am accessing the cluster from does not have Hadoop installed and I point the required environment variable(s) to root of the Hadoop installation on the namenode of the cluster. I have spent considerable time attempting all fixes I could find assuming that Hadoop 3.1.1 is compatable with Matlab, but am suspecting that may be the root of the problem.
-Brian
0 Comments
Accepted Answer
Onomitra Ghosh
on 1 Oct 2019
Brian, can you provide some more details on the problem you are running into when running the mapreduce command. A code example of how you are setting the environment variable, creating datastore and running mapreduce will be good. Also, is the error message exactly the same as before? Is this the complete error stack?
2 Comments
Onomitra Ghosh
on 2 Oct 2019
Brian, thanks for finding this. You have uncovered an issue in the way we seek for Hadoop version. We will work on finding a better solution for this. Until then, please continue to have the HDFS installation on the worker nodes as a workaround. We will reach out to you if we need more information.
More Answers (1)
See Also
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!