Hadoop MapReduce job failing in launch_container.sh

2022-12-09T23:40:59

MapReduce job is failing with following error even though JAVA_HOME is set.

/bin/bash: /bin/java: No such file or directory

I am trying to setup hadoop (3.3.4) on my Mac M1. I have set JAVA_HOME in /etc/hadoop/hadoop-env.sh

export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0_351.jdk/Contents/Home

I am able to add files to the HDFS but on running the sample MapReduce job - hadoop-mapreduce-examples-3.3.4.jar wordcount, or any other MapReduce job I am getting the following error in launch_container.sh,

[2022-12-09 20:39:39.415]Container exited with a non-zero exit code 127. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
/bin/bash: /bin/java: No such file or directory

I suspect this the following line is failing and JAVA_HOME is coming empty.

launch_container.sh

exec /bin/bash -c "$JAVA_HOME/bin/java -Djava.io.tmpdir=$PWD/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/opt/homebrew/Cellar/hadoop/3.3.4/libexec/logs/userlogs/application_1670596645240_0001/container_1670596645240_0001_01_000001 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog  -Xmx1024m org.apache.hadoop.mapreduce.v2.app.MRAppMaster 1>/opt/homebrew/Cellar/hadoop/3.3.4/libexec/logs/userlogs/application_1670596645240_0001/container_1670596645240_0001_01_000001/stdout 2>/opt/homebrew/Cellar/hadoop/3.3.4/libexec/logs/userlogs/application_1670596645240_0001/container_1670596645240_0001_01_000001/stderr "

but JAVA_HOME is set in all these places

hadoop-env.sh 
~/.bash_profile 
~/.zprofile

I have tried reinstalling hadoop 3-4 times but everytime its getting stuck here. Most other solutions say to set JAVA_HOME in hadoop-env.sh, but its already set there.

Any help will be appreciated, Thanks.

Copyright License:
Author:「Jatin」,Reproduced under the CC 4.0 BY-SA copyright license with link to original source & disclaimer.
Link to:https://stackoverflow.com/questions/74745268/hadoop-mapreduce-job-failing-in-launch-container-sh

About “Hadoop MapReduce job failing in launch_container.sh” questions

MapReduce job is failing with following error even though JAVA_HOME is set. /bin/bash: /bin/java: No such file or directory I am trying to setup hadoop (3.3.4) on my Mac M1. I have set JAVA_HOME i...
Running stand-alone hadoop server 2.6.0 sqoop - 1.4.6 trying to import data from MySQL to HBASE sqoop import --connect jdbc:mysql://localhost/WeatherDataSet --username root -P --table EuropeDataS...
I am trying to simulate the Hadoop environment using latest Hadoop version 2.6.0, Java SDK 1.70 on my Ubuntu desktop. I configured the hadoop with necessary environment parameters and all its proc...
HI I am running a simple load and dump command on my linusystem. grunt> a= load 'a.txt'; grunt> dump a; I am getting following error 2016-04-07 08:27:07,029 [main] INFO org.apache.pig...
I am relatively new to hadoop 2 (hadoop 2.2.0) and I don't understand why M/R job ~ application on Resource manager is marked as failed : application_1399458460502_0015 pig Max temperature MAPRED...
I am trying to run my first mapreduce job, which aggregates some data from xml files. My job is failing, and as I am a newbie at Hadoop, I would appreciate if someone could please take a look at wh...
I have setup a 2 node cluster of Hadoop 2.3.0. Its working fine and I can successfully run distributedshell-2.2.0.jar example. But when I try to run any mapreduce job I get error. I have setup MapR...
I am trying to run wordcount example Here is the code import java.io.IOException; import java.util.StringTokenizer; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs....
I am using Hadoop 2.7.0 in pseudo node mode, on a Fedora 22 Virtual Machine. A few days back the MapReduce jobs ran fine, but after installed Oozie and made modifications to the yarn-site.xml . I am
I am currently learning to use Hadoop mapred an have come across this error: packageJobJar: [/home/hduser/mapper.py, /home/hduser/reducer.py, /tmp/hadoop-unjar4635332780289131423/] [] /tmp/

Copyright License:Reproduced under the CC 4.0 BY-SA copyright license with link to original source & disclaimer.