Hadoop nodemanager not starting

2020-08-07T16:57:45

I am setting up a single node cluster. I am following the https://dev.to/awwsmm/installing-and-running-hadoop-and-spark-on-ubuntu-18-393h tutorial. However, My nodemanager is not starting.

14468 SecondaryNameNode
14648 ResourceManager
14074 NameNode
15293 Jps
14223 DataNode

Below are the node manager logs:


2020-08-07 14:13:52,796 FATAL org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Failed to initialize mapreduce_shuffle 
java.lang.IllegalArgumentException: The ServiceName: mapreduce_shuffle  set in yarn.nodemanager.aux-services is invalid.The valid service name should only contain a-zA-Z0-9_ and can not start with numbers
    at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:114)
    at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
    at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:237)
    at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
    at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
    at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:255)
    at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
    at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:480)
    at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:527)
2020-08-07 14:13:52,800 INFO org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices failed in state INITED; cause: java.lang.IllegalArgumentException: The ServiceName: mapreduce_shuffle  set in yarn.nodemanager.aux-services is invalid.The valid service name should only contain a-zA-Z0-9_ and can not start with numbers
java.lang.IllegalArgumentException: The ServiceName: mapreduce_shuffle  set in yarn.nodemanager.aux-services is invalid.The valid service name should only contain a-zA-Z0-9_ and can not start with numbers
    at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:114)
    at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
    at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:237)
    at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
    at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
    at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:255)
    at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
    at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:480)
    at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:527)
2020-08-07 14:13:52,801 INFO org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl failed in state INITED; cause: java.lang.IllegalArgumentException: The ServiceName: mapreduce_shuffle  set in yarn.nodemanager.aux-services is invalid.The valid service name should only contain a-zA-Z0-9_ and can not start with numbers
java.lang.IllegalArgumentException: The ServiceName: mapreduce_shuffle  set in yarn.nodemanager.aux-services is invalid.The valid service name should only contain a-zA-Z0-9_ and can not start with numbers
    at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:114)
    at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
    at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:237)
    at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
    at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
    at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:255)
    at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
    at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:480)
    at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:527)
2020-08-07 14:13:52,802 INFO org.apache.hadoop.service.AbstractService: Service NodeManager failed in state INITED; cause: java.lang.IllegalArgumentException: The ServiceName: mapreduce_shuffle  set in yarn.nodemanager.aux-services is invalid.The valid service name should only contain a-zA-Z0-9_ and can not start with numbers
java.lang.IllegalArgumentException: The ServiceName: mapreduce_shuffle  set in yarn.nodemanager.aux-services is invalid.The valid service name should only contain a-zA-Z0-9_ and can not start with numbers
    at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:114)
    at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
    at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:237)
    at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
    at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
    at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:255)
    at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
    at 

2020-08-07 14:13:52,816 INFO org.apache.hadoop.yarn.server.nodemanager.NodeManager: SHUTDOWN_MSG: 
/************************************************************
SHUTDOWN_MSG: Shutting down NodeManager at es/127.0.1.1
************************************************************/

Here is my yarn-site.xml:

<?xml version="1.0"?>


<configuration>

<!-- Site specific YARN configuration properties -->

  <property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
  </property>
  <property>
    <name>yarn.nodemanager.auxservices.mapreduce.shuffle.class</name>  
    <value>org.apache.hadoop.mapred.ShuffleHandler</value>
  </property>
</configuration>

I have saw many posts where mapreduce.shuffle has to be changed to mapreduce_shuffle. However, in my yarn-site.xml I already have mapreduce_shuffle.

Copyright License:
Author:「Ayush Goyal」,Reproduced under the CC 4.0 BY-SA copyright license with link to original source & disclaimer.
Link to:https://stackoverflow.com/questions/63298528/hadoop-nodemanager-not-starting

About “Hadoop nodemanager not starting” questions

I am setting up a single node cluster. I am following the https://dev.to/awwsmm/installing-and-running-hadoop-and-spark-on-ubuntu-18-393h tutorial. However, My nodemanager is not starting. 14468
I am trying to setup the latest Hadoop 2.2 single node cluster on Ubuntu 13.10 64 bit. the OS is a fresh installation, and I have tried using both java-6 64 bit and java-7 64 bit. After following ...
i am using hadoop and it was working fine. but today when i restarted the server, now jps command is showing something like 9801 NameNode 12142 Jps 10207 SecondaryNameNode i have reconfigured the...
I change some configuration and need to restart nodemanager. I get the following error message: Error starting NodeManager java.lang.UnsatisfiedLinkError: Could not load library. Reasons: [no
I have compiled the hadoop binaries on windows x64 machine.. I also built the native source in the path hadoop-2.6.0-src.tar\hadoop-2.6.0-src\hadoop-2.6.0-src\hadoop-common-project\hadoop-common\sr...
In my Windows 10 environment, I installed Hadoop 2.10.1 While starting the nodemanager using .\start-yarn.cmd command, I am getting following error message. The DataNode, ResourceManager and Nam...
I have setup hadoop and yarn in standalone mode for now. I am trying to start all process in yarn. All process are started except nodemanager. It is throwing jvm error everytime. [root@ip-10-100-...
I have followed all the steps from this website. On my PC all are running perfectly but on my laptop nodeManager is shutting down. https://muhammadbilalyar.github.io/blogs/How-to-install-Hadoop-on-
NodeManager in my slave node stops after starting. I have 3 node 1 master and 2 slaves when I use command start-yarn.sh my resourcemanager and nodemanagers start correctly but when I query in hive my
I have enabled kerberos security for hadoop 2.2.0 cluster. But while starting nodemanager i am getting below error: Machine-1 namenode works as datanode at this node nodemanager is running but at m...

Copyright License:Reproduced under the CC 4.0 BY-SA copyright license with link to original source & disclaimer.