UnknownHostException on tasktracker in Hadoop cluster

2011-01-19T07:14:13

I have set up a pseudo-distributed Hadoop cluster (with jobtracker, a tasktracker, and namenode all on the same box) per tutorial instructions and it's working fine. I am now trying to add in a second node to this cluster as another tasktracker.

When I examine the logs on Node 2, all the logs look fine except for the tasktracker. I'm getting an infinite loop of the error message listed below. It seems that the Task Tracker is trying to use the hostname SSP-SANDBOX-1.mysite.com rather than the ip address. This hostname is not in /etc/hosts so I'm guessing this is where the problem is coming from. I do not have root access in order to add this to /etc/hosts.

Is there any property or configuration I can change so that it will stop trying to connect using the hostname?

Thanks very much,

2011-01-18 17:43:22,896 ERROR org.apache.hadoop.mapred.TaskTracker: 
Caught exception: java.net.UnknownHostException: unknown host: SSP-SANDBOX-1.mysite.com
        at org.apache.hadoop.ipc.Client$Connection.<init>(Client.java:195)
        at org.apache.hadoop.ipc.Client.getConnection(Client.java:850)
        at org.apache.hadoop.ipc.Client.call(Client.java:720)
        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
        at $Proxy5.getProtocolVersion(Unknown Source)
        at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359)
        at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:106)
        at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:207)
        at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:170)
        at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:82)
        at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1378)
        at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
        at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1390)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:196)
        at org.apache.hadoop.fs.Path.getFileSystem(Path.java:175)
        at org.apache.hadoop.mapred.TaskTracker.offerService(TaskTracker.java:1033)
        at org.apache.hadoop.mapred.TaskTracker.run(TaskTracker.java:1720)
        at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:2833)

Copyright License:
Author:「knt」,Reproduced under the CC 4.0 BY-SA copyright license with link to original source & disclaimer.
Link to:https://stackoverflow.com/questions/4730148/unknownhostexception-on-tasktracker-in-hadoop-cluster

About “UnknownHostException on tasktracker in Hadoop cluster” questions

I have set up a pseudo-distributed Hadoop cluster (with jobtracker, a tasktracker, and namenode all on the same box) per tutorial instructions and it's working fine. I am now trying to add in a sec...
I had Hadoop 3 node cluster which works perfectly. Next day the namenode and jobtracker stops working and the datanode and tasktracker working continuously . After starting the hadoop it works for ...
I've just set up a Hadoop cluster with Hadoop 0.20.205. I have a master (NameNode and JobTracker) and two other boxes (slaves). I'm trying to understand, how to define the number of map and reduce...
Is it possible to configure cgroups or Hadoop in a way that each process that is spawned by the TaskTracker is assigned to a specific cgroup? I want to enforce memory limits using cgroups. It is
I'm running a hadoop cluster (version 0.20.205), and I have to periodically deploy new code to the cluster, which involves taking the cluster down and bringing it back up again with the new code. My
I am new to hadoop. I have local dir and system dir mentioned in the mapred config. The error log is saying cannot create the directory so the start up failed. root@SYS-01:/etc/hadoop/conf.
I am trying to use hadoop under windows and I am running into a problem when I want to start tasktracker. For example: $bin/start-all.sh then the logs writes: 2011-06-08 16:32:18,157 ERROR org.a...
i am following http://ebiquity.umbc.edu/Tutorials/Hadoop/00%20-%20Intro.html Start the namenode in the first window by executing cd hadoop-0.19.1 bin/hadoop namenode Start the secondary namenode ...
I am running a Hadoop cluster on 4 servers. I see that all servers has TaskTracker and DataNone I start cluster with hadoop/bin/start-all.sh I have 2 servers which have very litter Hardware Disk ...
I am new to hadoop and only have a Windows machine available. On that, I have created 2 virtual machines in VirtualBox that run CentOS 6.2. I have gone through several problems already with enabli...

Copyright License:Reproduced under the CC 4.0 BY-SA copyright license with link to original source & disclaimer.