What is the proper value of HADOOP_HOME and PIG_CLASSPATH for APACHE HADOOP version 2.8.0?

2017-05-12T04:24:21

I have the problem executing the Hadoop command from the PIG command line. The command and the error stack is below

My instructor suspects that it is because HADDOP_HOME and PIG_CLASSPATH are incorrect. I am on the HADOOP version 2.8.0.

So, originally I had HADOOP_HOME as

HADOOP_HOME=<CELLAR_DIRECTORY>/hadoop/2.8.0/

Then I switched the following setup:

HADOOP_HOME=<CELLAR_DIRECTORY>/hadoop/2.8.0/libexec/etc/hadoop

PIG_CLASSPATH is defined as $HADOOP_HOME

Commands I used in pig:

A = LOAD '/Users/anarinsky/Downloads/loaddata1.txt';

B = MAPREDUCE '/Users/anarinsky/workspace/wordcount/target/wordcount-1.jar' STORE A INTO '/Users/anarinsky/Downloads/tempwrite2' LOAD  '/Users/anarinsky/Downloads/tempwrite2' AS (word:chararray, count:int) `com.systemskills.hadoop.wordcount.WordCountDriver /wordcountdata /Users/anarinsky/Downloads/pigoptdir`;

Pig Stack Trace

ERROR 2025: Expected leaf of reduce plan to always be POStore. Found PONative

org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias B
    at org.apache.pig.PigServer.openIterator(PigServer.java:1019)
    at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:747)
    at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:376)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:231)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:206)
    at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:66)
    at org.apache.pig.Main.run(Main.java:564)
    at org.apache.pig.Main.main(Main.java:176)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:234)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
Caused by: org.apache.pig.PigException: ERROR 1002: Unable to store alias B
    at org.apache.pig.PigServer.storeEx(PigServer.java:1122)
    at org.apache.pig.PigServer.store(PigServer.java:1081)
    at org.apache.pig.PigServer.openIterator(PigServer.java:994)
    ... 13 more
Caused by: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompilerException: ERROR 2025: Expected leaf of reduce plan to always be POStore. Found PONative
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler.compile(MRCompiler.java:321)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.compile(MapReduceLauncher.java:629)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:152)
    at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.launchPig(HExecutionEngine.java:308)
    at org.apache.pig.PigServer.launchPlan(PigServer.java:1474)
    at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1459)
    at org.apache.pig.PigServer.storeEx(PigServer.java:1118)
    ... 15 more

Copyright License:
Author:「Alex」,Reproduced under the CC 4.0 BY-SA copyright license with link to original source & disclaimer.
Link to:https://stackoverflow.com/questions/43924904/what-is-the-proper-value-of-hadoop-home-and-pig-classpath-for-apache-hadoop-vers

About “What is the proper value of HADOOP_HOME and PIG_CLASSPATH for APACHE HADOOP version 2.8.0?” questions

I have the problem executing the Hadoop command from the PIG command line. The command and the error stack is below My instructor suspects that it is because HADDOP_HOME and PIG_CLASSPATH are inc...
I have an existing Hadoop client (hadoop-0.20.2) at $HADOOP_HOME. With this version of Hadoop all the client configuration files are placed in the directory: $HADOOP_HOME/conf To get Pig to work, ...
I downloaded the binary tarball of hadoop from here: http://hadoop.apache.org/releases.html (ver 2.8.4). I unpacked the tar.gz file and then changed the etc/hadoop-env.sh from export JAVA_HOME={$
Hi I am trying to run hadoop on a server that has hadoop installed but I have no idea the directory where hadoop resides. The server was configure by the server admin. In order to load hadoop I u...
I want to install Java and Hadoop on Windows10. So far I have Java jdk1.8.0_211 as when I try c:\&gt;java -version it returns: java version "1.8.0_211" Java(TM) SE Runtime Environment (build 1.8.
I am working on a Java program to interface with an already running hadoop cluster. The program has HADOOP_HOME passed to it as an environment variable. Based on this value, I need to load all of ...
I understand this question might have been answered already, well, my issue is still here: I have a vm created for hadoop on vmware using CentOS7, I can start namenode and datanode, however, when I
I have installed hadoop 3.1.0 version and set env variables. When i run $HADOOP_HOME/bin/hadoop version, it is shown that i installed but running hadoop version is not able to find the hadoop comma...
I'm setting up a single Hadoop node, but when running $HADOOP_HOME/sbin/start-dfs.sh it prints that it cannot find $HADOOP_HOME/bin/hdfs. The file at that location exists though, and I can read it
My question is in cloudera 5.12; To what value is $HADOOP_HOME set? Can someone help me on this?

Copyright License:Reproduced under the CC 4.0 BY-SA copyright license with link to original source & disclaimer.