Executing map reduce using hadoop streaming through php

2013-11-18T17:15:27

I have the following script :

<?php
$a =isset($_POST['text'])?$_POST['text']:'not yet';
$comm='/usr/local/hadoop/bin/hadoop jar /usr/local/hadoop/contrib/streaming/hadoop-*streaming*.jar -mapper "/usr/bin/python /var/www/DataMining/AnalysisByYear/AnalysisByYear_mapper.py '.$a.'" -reducer "/usr/bin/python /var/www/DataMining/AnalysisByYear/AnalysisByYear_reducer.py" -input /user/hduser/dataset/final_eval.txt -output /user/hduser/dataset-outputyear';
$mys=exec($comm);
$comm1='/usr/local/hadoop/bin/hadoop dfs -copyToLocal /user/hduser/dataset-outputyear/part-00000 /var/www/year.txt';
$mys1=exec($comm1);
$file = '/var/www/year.txt';
$contents = file($file); 
$string = implode($contents);
echo $string;
?>

I am trying to execute the $comm and $comm1 commands through shell via php . However I am not getting any result.Please help

Copyright License:
Author:「user2490585」,Reproduced under the CC 4.0 BY-SA copyright license with link to original source & disclaimer.
Link to:https://stackoverflow.com/questions/20043997/executing-map-reduce-using-hadoop-streaming-through-php

About “Executing map reduce using hadoop streaming through php” questions

I have the following script : &lt;?php $a =isset($_POST['text'])?$_POST['text']:'not yet'; $comm='/usr/local/hadoop/bin/hadoop jar /usr/local/hadoop/contrib/streaming/hadoop-*streaming*.jar -mappe...
I would like to use the hadoop-streaming functionality with perl scripts as the mapper and reducer. I found out this explanation that partially answer my question, however it does not contain the
I'm currently processing about 300 GB of log files on a 10 servers hadoop cluster. My data is being saved in folders named YYMMDD so each day can be accessed quickly. My problem is that I just fou...
I want to run python map reduce job in Google Cloud Dataproc using hadoop streaming method. My map reduce python script, input file and job result output are located in Google Cloud Storage. I tri...
I ran into these issues while using Hadoop Streaming. I'm writing code in python 1) Aggregate library package According to the hadoop streaming docs ( http://hadoop.apache.org/common/docs/r0.20.0/
I'm using hadoop 1.0.1 on a single node and I'm trying to stream a tab delimited file using python 2.7. I can get Michael Noll's word count scripts to run using hadoop/python, but can't get this
I am learning Python and Hadoop. I completed the setup and basic examples provided in official site using pythong+hadoop streaming. I considered implementing join of 2 files. I completed equi-join ...
I am looking for a ballpark if any one has experience with this... Does anyone have benchmarks on the speed of AWS's map reduce? Lets say I have 100 million records and I am using hadoop streamin...
How to reduce number of splits in map reduce job using hadoop streaming jar. I tried to modify mapreduce.input.fileinputformat.split.minsize/maxsize but the number is still the same. Number of sp...
I am currently using a hadoop 5 node cluster with 5 slaves Each datanode has a capacity of 8.7 TB. I am executing a map reduce job to execute 312 GB of data but got a error of Apllication failed a...

Copyright License:Reproduced under the CC 4.0 BY-SA copyright license with link to original source & disclaimer.