Pentaho unable to copy files to Hadoop HDFS file system 1.0.3

2015-09-08T00:35:19

This is my first thread and am using using 5.4.0.1-130 Pentaho kettle version.

I have installed hadoop-1.0.3 version in a VM player and I have bridged it using bridged network.

I have Pentaho installed on my desktop on Windows10 and the hadoop is available in the above mentioned VMplayer.

I'm trying to do a "Hadoop Copy File" job, but it fails with the following error.

SourceEnvironment: <static>
SourceFile/Folder: file:///C:/Study/Pentaho/data-integrationC:/Study/Pentaho/data-integration
DestinationEnvironment: <static>
Destination File/Folder: hdfs://notroot/[email protected]:8020/input

I tried to copy and create a folder C:\Study\Pentaho\data-integration\plugins\pentaho-big-data-plugin\hadoop-configurations\hadoop-103\lib as per the instruction mentioned in this website(http://funpdi.blogspot.in/2013/03/pe...nd-hadoop.html), still no luck.

Kindly advice what am I doing wrong? Thank you!

015/09/07 11:56:02 - Hadoop Copy Files - Processing row source File/folder source : [file:///C:/Study/Pentaho/data-integrationC:/Study/Pentaho/data-integration] ... destination file/folder : [hdfs://notroot/[email protected]:8020/input]... wildcard : [^.*\.txt]
2015/09/07 11:56:04 - Hadoop Copy Files - ERROR (version 5.4.0.1-130, build 1 from 2015-06-14_12-34-55 by buildguy) : Can not copy file/folder [file:///C:/Study/Pentaho/data-integrationC:/Study/Pentaho/data-integration] to [hdfs://notroot/[email protected]:8020/input]. Exception : [
2015/09/07 11:56:04 - Hadoop Copy Files - 
2015/09/07 11:56:04 - Hadoop Copy Files - Unable to get VFS File object for filename 'hdfs://notroot/[email protected]:8020/input' : Could not resolve file "hdfs://notroot/[email protected]:8020/input".
2015/09/07 11:56:04 - Hadoop Copy Files - 
2015/09/07 11:56:04 - Hadoop Copy Files - ]
2015/09/07 11:56:04 - Hadoop Copy Files - ERROR (version 5.4.0.1-130, build 1 from 2015-06-14_12-34-55 by buildguy) : org.pentaho.di.core.exception.KettleFileException: 
2015/09/07 11:56:04 - Hadoop Copy Files - 
2015/09/07 11:56:04 - Hadoop Copy Files - Unable to get VFS File object for filename 'hdfs://notroot/[email protected]:8020/input' : Could not resolve file "hdfs://notroot/[email protected]:8020/input".
2015/09/07 11:56:04 - Hadoop Copy Files - 
2015/09/07 11:56:04 - Hadoop Copy Files - 
2015/09/07 11:56:04 - Hadoop Copy Files - at org.pentaho.di.core.vfs.KettleVFS.getFileObject(KettleVFS.java:154)
2015/09/07 11:56:04 - Hadoop Copy Files - at org.pentaho.di.core.vfs.KettleVFS.getFileObject(KettleVFS.java:102)
2015/09/07 11:56:04 - Hadoop Copy Files - at org.pentaho.di.job.entries.copyfiles.JobEntryCopyFiles.ProcessFileFolder(JobEntryCopyFiles.java:421)
2015/09/07 11:56:04 - Hadoop Copy Files - at org.pentaho.di.job.entries.copyfiles.JobEntryCopyFiles.execute(JobEntryCopyFiles.java:375)
2015/09/07 11:56:04 - Hadoop Copy Files - at org.pentaho.di.job.Job.execute(Job.java:716)
2015/09/07 11:56:04 - Hadoop Copy Files - at org.pentaho.di.job.Job.execute(Job.java:859)
2015/09/07 11:56:04 - Hadoop Copy Files - at org.pentaho.di.job.Job.execute(Job.java:532)
2015/09/07 11:56:04 - Hadoop Copy Files - at org.pentaho.di.job.Job.run(Job.java:424)
2015/09/07 11:56:04 - pentaho_to_hadoop_ex3ktr - Finished job entry [Hadoop Copy Files] (result=[false])
2015/09/07 11:56:04 - pentaho_to_hadoop_ex3ktr - Job execution finished
2015/09/07 11:56:04 - Spoon - Job has ended.

Copyright License:
Author:「Harinath Arasu」,Reproduced under the CC 4.0 BY-SA copyright license with link to original source & disclaimer.
Link to:https://stackoverflow.com/questions/32442948/pentaho-unable-to-copy-files-to-hadoop-hdfs-file-system-1-0-3

About “Pentaho unable to copy files to Hadoop HDFS file system 1.0.3” questions

This is my first thread and am using using 5.4.0.1-130 Pentaho kettle version. I have installed hadoop-1.0.3 version in a VM player and I have bridged it using bridged network. I have Pentaho ins...
I am fairly new to Hadoop and I have been trying to setup my local machine and run few examples to understand how the process works. I have setup hadoop 1.0.3 on my MAC.I have a series of question ...
Using hadoop streaming: hadoop jar hadoop/contrib/streaming/hadoop-streaming-1.0.3.jar -file mapper.rb -mapper mapper.rb -file reducer.rb -reducer reducer.rb -input textfile.txt -output output As...
I am trying to do hadoop-mapreduce in pentaho.I have hadoopcopyfiles step in a job to specify input path of file.All works fine if my input file location is with root access.(ie.)files created alre...
I am new to hadoop ,i am using apache hadoop 1.0.3 and using redhat linux 6.0 vm ,while i am trying to load a sample file from local windows 7 to Hadoop 'HDFS' file system, ,i used the sample guide...
I need to copy a folder from local file system to HDFS. I could not find any example of moving a folder(including its all subfolders) to HDFS $ hadoop fs -copyFromLocal /home/ubuntu/Source-Folder-To-
I'm trying to copy files from a local file system to the HDFS and it throws up an error indicating the file could only be replicated to 0 nodes instead of 1. ckharide-mac:bin hadoop$ ./hadoop dfs -
I'm trying to retrieve data from an standalone Hadoop (version 2.7.2 qith properties configured by default) HDFS using Pentaho Kettle (version 6.0.1.0-386). Pentaho and Hadoop are not in the same m...
I see hadoop is split into three branches, and the release notes contain HDFS fixes pointing to a HDFS release. Does anyone know the website where I can definitively find out the answer to: hadoop...
I have installed hadoop on Ubuntu 14.04. I am getting the following error whenever I copy files from the local file system to HDFS. I am using this command: /usr/local/hadoop$ bin/hadoop dfs -

Copyright License:Reproduced under the CC 4.0 BY-SA copyright license with link to original source & disclaimer.