Is it possible to save files in Hadoop without saving them in local file system?

2018-07-27T03:17:43

Is it possible to save files in Hadoop without saving them in local file system? I would like to do something like shown below however I would like to save file directly in HDFS. At the moment I save files in documents directory and only then I can save them in HDFS for instance using hadoop fs -put.

class DataUploadView(GenericAPIView):

    def post(self, request):

            myfile = request.FILES['photo']
            fs = FileSystemStorage(location='documents/')
            filename = fs.save(myfile.name, myfile)
            local_path = 'my/path/documents/' + str(myfile.name)            
            hdfs_path = '/user/user1/' + str(myfile.name)
            run(['hadoop', 'fs', '-put', local_path, hdfs_path], shell=True)            

Copyright License:
Author:「thedbogh」,Reproduced under the CC 4.0 BY-SA copyright license with link to original source & disclaimer.
Link to:https://stackoverflow.com/questions/51546108/is-it-possible-to-save-files-in-hadoop-without-saving-them-in-local-file-system

About “Is it possible to save files in Hadoop without saving them in local file system?” questions

Is it possible to save files in Hadoop without saving them in local file system? I would like to do something like shown below however I would like to save file directly in HDFS. At the moment I save
I have a python script which saves a spark dataframe as csv to HDFS. df.write.format('com.databricks.spark.csv').save(filename, header='true', sep=',') I then need to combine the part .csv files p...
I am trying to save file in Hadoop with python 2.7. I searched on the internet. I got some code to save a file in Hadoop but it is taking the entire folder while saving (total files in the folder are
I have a scenario where I process 1000's of small files using Hadoop. The output of the Hadoop job is then to be used as input for a non-Hadoop algorithm. In the current workflow, data is read, con...
One easy way is to create a directory and populate it with files. Then archive and compress that directory into a zip file called, say, file.zip. But this approach is needless since my files are in
Is it possible to save a file to the users local Downloads folder without prompting them with a save as dialog? I have an application where a user right mouse clicks to choose an option to save a
I am learning hadoop. I came across a problem now. I ran the mapreduce job and output was stored in multiple files but not as single file. I want to append all of them into a single file in hdfs. I...
I have NOT installed hadoop on my Linux file System. I would like to run hadoop and copy the file from local file system to HDFS WITHOUT installing hadoop on my Linux file System. I have created a ...
is it possible to upload file/image without saving it in local file system in selenum webdriver ? Can we use in memory stream or file/image url in send_keys args ? I am trying to do this driver.
Hi I had successfully installed hadoop in pseudo-distributed mode on a VM. I am writing code in eclipse and than exporting as jar file onto hadoop cluster and than doing debugging there. Now just for

Copyright License:Reproduced under the CC 4.0 BY-SA copyright license with link to original source & disclaimer.