How to use Azure Storage Emulator for Hadoop?

2015-06-27T01:29:27

I have a HDP cluster as well as the Azure Storage Emulator installed on my machine and would like to access the Azure Storage Emulator from my hadoop job. Obviously this is not default HDFS for my Hadoop cluster. I know that if I had HDInsight emulator I was able to access that because it automatically set the configuration. I wonder to know what configurations I need to set in my hadoop cluster in order to access the azure storage emulator?

When I run 'hdfs dfs -ls wasb://127.0.0.1:10000/' I get the following error message

org.apache.hadoop.fs.azure.AzureException: Unable to access container $root in account 127.0.0.1:10000 using anonymous credentials, and no credentials found for them  in the configuration.

Copyright License:
Author:「HHH」,Reproduced under the CC 4.0 BY-SA copyright license with link to original source & disclaimer.
Link to:https://stackoverflow.com/questions/31078923/how-to-use-azure-storage-emulator-for-hadoop

About “How to use Azure Storage Emulator for Hadoop?” questions

I have a HDP cluster as well as the Azure Storage Emulator installed on my machine and would like to access the Azure Storage Emulator from my hadoop job. Obviously this is not default HDFS for my ...
I have a scala codebase where i am accessing azure blob files using Hadoop FileSystem Apis (and not the azure blob web client). My usage is of the format: val hadoopConfig = new Configuration()
I'm trying to run Azure Function locally using Visual Studio and doing timed trigger, and I have set this in local.settings.json "AzureWebJobsStorage": "UseDevelopmentStorage=true", But when I ru...
I have some basic clarifications about azure hdInsight. The following article gives some basic input on using hdinsight. https://azure.microsoft.com/en-in/documentation/articles/hdinsight-hadoop-em...
Currently I am using MSI in my code. But when I test it locally I want to use azure storage emulator and blob storage in it. MSI is working well with the cloud storage, but when it comes to the local
We use Azure Storage Emulator on the development machines and the CI server to be able to use storage queues locally. Now every time I sign out of Windows or reboot, I need to start the storage emu...
I am putting together a website which I would like to host static content via Azure Blob. The documentation is very clear on how to set "Public read access for blobs only" to a container via this
This error is really driving me crazy. (Terminal running in administrator mode) Initialization of azure storage emulator in sql server 2014: C:\Program Files (x86)\Microsoft SDKs\Windows Azure\St...
I am trying to implement the Azure Storage Emulator in Laravel 5.8 It works on Azure dev & production, but not on Windows locally. Postman returns "message": "Fail:\nCode: 403\nValue: Server ...
Is it possible to access the local Azure Storage Emulator from an Android Emulator running the Azure Client libraries (xamarin client)? I'm running into connection issues. Perhaps by specifying a ...

Copyright License:Reproduced under the CC 4.0 BY-SA copyright license with link to original source & disclaimer.