Hadoop on Azure using IaaS

2018-01-27T03:22:59

I am looking at having a Hadoop cluster setup for Big Data analytics using the virtualized environment in Azure. As the data volume is very high, I am looking at having data stored in secondary storage like Azure Data Lake Store and Hadoop cluster storage will act as the primary storage. I would like to know, how can this be configured so that when i create a Hive table and partition, part of the data can reside in Primary storage and the rest in the secondary storage?

Thanks Regards, Madhu

Copyright License:
Author:「Madhusudhanan Padmanabhan」,Reproduced under the CC 4.0 BY-SA copyright license with link to original source & disclaimer.
Link to:https://stackoverflow.com/questions/48468187/hadoop-on-azure-using-iaas

About “Hadoop on Azure using IaaS” questions

I am looking at having a Hadoop cluster setup for Big Data analytics using the virtualized environment in Azure. As the data volume is very high, I am looking at having data stored in secondary sto...
I am trying to use Polybase in Azure SQLDW to access delimited files in HDFS in Cloudera Cluster (provisioned as IaaS in Azure) however I am running into the following error: Msg 105019, Level 16,
This is regarding Microsoft's Azure DevOps(Formerly VSTS). Just wanted to clear things, can Azure DevOps be classified as a Platform as a Service. Since it is a cloud service it should be categorized
We have a public website hosted on two Azure IaaS VMs which are behind a Network Load Balancer. What are the available solutions to auto patch and reboot without impacting site availability? I am
I have already implemented production bigdata solutions mainly on premises using Hadoop and NoSQL products but never on Cloud. Today I need to turn into the cloud, hence i'm wandering what are the...
What do the following terms mean? SaaS PaaS IaaS? There are various cloud services available today, such as Amazon's EC2 and AWS, Apache Hadoop, Microsoft Azure and many others. Which category d...
I published a web api in Azure via Visual Studio. I would like to know if it is PaaS or IaaS. When I search for the answer, I find lot of definitions about the PaaS and IaaS. But I did not find ...
I will be migrating databases from Azure IaaS infrastructure using mssql to Azure SQL. After migration I need a method to efficiently rollback to IaaS infrastructure if issues are encountered minutes/
I would like some clarification on whether Azure VM's are strictly IaaS or can be PaaS depending on the amount responsibilities the creator/user has. I've seen multiple different website saying V...
I know its possible to inject certificates to PAAS worker roles in azure. In below video at time 0:31:00 talks about injecting service certificates to IAAS and PAAS both. Is it possible for IAAS li...

Copyright License:Reproduced under the CC 4.0 BY-SA copyright license with link to original source & disclaimer.