Hive Table Creation Using MongoDB Hadoop Driver


I am trying to connect from a Hive Database to a collection in MongoDB using a driver (jars) provided on the wiki site. Here are the steps I did: -

I created a collection in MongoDB called "Diamond" under a database called "Moe" and it has got 20 documents:

I wanted to connect from Hive via the Hadoop MongoDB Driver and view these documents via Hive.

I have both MongoDB and Hive installed on the same server and configured. However I don't see any variable called the HIVE_CLASPATH I wonder where that is.

So I installed 3 divers on the server: -

Now, I connect to Hive, and then add these 2 jar's to my classpath by the following commands: - (they get added successfully)

add jar /hadoopgdc/hadoop-2.6.0/share/hadoop/common/lib/mongo-hadoop-hive-1.5.2.jar;
add jar /hadoopgdc/hadoop-2.6.0/share/hadoop/common/lib/mongo-hadoop-core-1.5.2.jar;
add jar /hadoopgdc/hadoop-2.6.0/share/hadoop/common/lib/mongo-java-driver-3.0.0.jar;

Now I create a table in HIVE: -

carat    DOUBLE,
cut      STRING,
color    STRING,
clarity  STRING,
depth    DOUBLE,
table    DOUBLE,
price    DOUBLE,
xcord    DOUBLE,
ycord    DOUBLE,
zcord    DOUBLE
STORED BY 'com.mongodb.hadoop.hive.MongoStorageHandler'
WITH SERDEPROPERTIES('mongo.columns.mapping'='{"carat":"carat","cut":"cut",                 
"color":"color", "clarity":"clarity", "depth":"depth", "table":"table", 
"price":"price", "xcord":"x", "ycord":"y", "zcord":"z"}')

However when I execute the above command in Hive I get the error below: -

java.lang.NoClassDefFoundError: com/mongodb/util/JSON
    at com.mongodb.hadoop.hive.BSONSerDe.initialize(
    at     org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(
    at org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(
    at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(
    at org.apache.hadoop.hive.ql.metadata.Table.getCols(
    at org.apache.hadoop.hive.ql.metadata.Hive.createTable(
    at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(
    at org.apache.hadoop.hive.ql.exec.DDLTask.execute(
    at org.apache.hadoop.hive.ql.exec.Task.executeTask(
    at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(
    at org.apache.hadoop.hive.ql.Driver.launchTask(
    at org.apache.hadoop.hive.ql.Driver.execute(
    at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(
    at org.apache.hadoop.hive.cli.CliDriver.processCmd(
    at org.apache.hadoop.hive.cli.CliDriver.processLine(
    at org.apache.hadoop.hive.cli.CliDriver.main(
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(
    at  sun.reflect.DelegatingMethodAccessorImpl.invoke(
    at java.lang.reflect.Method.invoke(
    at org.apache.hadoop.util.RunJar.main(
    Caused by: java.lang.ClassNotFoundException: com.mongodb.util.JSON
    at Method)
    at java.lang.ClassLoader.loadClass(
    at sun.misc.Launcher$AppClassLoader.loadClass(
    at java.lang.ClassLoader.loadClass(
    ... 23 more
    FAILED: Execution Error, return code -101 from 

I have tried the following: - - placing the jars in every possible directory with no effect - The class that is supposed to be missing, is pretty much present in the jar file. - oh yes and the MongoStorageHandler class is very much in the jar.

I am done breaking my head with this !! If anyone can shed some light on what I could do to alleviate my anxiety, it would be great.

Thanks again. Mario

Copyright License:
Author:「Mario」,Reproduced under the CC 4.0 BY-SA copyright license with link to original source & disclaimer.
Link to:

About “Hive Table Creation Using MongoDB Hadoop Driver” questions

I am trying to connect from a Hive Database to a collection in MongoDB using a driver (jars) provided on the wiki site. Here are the steps I did: - I created a collection in MongoDB called "Diamond"
I am trying to install and configure hive with mongo-hadoop-core 2.0.2, for the first time. I have installed hadoop 2.8.0, Hive 2.1.1 and MongoDB 3.4.6. and everything works fine when running
I am currently trying to migrate mongodb collection into hdfs by using hive. I succeeded in creating the mongodb-based hive table and it is present in my hdfs directory when i browse the hdfs UI.
I am trying to integrate mongodb hive and getting this error while executing: Query: Select * from minute_bars; Exception in thread "main" java.lang.NoSuchMethodError: com.mongodb.DB.isAuthentica...
I'm facing a strange error I can't find in any logs on my Azure Hortonworks Sandbox. I'm trying to execute the following how-to to try the Mongodb integration with Hadoop (Hive) : https://www.mongo...
I am using MongoDB hadoop connector to query mongoDB using hive table in hadoop. I am able to execute select * from mongoDBTestHiveTable; But when I try to execute following query select id from
I like to connect an EMR Cluster to our MongoDB via an MongoDB Connection (not via BSON Dumps). For that i spawned the cluster over the AWS Management Console. On the Bootstrap configuration I poi...
I am new to hadoop. I have installed hortonworks sandbox 2.1. I am trying to execute Hive script using Hive UI. I want to access mongo collection inside Hive. I used following query for that: CRE...
I have installed Robomongo on my desktop.But i an mot able to ingest data into a hive table from Robomongo. I have applied the following steps:- downloaded the required jars- mongo-java-driver-2.1...
This is about establishing connectivity between HIVE and MongoDB using the MongoDB driver. I have been through the links online and have noticed that many people just breeze through it. However I...

Copyright License:Reproduced under the CC 4.0 BY-SA copyright license with link to original source & disclaimer.