Why does ./bin/spark-shell give WARN NativeCodeLoader: Unable to load native-hadoop library for your platform?

2014-05-10T03:44:07

On Mac OS X, I compiled Spark from the sources using the following command:

jacek:~/oss/spark
$ SPARK_HADOOP_VERSION=2.4.0 SPARK_YARN=true SPARK_HIVE=true SPARK_GANGLIA_LGPL=true xsbt
...

[info] Set current project to root (in build file:/Users/jacek/oss/spark/)
> ; clean ; assembly
...
[info] Packaging /Users/jacek/oss/spark/examples/target/scala-2.10/spark-examples-1.0.0-SNAPSHOT-hadoop2.4.0.jar ...
[info] Done packaging.
[info] Done packaging.
[success] Total time: 1964 s, completed May 9, 2014 5:07:45 AM

When I started ./bin/spark-shell I noticed the following WARN message:

WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

What might be the issue?

jacek:~/oss/spark
$ ./bin/spark-shell
Spark assembly has been built with Hive, including Datanucleus jars on classpath
14/05/09 21:11:17 INFO SecurityManager: Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
14/05/09 21:11:17 INFO SecurityManager: Changing view acls to: jacek
14/05/09 21:11:17 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(jacek)
14/05/09 21:11:17 INFO HttpServer: Starting HTTP Server
Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /___/ .__/\_,_/_/ /_/\_\   version 1.0.0-SNAPSHOT
      /_/

Using Scala version 2.10.4 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0)
Type in expressions to have them evaluated.
Type :help for more information.
...
14/05/09 21:11:49 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
...

Copyright License:
Author:「Jacek Laskowski」,Reproduced under the CC 4.0 BY-SA copyright license with link to original source & disclaimer.
Link to:https://stackoverflow.com/questions/23572724/why-does-bin-spark-shell-give-warn-nativecodeloader-unable-to-load-native-had

About “Why does ./bin/spark-shell give WARN NativeCodeLoader: Unable to load native-hadoop library for your platform?” questions

On Mac OS X, I compiled Spark from the sources using the following command: jacek:~/oss/spark $ SPARK_HADOOP_VERSION=2.4.0 SPARK_YARN=true SPARK_HIVE=true SPARK_GANGLIA_LGPL=true xsbt ... [info] ...
On Mac OS X, I compiled Spark from the sources using the following command: jacek:~/oss/spark $ SPARK_HADOOP_VERSION=2.4.0 SPARK_YARN=true SPARK_HIVE=true SPARK_GANGLIA_LGPL=true xsbt ... [info] ...
I am following this tutorial. start-all.sh output: hduser@ubuntu12:~/hadoop/sbin$ start-dfs.sh 15/07/27 15:24:11 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform...
alpesh@alpesh-Inspiron-3647:~/hadoop-2.7.2/sbin$ hadoop fs -ls 16/07/05 13:59:17 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where
I'm using Mac OSX El-Capitan, and new to hadoop. After installation I'm getting this warning: WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-j...
[PyCharm/PySpark Error] Getting below error in PyCharm: WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Using Spark's de...
I have problem when i install hadoop on debian jessie, i got warning when run script start-dfs.sh hadoop@debian1:~$ /usr/local/hadoop/sbin/start-dfs.sh 17/03/24 10:40:00 WARN util.NativeCodeLoader:
I'm a dummy on Ubuntu 16.04, desperately attempting to make Spark work. I've tried to fix my problem using the answers found here on stackoverflow but I couldn't resolve anything. Launching spark w...
I'm a dummy on Ubuntu 16.04, desperately attempting to make Spark work. I've tried to fix my problem using the answers found here on stackoverflow but I couldn't resolve anything. Launching spark w...
I am following the tutorial on BigTable: https://cloud.google.com/bigtable/docs/hbase-shell-quickstart When I try to start my Docker image, I get the following error: C:\dev\GoogleCloudBigtable-

Copyright License:Reproduced under the CC 4.0 BY-SA copyright license with link to original source & disclaimer.