解决 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform...警告
解决 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable警告问题
先翻译:警告util.NativeCodeLoader：无法为您的平台加载本机Hadoop库…在适用的情况下使用内置的java类
这个警告实在是虽然可能没有什么阻挡使用Hadoop的
/但是它会在你启动Hadoop时候出现，还会在你输入每一个命令enter之后出现/
/在我配置完Hadoop的环境之后就出现了这个警告，/希望我自己只是个例，大家能够没有这个问题，如果有，那就跟着我一起解决吧
- 首先== 判断好自己的警告是否和我的一样==这个很重要，有的教程通过升级glib库达到消除警告的目的，但是我们并不清楚当前我们的Hadoop的期望是哪个版本的glib，所以需要保守起见还是不要升级，先判断自己的问题，如果不符合我的问题那么可以看大佬博客
- 先进入这个目录
/usr/local/hadoop/lib/native
看这个目录下都存了什么
可以看出，libhadoop.so 存放在这个目录下
- 接下来我们回到/usr/local/hadoop目录
紧接着输入以下代码
grep -R 'java.library.path' *
可以看到：
bin/hadoop: hadoop_add_subcommand "jnipath" client "prints the java.library.path"
bin/hadoop.cmd: @echo jnipath prints the java.library.path
bin/yarn.cmd: set YARN_OPTS=%YARN_OPTS% -Djava.library.path=%JAVA_LIBRARY_PATH%
etc/hadoop/yarn-env.cmd: set YARN_OPTS=%YARN_OPTS% -Djava.library.path=%JAVA_LIBRARY_PATH%
etc/hadoop/shellprofile.d/example.sh:# and into the java.library.path system property. In the majority
libexec/hadoop-functions.sh: hadoop_add_param HADOOP_OPTS java.library.path \
libexec/hadoop-functions.sh: "-Djava.library.path=${JAVA_LIBRARY_PATH}"
libexec/hadoop-config.cmd:@rem setup 'java.library.path' for native hadoop code if necessary
libexec/hadoop-config.cmd: set HADOOP_OPTS=%HADOOP_OPTS% -Djava.library.path=%JAVA_LIBRARY_PATH%
share/doc/hadoop/hadoop-yarn/hadoop-yarn-site/WritingYarnApplications.html:<p>Setting <tt>-Djava.library.path</tt> on the command line while launching a container can cause native libraries used by Hadoop to not be loaded correctly and can result in errors. It is cleaner to use <tt>LD_LIBRARY_PATH</tt> instead.</p></div></div>
share/doc/hadoop/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml: Usage of -Djava.library.path can cause programs to no longer function if
share/doc/hadoop/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml: Usage of -Djava.library.path can cause programs to no longer function if
share/doc/hadoop/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml: Usage of -Djava.library.path can cause programs to no longer function if
share/doc/hadoop/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html:<p>The child-task inherits the environment of the parent <tt>MRAppMaster</tt>. The user can specify additional options to the child-jvm via the <tt>mapreduce.{map|reduce}.java.opts</tt> and configuration parameter in the <tt>Job</tt> such as non-standard paths for the run-time linker to search shared libraries via <tt>-Djava.library.path=<></tt> etc. If the <tt>mapreduce.{map|reduce}.java.opts</tt> parameters contains the symbol <i>@taskid@</i> it is interpolated with value of <tt>taskid</tt> of the MapReduce task.</p>
share/doc/hadoop/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html:<p>Here is an example with multiple arguments and substitutions, showing jvm GC logging, and start of a passwordless JVM JMX agent so that it can connect with jconsole and the likes to watch child memory, threads and get thread dumps. It also sets the maximum heap-size of the map and reduce child jvm to 512MB & 1024MB respectively. It also adds an additional path to the <tt>java.library.path</tt> of the child-jvm.</p>
share/doc/hadoop/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html: -Xmx512M -Djava.library.path=/home/mycompany/lib -verbose:gc -Xloggc:/tmp/@taskid@.gc
share/doc/hadoop/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html: -Xmx1024M -Djava.library.path=/home/mycompany/lib -verbose:gc -Xloggc:/tmp/@taskid@.gc
share/doc/hadoop/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html:<p>The <a href="#DistributedCache">DistributedCache</a> can also be used to distribute both jars and native libraries for use in the map and/or reduce tasks. The child-jvm always has its <i>current working directory</i> added to the <tt>java.library.path</tt> and <tt>LD_LIBRARY_PATH</tt>. And hence the cached libraries can be loaded via <a class="externalLink" href="http://docs.oracle.com/javase/7/docs/api/java/lang/System.html">System.loadLibrary</a> or <a class="externalLink" href="http://docs.oracle.com/javase/7/docs/api/java/lang/System.html">System.load</a>. More details on how to load shared libraries through distributed cache are documented at <a href="../../hadoop-project-dist/hadoop-common/NativeLibraries.html#Native_Shared_Libraries">Native Libraries</a>.</p></div></div>
share/doc/hadoop/hadoop-project-dist/hadoop-common/NativeLibraries.html:<p>The bin/hadoop script ensures that the native hadoop library is on the library path via the system property: <tt>-Djava.library.path=<path></tt></p>
share/doc/hadoop/hadoop-project-dist/hadoop-common/CommandsManual.html:<p>Print the computed java.library.path.</p></div>
share/doc/hadoop/hadoop-project-dist/hadoop-common/release/0.23.7/CHANGES.0.23.7.html:<td align="left"> Warn if java.library.path is used for AM or Task </td>
share/doc/hadoop/hadoop-project-dist/hadoop-common/release/0.10.1/CHANGES.0.10.1.html:<td align="left"> java.library.path is wrongly initialized by bin/hadoop when only pre-built libs are present, but custom-built ones aren’t </td>
share/doc/hadoop/hadoop-project-dist/hadoop-common/release/0.10.1/CHANGES.0.10.1.html:<td align="left"> native libraries aren’t loaded unless the user specifies the java.library.path in the child jvm options </td>
share/doc/hadoop/hadoop-project-dist/hadoop-common/release/2.0.3-alpha/CHANGES.2.0.3-alpha.html:<td align="left"> Warn if java.library.path is used for AM or Task </td>
share/doc/hadoop/hadoop-project-dist/hadoop-common/release/2.0.3-alpha/CHANGES.2.0.3-alpha.html:<td align="left"> Fix SEGV when libsnappy is in java.library.path but not LD_LIBRARY_PATH </td>
share/doc/hadoop/hadoop-project-dist/hadoop-common/release/0.23.0/CHANGES.0.23.0.html:<td align="left"> ContainerLocalizer should get the proper java.library.path from LinuxContainerExecutor </td>
share/doc/hadoop/hadoop-project-dist/hadoop-common/release/2.0.2-alpha/CHANGES.2.0.2-alpha.html:<td align="left"> User set java.library.path seems to overwrite default creating problems native lib loading </td>
share/doc/hadoop/hadoop-project-dist/hadoop-common/release/2.0.2-alpha/RELEASENOTES.2.0.2-alpha.html:<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-4072">MAPREDUCE-4072</a> | <i>Major</i> | <b>User set java.library.path seems to overwrite default creating problems native lib loading</b></li>
share/doc/hadoop/hadoop-project-dist/hadoop-common/release/2.0.2-alpha/RELEASENOTES.2.0.2-alpha.html:<p>-Djava.library.path in mapred.child.java.opts can cause issues with native libraries. LD_LIBRARY_PATH through mapred.child.env should be used instead.</p><hr />
share/doc/hadoop/hadoop-project-dist/hadoop-common/release/0.10.0/CHANGES.0.10.0.html:<td align="left"> TaskRunner.run() doesn’t pass along the ‘java.library.path’ to the child (task) jvm </td>
share/doc/hadoop/hadoop-project-dist/hadoop-common/release/0.22.0/CHANGES.0.22.0.html:<td align="left"> java.library.path missing basedir </td>
share/doc/hadoop/hadoop-project-dist/hadoop-common/release/0.14.0/CHANGES.0.14.0.html:<td align="left"> possible double setting of java.library.path introduced by HADOOP-838 </td>
share/doc/hadoop/hadoop-project-dist/hadoop-common/release/0.23.3/RELEASENOTES.0.23.3.html:<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-4072">MAPREDUCE-4072</a> | <i>Major</i> | <b>User set java.library.path seems to overwrite default creating problems native lib loading</b></li>
share/doc/hadoop/hadoop-project-dist/hadoop-common/release/0.23.3/RELEASENOTES.0.23.3.html:<p>-Djava.library.path in mapred.child.java.opts can cause issues with native libraries. LD_LIBRARY_PATH through mapred.child.env should be used instead.</p><hr />
share/doc/hadoop/hadoop-project-dist/hadoop-common/release/0.23.3/CHANGES.0.23.3.html:<td align="left"> User set java.library.path seems to overwrite default creating problems native lib loading </td>
在hadoop包中搜索发现，java.library.path被定义为$JAVA_LIBRARY_PATH，但是我们没有定义JAVA_LIBRARY_PATH
那么问题就好解决了，配置这个环境变量就好啦
- 输入代码
vim ~/.bashrc
进入编辑 在最上面输入
export JAVA_LIBRARY_PATH=/usr/local/hadoop/lib/native
接着保存退出
然后输入
source ~/.bashrc
使环境变量配置有效
然后再试验一下，重启Hadoop时已经没有警告了