hadoop-2.6.0-cdh5.7.0源码编译支持压缩

5次阅读

共计 1678 个字符,预计需要花费 5 分钟才能阅读完成。

在 hadoop-2.6.0-cdh5.7.0 源码中有个 BUILDING.txt 文件,里面列出了编译所需依赖组件。

Requirements:

Unix System
JDK 1.7+
Maven 3.0 or later
Findbugs 1.3.9 (if running findbugs)
ProtocolBuffer 2.5.0
CMake 2.6 or newer (if compiling native code), must be 3.0 or newer on Mac
Zlib devel (if compiling native code)
openssl devel (if compiling native hadoop-pipes)
Internet connection for first build (to fetch all Maven and Hadoop dependencies)

转载链接文章中或通过 yum 安装或通过自己下载提供了所有正确版本组件,并将已下载组件通过百度云分享。我想补充一点的是,yum 安装可能报出 Another app is currently holding the yum lock; waiting for it to exit… 的错误,可参考 https://blog.csdn.net/testcs_… 强制关闭 yum 进程来解决。
根据 BUILDING.txt 提示

Build options:
Use -Pnative to compile/bundle native code * Use -Pdocs to
generate & bundle the documentation in the distribution (using -Pdist)
Use -Psrc to create a project source TAR.GZ * Use -Dtar to create a TAR with the distribution (using -Pdist)
Building distributions:
Create binary distribution without native code and withoutdocumentation:
$ mvn package -Pdist -DskipTests -Dtar
Create binary distribution with native code and with documentation:
$ mvn package -Pdist,native,docs -DskipTests -Dtar

使用 mvn clean package -Pdist,native -DskipTests -Dtar 编译,编译完成后,默认会在源码根目录的 hadoop-dist 目录下生成 target,里面的 hadoop-2.6.0-cdh5.7.0 文件夹就是已经编译好并已解压的 hadoop 了(直接用的),可以将其拷贝到自定义的位置进行部署配置。
[root@NN1 hadoop-2.6.0-cdh5.7.0]# ./bin/hadoop checknative
19/04/10 11:22:34 INFO bzip2.Bzip2Factory: Successfully loaded & initialized native-bzip2 library system-native
19/04/10 11:22:34 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library
Native library checking:
hadoop: true /root/hadoop-c/hadoop-2.6.0-cdh5.7.0-target/hadoop-2.6.0-cdh5.7.0/lib/native/libhadoop.so.1.0.0
zlib: true /lib64/libz.so.1
snappy: true /lib64/libsnappy.so.1
lz4: true revision:99
bzip2: true /lib64/libbz2.so.1
openssl: true /lib64/libcrypto.so

可以看到编译的 hadoop 支持了各种压缩,亲测有效!

正文完
 0