自己编译hadoop:x64
1.安装依赖包
sudo apt-get install g++ autoconf automake libtool cmake zlib1g-dev pkg-config libssl-dev openssh-server maven openssh-client
2.下载hadoop源码
wget http://mirrors.hust.edu.cn/apache/hadoop/common/hadoop-2.4.0/hadoop-2.4.0-src.tar.gz
3. 安装protobuf 2.5版本
wget https://protobuf.googlecode.com/files/protobuf-2.5.0.tar.gz
cd protobuf-2.5.0/
./configure --prefix=/usr
Make
Make install
4.生成登录密钥:
cd ~
ssh-keygen -t rsa -P ""
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
检测是否能够登录
ssh localhost
5.编译hadoop:
tar zxvf hadoop-2.4.0-src.tar.gz
cd hadoop-2.4.0-src
vi hadoop-common-project/hadoop-auth/pom.xml
确认下述行存在
mvn clean package -Pdist,native -DskipTests –Dtar
在网络较好的情况下需要1小时左右。
成功后tar包在hadoop-2.2.0-src/hadoop-dist/target下面
6.我编译成功的版本:链接