max系统编译hadoop-2.6.0源码以及导入到eclipse中


安装protobuf

点击从百度云网盘下载

#下载后解压,进入到根目录
$ ./configure --prefix=/usr
$ sudo make
$ sudo make check
$ sudo make install
#最后可以查询一下版本,显示版本表示安装成功
nemodeMacBook-Pro:git nemo$ protoc --version
libprotoc 2.5.0

hadoop-2.6.0源码编译

#解压hadoop-2.6.0-src
$ cd $HADOOP_SRC_HOME
$ mvn package -Pdist,native -DskipTests -Dtar
$ cd $HADOOP-2.6.0-src/hadoop-dist/target
#jar包生成在这个目录

hadoop-2.6.0项目导入到eclipse

1、生成eclipse工程文件

#进入到源码目录
$ cd hadoop-maven-plugins  
$ mvn install $ cd hadoop-2.5.0-src $ mvn eclipse:eclipse -DskipTests

遇到错误:

Exception in thread "main" java.lang.AssertionError: Missing tools.jar at: /Library/Java/JavaVirtualMachines/jdk1.7.0_71.jdk/Contents/Home/Classes/classes.jar. Expression: file.exists()

解决办法:

$ sudo mkdir $JAVA_HOME/Classes
$ sudo ln -sf $JAVA_HOME/lib/tools.jar $JAVA_HOME/Classes/classes.jar

2、导入到eclipse
File >> Import >> Existing Projects into Workspace,选择hadoop-2.6.0-src,然后确认。

3、项目错误解决
错误1:org.apache.hadoop.ipc.protobuf.× 无法解析

$ cd hadoop-2.6.0-src/hadoop-common-project/hadoop-common/src/test/proto
$ protoc --java_out=../java *.proto


错误2:AvroRecord cannot be resolved to a type TestAvroSerialization.java
下载 avro-tools-1.7.4.jar,放到hadoop-2.6.0-src目录下

$ cd hadoop-2.6.0-src/hadoop-common-project/hadoop-common/src/test/avro
$ java -jar ~/hadoop-2.6.0-src/avro-tools-1.7.4.jar compile schema avroRecord.avsc ../java


错误3:Project 'hadoop-streaming' is missing required source ... Build Path Problem
右键 hadoop-streaming 项目->properties->左边Java Build Path->Source->选定错误项->右边Remove

nemotan /
Published under (CC) BY-NC-SA in categories hadoop  tagged with hadoop