安装protobuf
#下载后解压,进入到根目录
$ ./configure --prefix=/usr
$ sudo make
$ sudo make check
$ sudo make install
#最后可以查询一下版本,显示版本表示安装成功
nemodeMacBook-Pro:git nemo$ protoc --version
libprotoc 2.5.0
hadoop-2.6.0源码编译
#解压hadoop-2.6.0-src
$ cd $HADOOP_SRC_HOME
$ mvn package -Pdist,native -DskipTests -Dtar
$ cd $HADOOP-2.6.0-src/hadoop-dist/target
#jar包生成在这个目录
hadoop-2.6.0项目导入到eclipse
1、生成eclipse工程文件
#进入到源码目录
$ cd hadoop-maven-plugins
$ mvn install
$ cd hadoop-2.5.0-src
$ mvn eclipse:eclipse -DskipTests
遇到错误:
Exception in thread "main" java.lang.AssertionError: Missing tools.jar at: /Library/Java/JavaVirtualMachines/jdk1.7.0_71.jdk/Contents/Home/Classes/classes.jar. Expression: file.exists()
解决办法:
$ sudo mkdir $JAVA_HOME/Classes
$ sudo ln -sf $JAVA_HOME/lib/tools.jar $JAVA_HOME/Classes/classes.jar
2、导入到eclipse
File >> Import >> Existing Projects into Workspace,选择hadoop-2.6.0-src,然后确认。
3、项目错误解决
错误1:org.apache.hadoop.ipc.protobuf.× 无法解析
$ cd hadoop-2.6.0-src/hadoop-common-project/hadoop-common/src/test/proto
$ protoc --java_out=../java *.proto
错误2:AvroRecord cannot be resolved to a type TestAvroSerialization.java
下载 avro-tools-1.7.4.jar,放到hadoop-2.6.0-src目录下
$ cd hadoop-2.6.0-src/hadoop-common-project/hadoop-common/src/test/avro
$ java -jar ~/hadoop-2.6.0-src/avro-tools-1.7.4.jar compile schema avroRecord.avsc ../java
错误3:Project 'hadoop-streaming' is missing required source ... Build Path Problem
右键 hadoop-streaming 项目->properties->左边Java Build Path->Source->选定错误项->右边Remove