맥북에 하둡 설치하기. (하둡2)

레퍼러스 문서가 조금 구버전이라 놀라긴했다만,

뭐 문제는 없으니까 시작해보도록하자.

1. Download Java 1.7 JDK and install it

하둡 구동에 필요한 자바(개인적으로 현재까지는 자바8을 추천)

2. Install protobuf 2.5.0 via Homebrew. Refer to link

$ brew tap homebrew/versions
$ brew install protobuf250
$ brew link --force --overwrite protobuf250
$ protoc --version
libprotoc 2.5.0

brew 로 손쉽게 다운받는게 이 얼마나 고마운 일인가.

3. Install cmake via Homebrew

$ brew install cmake

4. Install Hadoop 2.7.2 via Homebrew

$ brew install hadoop

심지어 하둡도 브루로 다운받을 수 있다니 세상에나...

5. Update Hadoop’s native libraries. Refer to link

$ wget http://apache.tt.co.kr/hadoop/common/hadoop-2.7.2/hadoop-2.7.2-src.tar.gz 
$ tar xvf hadoop-2.7.2-src.tar.gz
$ cd hadoop-2.7.2-src
$ mvn package -Pdist,native -DskipTests -Dtar
$ mv hadoop-dist/target/hadoop-2.7.2/lib /usr/local/Cellar/hadoop/2.7.2/

하.. 패키징 부분에서 1번 에러가 떳다..

6. Open hadoop-env.sh and add below variables

# /usr/local/Cellar/hadoop/2.7.2/libexec/etc/hadoop/hadoop-env.sh

export HADOOP_HOME="/usr/local/Cellar/hadoop/2.7.2"
export HADOOP_OPTS="$HADOOP_OPTS -Djava.library.path=$HADOOP_HOME/lib/native"

7. Edit core-site.xml

<configuration>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://localhost:9000</value>
    </property>
</configuration>

8. Edit hdfs-site.xml

<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
</configuration>

9. Activate Remote Login option. Refer to this link

10. Set passphraseless ssh

$ ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa
$ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
$ ssh localhost

11. Format filesystem

$ hdfs namenode -format

12. Execute NameNode daemon, DataNode daemon

$ /usr/local/Cellar/hadoop/2.7.2/sbin/start-dfs.sh

13. Stop NameNode daemon, DataNode daemon

$ /usr/local/Cellar/hadoop/2.7.2/sbin/stop-dfs.sh