Hadoop(CDH)와 Apache Zeppline 연동하기
- IT 정보/Hadoop Eco System
- 2019. 7. 20.
1. Apache Zeppline 다운로드 및 압축 풀기
https://zeppelin.apache.org/download.html
Download
Note: From Zeppelin version 0.6.2, Spark interpreter in binary package is compatible with Spark 2.0 & Scala 2.11 and Spark 1.6(or previous) & Scala 2.10. You can use even different version of Spark at the same time if you set different SPARK_HOME in interp
zeppelin.apache.org
wget http://apache.mirror.cdnetworks.com/zeppelin/zeppelin-0.8.1/zeppelin-0.8.1-bin-all.tgz
tar xfvz zeppelin-0.8.1-bin-all.tgz
mv xfvz zeppelin-0.8.1-bin-all.tgz /opt/zeppelin
2. 환경설정
cp zeppelin-env.sh.template zeppelin-env.sh
vi zeppelin-env.sh
~~~~~~~~~~~~~~~~~~~~~~~~~~~ 하단 입력
export JAVA_HOME="/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.181-7.b13.el7.x86_64/"
export HADOOP_CONF_DIR="/etc/hadoop/conf"
export SPARK_HOME="/opt/cloudera/parcels/CDH/lib/spark"*Zeppline Server address, port 등 변경시에는 zeppelin-site.xml 파일 설정 변경
3. Zeppelin 서버 시작
cd /opt/zeppelin/bin
./zeppelin-daemon.sh start4. Log 확인
tail -f /opt/zeppelin/logs/zeppelin-root-dhlee-un01.dhleecluster.com.log