...wd......wd......wd...Cloudera大數(shù)據(jù)平臺環(huán)境搭建〔CDH5.13.1版〕版本v1.0作者郭維忠發(fā)布日期2018-03-10目錄Cloudera大數(shù)據(jù)平臺環(huán)境搭建1〔CDH5.13.1版〕11.根基環(huán)境41.1.軟件環(huán)境41.2.配置規(guī)劃41.3.所需要的軟件資源41.4.修改機(jī)器名〔所有節(jié)點(diǎn)〕51.5.設(shè)置防火墻〔所有節(jié)點(diǎn)〕51.6.配置免密碼登錄SSH61.7.關(guān)閉SELINUX〔所有節(jié)點(diǎn)〕81.8.修改Linux內(nèi)核參數(shù)〔所有節(jié)點(diǎn)〕81.9.其他安裝與配置〔所有節(jié)點(diǎn)〕101.10.配置NTP服務(wù)101.11.安裝oracleJDK1.8〔所有節(jié)點(diǎn)〕121.12.Mysql安裝〔主節(jié)點(diǎn)〕132.安裝CM152.1.傳包，解包152.2.創(chuàng)立數(shù)據(jù)庫152.3.創(chuàng)立用戶162.4.制作本地YUM源162.5.拷貝jar包172.6.修改cloudera-scm-agent配置172.7.啟動CMServer和Agent182.8.訪問CM183.安裝CDH183.1.登錄后界面183.2.選擇CM版本193.3.指定主機(jī)203.4.選擇CDH版本213.4.1.出現(xiàn)“主機(jī)運(yùn)行狀態(tài)不良〞錯誤223.5.檢查主機(jī)正確性233.6.選擇安裝的服務(wù)233.7.角色分配243.8.數(shù)據(jù)庫設(shè)置243.8.1.測試連接報(bào)錯：253.9.群集設(shè)置263.10.開場安裝283.11.安裝完成293.11.1.警告信息：294.常見錯誤314.1.初始化數(shù)據(jù)庫錯誤：314.2.未能連接到HostMonitor32根基環(huán)境軟件環(huán)境本文將介紹Centos7.4離線安裝CDH和ClouderaManager過程，軟件版本如下：NO.軟件名稱版本操作系統(tǒng)Centos7.464位JDKjdk-8u151-linux-x64ClouderManager5.13.1CDH5.13.1數(shù)據(jù)庫Mysql5.7JDBCmysql-connector-java-5.1.43.jar配置規(guī)劃本次安裝共5臺服務(wù)器，服務(wù)器配置及用途如下：NO.機(jī)器名稱內(nèi)網(wǎng)IP配置用途node168688C/32Gb/1TB主，CM，MysqlDBnode169698C/32Gb/1TB從node170708C/32Gb/1TB從node171718C/32Gb/1TB從node172728C/32Gb/1TB從所需要的軟件資源JDK環(huán)境：JDK版本：1.8.0_151jdk-8u151-linux-x64.rpm下載地址：://oracle/technetwork/java/javase/downloads/jdk8-downloads-2133151.htmlCM包：CM版本：5.13.1cloudera-manager-centos7-cm5.13.1_x86_64.tar.gz下載地址：://archive-primary.cloudera/cm5/cm/5/cloudera-manager-centos7-cm5.13.1_x86_64.tar.gzCDH包CDH版本：5.13.1，CDH-5.13.1-1.cdh5.13.1.p0.2-el7.parcel；CDH-5.13.1-1.cdh5.13.1.p0.2-el7.parcel.sha1；manifest.json下載地址：://archive-primary.cloudera/cdh5/parcels/5.13.1/manifest.json://archive-primary.cloudera/cdh5/parcels/5.13.1/CDH-5.13.1-1.cdh5.13.1.p0.2-el7.parcel.sha1://archive-primary.cloudera/cdh5/parcels/5.13.1/CDH-5.13.1-1.cdh5.13.1.p0.2-el7.parcelJDBC連接jar包：jar包版本：5.1.43，mysql-connector-java-5.1.43.jar下載地址：:///maven2/mysql/mysql-connector-java/5.1.43/mysql-connector-java-5.1.43.jar修改機(jī)器名〔所有節(jié)點(diǎn)〕修改機(jī)器名稱#hostnamectlset-hostnamexxx//設(shè)置機(jī)器名#hostname//查看本機(jī)名這種方式，在Centos7中可以永久性改變主機(jī)名稱。每個節(jié)點(diǎn)機(jī)器，都要配置hosts#vi/etc/hosts內(nèi)容都改為：localhost//建議保存此行68node16869node16970node17071node17172node172設(shè)置防火墻〔所有節(jié)點(diǎn)〕注：為了簡便，可以先關(guān)閉防所有節(jié)點(diǎn)火墻，安裝完成后，再按照下面步驟設(shè)置。防火墻常用命令：#firewall-cmd--state〔查詢防火墻狀態(tài)〕#systemctlstopfirewalld.service〔關(guān)閉防火墻〕#systemctlstartfirewalld.service〔開啟防火墻〕#systemctldisablefirewalld.service〔制止firewall開機(jī)啟動〕設(shè)置防火墻策略，在所有節(jié)點(diǎn)執(zhí)行下面腳本〔執(zhí)行前要啟動防火墻〕：//集群機(jī)器間可以相互訪問firewall-cmd--permanent--add-rich-rule="rulefamily='ipv4'sourceaddress='68'portprotocol='tcp'port='0-65535'accept"firewall-cmd--permanent--add-rich-rule="rulefamily='ipv4'sourceaddress='69'portprotocol='tcp'port='0-65535'accept"firewall-cmd--permanent--add-rich-rule="rulefamily='ipv4'sourceaddress='70'portprotocol='tcp'port='0-65535'accept"firewall-cmd--permanent--add-rich-rule="rulefamily='ipv4'sourceaddress='71'portprotocol='tcp'port='0-65535'accept"firewall-cmd--permanent--add-rich-rule="rulefamily='ipv4'sourceaddress='72'portprotocol='tcp'port='0-65535'accept"http://設(shè)置可以訪問的端口firewall-cmd--permanent--add-port=0-65535/tcpfirewall-cmd--reload注意：需要在所有的節(jié)點(diǎn)上執(zhí)行，因?yàn)樯婕暗降亩丝谔嗔?，臨時開放所有端口是為了安裝起來方便，安裝完畢后可以根據(jù)需要設(shè)置防火墻策略，保證集群安全。配置免密碼登錄SSH將子節(jié)點(diǎn)設(shè)置為從主節(jié)點(diǎn)ssh無密碼登陸〔主節(jié)點(diǎn)訪問從節(jié)點(diǎn)需要無密碼互通，否那么后面會出現(xiàn)問題〕在主節(jié)點(diǎn)上執(zhí)行：#cd~//進(jìn)入到根目錄#ssh-keygen-trsa一路回車，生成無密碼的密鑰對。把公鑰拷貝至node169服務(wù)器上：#scp/root/.ssh/id_rsa.pubroot@node169:~//輸入node169機(jī)器上root用戶密碼拷備成功登錄到node169服務(wù)器上：然后在node169服務(wù)器上，把公鑰的內(nèi)容追加到authorized_keys文件末尾〔這個文件也在隱藏文件夾.ssh下，如果沒有可以新建〕：#catid_rsa.pub>>~/.ssh/authorized_keys如果提示下面信息，那么需要創(chuàng)立./ssh目錄：-bash:/root/.ssh/authorized_keys:Nosuchfileordirectory//創(chuàng)立/.ssh目錄,再追加公鑰內(nèi)容#mkdir~/.ssh#catid_rsa.pub>>~/.ssh/authorized_keys并設(shè)置authorized_keys的訪問權(quán)限：#chmod600~/.ssh/authorized_keys測試：在node168點(diǎn)上執(zhí)行sshnode169，正常情況下，不需要密碼就能直接登陸進(jìn)去了。在node168節(jié)點(diǎn)出現(xiàn)下面表示成功登錄node169節(jié)點(diǎn)：以同樣的方式將公鑰拷貝至其他節(jié)點(diǎn)服務(wù)器上。關(guān)閉SELINUX〔所有節(jié)點(diǎn)〕關(guān)閉linuxSELINUX安全內(nèi)核#setenforce0〔臨時生效〕修改/etc/selinux/config下的SELINUX=disabled〔重啟后永久生效〕#vi/etc/selinux/config內(nèi)容增加：SELINUX=disabled重啟后生效：#reboot查看SELINUX是否關(guān)閉：#sestatus〔在Centos7中遇到SELINUX無法關(guān)閉的問題，重新啟動機(jī)器恢復(fù)正常〕修改Linux內(nèi)核參數(shù)〔所有節(jié)點(diǎn)〕為防止安裝過程中出現(xiàn)的異常問題，首先調(diào)整Linux內(nèi)核參數(shù)。設(shè)置swappiness，控制換出運(yùn)行時內(nèi)存的相對權(quán)重，Cloudera建議將swappiness設(shè)置為10：//查看swappiness#cat/proc/sys/vm/swappiness//永久性修改，執(zhí)行下面兩條命令#sysctl-wvm.swappiness=10#echovm.swappiness=10>>/etc/sysctl.conf關(guān)閉透明大頁面：自CentOS6版本開場引入了TransparentHugePages(THP)，從CentOS7版本開場，該特性默認(rèn)就會啟用。盡管THP的本意是為提升內(nèi)存的性能，不過某些數(shù)據(jù)庫廠商還是建議直接關(guān)閉THP，否那么可能會導(dǎo)致性能出現(xiàn)下降。首先查看透明大頁是否啟用，[always]never表示已啟用，always[never]表示已禁用：#cat/sys/kernel/mm/transparent_hugepage/defrag[always]madvisenever#cat/sys/kernel/mm/transparent_hugepage/enabled[always]madvisenever以上狀態(tài)就說明是啟用的。臨時關(guān)閉〔重啟機(jī)器會變回默認(rèn)開啟狀態(tài)〕：#echonever>/sys/kernel/mm/transparent_hugepage/defrag#echonever>/sys/kernel/mm/transparent_hugepage/enabled永久關(guān)閉：//編輯/etc/rc.d/rc.local#vi/etc/rc.d/rc.local//在文件后添加下面內(nèi)容:iftest-f/sys/kernel/mm/transparent_hugepage/enabled;thenechonever>/sys/kernel/mm/transparent_hugepage/enabledfiiftest-f/sys/kernel/mm/transparent_hugepage/defrag;thenechonever>/sys/kernel/mm/transparent_hugepage/defragfi保存退出，然后賦予rc.local文件執(zhí)行權(quán)限：#chmod+x/etc/rc.d/rc.local重啟系統(tǒng)，以后再檢查THP狀態(tài)，顯示狀態(tài)被禁用了。修改文件句柄數(shù)：修改系統(tǒng)文件句柄數(shù)限制：//查看文件句柄數(shù)，顯示1024，顯然太小#ulimit-n1024//修改限制#vi/etc/security/limits.conf//在文件后參加下面內(nèi)容：*softnofile100000*hardnofile100000修改后需要重啟機(jī)器。其他安裝與配置〔所有節(jié)點(diǎn)〕為防止安裝過程中出現(xiàn)異常問題，安裝相關(guān)依賴包：#yum-yinstallpsmiscMySQL-pythonatbcbind-libsbind-utilscups-clientcups-libscyrus-sasl-gssapicyrus-sasl-plainedfusefuse-libsdd-toolskeyutils-libs-develkrb5-devellibcom_err-devellibselinux-devellibsepol-devellibverto-develmailcapnoarchmailxmod_sslopenssl-develpcre-develpostgresql-libspython-psycopg2redhat-lsb-coreredhat-lsb-submod-securityx86_64spaxtimezlib-devel#yuminstall-ypython-lxml#yuminstallkrb5-develcyrus-sasl-gssapicyrus-sasl-develibxml2-devellibxslt-develmysqlmysql-developenldap-develpython-develpython-simplejsonsqlite-devel#chmod+x/etc/rc.d/rc.local#yum-yinstallrpcbind#systemctlstartrpcbind#echo"systemctlstartrpcbind">>/etc/rc.d/rc.local配置NTP服務(wù)集群中所有主機(jī)必須保持時間同步，如果時間相差較大會引起各種問題。具體建設(shè)過程如下：所有節(jié)點(diǎn)安裝相關(guān)組件：#yum

install

ntp

ntpdate

-yNTP服務(wù)端〔主節(jié)點(diǎn)〕：查找時間同步服務(wù)器:///zone/asia：翻開網(wǎng)址，內(nèi)容如下：編輯/etc/ntp.conf：#vi/etc/ntp.conf//在文件中輸入上面網(wǎng)頁內(nèi)容：啟動ntp服務(wù)：#systemctlstart

ntpd配置開機(jī)啟動：#systemctl

enable

ntpd.service

注意：如果ntpd開機(jī)啟動失效，有可能是因?yàn)榘惭b了chronyd

并且是開機(jī)自啟狀態(tài)，所以導(dǎo)致ntpd開機(jī)自啟失敗。#查看

chronyd設(shè)置狀態(tài)#systemctlstatuschronyd顯示如下：以面說明，chronyd顯示為開機(jī)啟動狀態(tài)。將chronyd設(shè)為禁用狀態(tài)：#systemctldisablechronyd.service此時，NTP的服務(wù)開機(jī)自啟動完成！檢查是否設(shè)置成功：#ntpq

-p//更新時間#timedatectl

NTP客戶端〔所有從節(jié)點(diǎn)〕：遠(yuǎn)程客戶端時間同步測試#date#ntpdate6868是NTP服務(wù)端IP，顯示如下信息，測試成功：客戶端設(shè)置方案任務(wù)，每30分鐘同步時間#crontab-e//參加內(nèi)容：0-59/30****/usr/sbin/ntpdate68&&/sbin/hwclock-w設(shè)置定時任務(wù)開機(jī)啟動//設(shè)置開機(jī)啟動#systemctlenablecrond.service//查看狀態(tài)#systemctlstatuscrond安裝oracleJDK1.8〔所有節(jié)點(diǎn)〕下載rpm安裝包jdk-8u151-linux-x64.rpmCentOS系統(tǒng)一般自帶OpenJdk，需要卸載掉自帶的OpenJdk，使用#rpm-qa|grepjava查詢java相關(guān)的包，使用#rpm-e–nodeps+包名強(qiáng)制卸載掉，下面安裝oralceJDK#rpm-ivhjdk-8u151-linux-x64.rpm配置JAVA_HOME環(huán)境變量：修改/etc/profile文件配置環(huán)境變量，執(zhí)行命令：#vi/etc/profile在profile文件最后，輸入下面內(nèi)容，保存并退出。exportJAVA_HOME=/usr/java/jdk1.8.0_151exportPATH=$PATH:$JAVA_HOME/bin執(zhí)行下面命令，讓配置生效#source/etc/profile//使修改立即生效#echo$PATH//查看PATH值#echo$JAVA_HOME//查看JAVA_HOME值在Centos7中修改/etc/profile文件配置環(huán)境變量，機(jī)器啟動后有時不生效，可以修改/etc/bashrc文件配置環(huán)境變量：#vi/etc/bashrc//在文件最后參加：exportJAVA_HOME=/usr/java/jdk1.8.0_151exportPATH=$PATH:$JAVA_HOME/bin//使修改立即生效#source/etc/bashrc//查看JAVA_HOME值#echo$JAVA_HOMEMysql安裝〔主節(jié)點(diǎn)〕到官網(wǎng)s://dev.mysql/downloads/file/?id=469456選擇對應(yīng)安裝包：下載安裝包#wgets://cdn.mysql//archives/mysql-5.7/mysql-5.7.18-1.el7.x86_64.rpm-bundle.tar解壓安裝包#tar-xvfmysql-5.7.18-1.el7.x86_64.rpm-bundle.tar//解壓后文件列表：mysql-community-embedded-devel-5.7.18-1.el7.x86_64.rpmmysql-community-devel-5.7.18-1.el7.x86_64.rpmmysql-community-client-5.7.18-1.el7.x86_64.rpmmysql-community-common-5.7.18-1.el7.x86_64.rpmmysql-community-embedded-5.7.18-1.el7.x86_64.rpmmysql-community-embedded-compat-5.7.18-1.el7.x86_64.rpmmysql-community-libs-5.7.18-1.el7.x86_64.rpmmysql-community-server-minimal-5.7.18-1.el7.x86_64.rpmmysql-community-test-5.7.18-1.el7.x86_64.rpmmysql-community-minimal-debuginfo-5.7.18-1.el7.x86_64.rpmmysql-community-libs-compat-5.7.18-1.el7.x86_64.rpm按次序安裝按common–>libs–>client–>server的順序安裝：#rpm-ivhmysql-community-common-5.7.18-1.el7.x86_64.rpm#rpm-ivhmysql-community-libs-5.7.18-1.el7.x86_64.rpm#rpm-ivhmysql-community-client-5.7.18-1.el7.x86_64.rpm#rpm-ivhmysql-community-server-5.7.18-1.el7.x86_64.rpm#rpm-ivhmysql-community-devel-5.7.18-1.el7.x86_64.rpm//安裝下面包，否那么Hue安裝報(bào)連接錯誤#rpm-ivhmysql-community-libs-compat-5.7.18-1.el7.x86_64.rpm啟動數(shù)據(jù)庫#systemctlstartmysqld查看狀態(tài)：#systemctlstatusmysqld修改MySQL初始密碼1、先修改配置文件/etc/f令MySQL跳過登錄時的權(quán)限檢驗(yàn)，在[mysqld]下參加一行：skip-grant-tables2、重啟MySQL#servicemysqldrestart3、免密碼登錄MySQL。#mysql4、mysql客戶端執(zhí)行如下命令，修改root密碼mysql>usemysql;mysql>UPDATEuserSETauthentication_string=password('your-password')WHEREhost='localhost'ANDuser='root';mysql>selecthost,user,authentication_string,password_expiredfromuser;mysql>updateusersetpassword_expired='N'wherepassword_expired='Y'//密碼不過期mysql>updateusersethost='%'whereuser='root'andhost='localhost';//遠(yuǎn)程可訪問mysql>flushprivileges;//刷新mysql>exit;//退出5、修改配置文件/etc/f刪除此前新增那一行skip-grant-tables，并重啟MySQL(這一步非常重要，不執(zhí)行可能導(dǎo)致嚴(yán)重的安全問題)#servicemysqldrestart//重啟Mysql安裝CM傳包，解包在主節(jié)點(diǎn)上下載相關(guān)軟件包，這里將軟件包下載到/data/cdh/目錄下。#cd/data/cdh下載MysqlJDBC包：#wget:///maven2/mysql/mysql-connector-java/5.1.43/mysql-connector-java-5.1.43.jar下載CM：#wget://archive-primary.cloudera/cm5/cm/5/cloudera-manager-centos7-cm5.13.1_x86_64.tar.gz下載CDH：#wget://archive-primary.cloudera/cdh5/parcels/5.13.1/manifest.json#wget://archive-primary.cloudera/cdh5/parcels/5.13.1/CDH-5.13.1-1.cdh5.13.1.p0.2-el7.parcel.sha1#wget://archive-primary.cloudera/cdh5/parcels/5.13.1/CDH-5.13.1-1.cdh5.13.1.p0.2-el7.parcel將CM解壓到/opt/目錄：#tar-zxvfcloudera-manager-centos7-cm5.13.1_x86_64.tar.gz-C/opt/#ls/opt/clouderacm-5.13.1創(chuàng)立數(shù)據(jù)庫在主節(jié)點(diǎn)上：#mysql-h-uroot-p//加參數(shù)-h指定本機(jī)方式，否那么可能不允許執(zhí)行g(shù)rantEnterpassword:\\輸入數(shù)據(jù)庫密碼//在MariaDB[(none)]>命令狀態(tài)輸入下面腳本：createdatabasehiveDEFAULTCHARSETutf8COLLATEutf8_general_ci;createdatabaseamonDEFAULTCHARSETutf8COLLATEutf8_general_ci;createdatabasehueDEFAULTCHARSETutf8COLLATEutf8_general_ci;createdatabasemonitorDEFAULTCHARSETutf8COLLATEutf8_general_ci;createdatabaseoozieDEFAULTCHARSETutf8COLLATEutf8_general_ci;grantallprivilegeson*.*toroot@localhostidentifiedby'root'withgrantoption;grantallon*.*toroot@"%"Identifiedby"root";flushprivileges;exit;//復(fù)制MysqlJDBC包到/opt/cm-5.13.1/share/cmf/lib/目錄#cpmysql-connector-java-5.1.43-bin.jar/opt/cm-5.13.1/share/cmf/lib///初始化CM，執(zhí)行下面腳本#/opt/cm-5.13.1/share/cmf/schema/scm_prepare_database.shmysqlcm-hnode168-uroot-proot--scm-hostnode168scmscmscm//如果出現(xiàn)下面錯誤：java.sql.SQLException:Accessdeniedforuser'root'@'localhost'(usingpassword:YES)//解決方法：在/etc/hosts文件中，加上localhost.node168主節(jié)點(diǎn)/etc/hosts內(nèi)容：腳本/scm_prepare_database.sh執(zhí)行成功界面：創(chuàng)立用戶在所有節(jié)點(diǎn)上執(zhí)行：#useradd--system--home=/opt/cm-5.13.1/run/cloudera-scm-server--shell=/bin/false--comment"ClouderaSCMUser"cloudera-scm制作本地YUM源在主節(jié)點(diǎn)上：//進(jìn)入軟件包目錄#cd/data/cdh//拷貝三個文件到/opt/cloudera/parcel-repo/目錄#cpCDH-5.13.1-1.cdh5.13.1.p0.2-el7.parcelCDH-5.13.1-1.cdh5.13.1.p0.2-el7.parcel.sha1manifest.json/opt/cloudera/parcel-repo///進(jìn)入/opt/cloudera/parcel-repo/目錄#cd/opt/cloudera/parcel-repo///修改文件名#mvCDH-5.13.1-1.cdh5.13.1.p0.2-el7.parcel.sha1CDH-5.13.1-1.cdh5.13.1.p0.2-el7.parcel.sha此時/opt/cloudera/parcel-repo/目錄下文件：拷貝jar包在所有節(jié)點(diǎn)上：//如果沒有/usr/share/java/目錄，那么創(chuàng)立#cpmysql-connector-java-5.1.43-bin.jar/usr/share/java/mysql-connector-java.jar在主節(jié)點(diǎn)上：//將mysqlJDBCjar包拷貝到/opt/cm-5.13.1/share/cmf/lib/目錄#cpmysql-connector-java-5.1.43-bin.jar/opt/cm-5.13.1/share/cmf/lib/修改cloudera-scm-agent配置在主節(jié)點(diǎn)上，修改/opt/cm-5.13.1/etc/cloudera-scm-agent/config.ini文件：//將config.iniserver_host=localhost內(nèi)容改為server_host=68#sed-i"s/server_host=localhost/server_host=68/"/opt/cm-5.13.1/etc/cloudera-scm-agent/config.ini在主節(jié)點(diǎn)上：//將cm-5.13.1打包，并復(fù)制到其他節(jié)點(diǎn)#cd/opt#tarczfcm-5.13.1.tar.gzcm-5.13.1///復(fù)制到其他節(jié)點(diǎn)#scpcm-5.13.1.tar.gznode169:/opt/#scpcm-5.13.1.tar.gznode170:/opt/#scpcm-5.13.1.tar.gznode171:/opt/#scpcm-5.13.1.tar.gznode172:/opt/在所有從節(jié)點(diǎn)上解壓：//將cm-5.13.1包解壓#cdopt#tar-xzvfcm-5.13.1.tar.gz//解壓后刪除#rm-rfcm-5.13.1.tar.gz啟動CMServer和Agent在主節(jié)點(diǎn)上，啟動cloudera-scm-server：#/opt/cm-5.13.1/etc/init.d/cloudera-scm-serverstart啟動過程較慢，可通過/opt/cm-5.13.1/log/cloudera-scm-server日志，查看啟動過程。在所有節(jié)點(diǎn)上，啟動cloudera-scm-agent：#/opt/cm-5.13.1/etc/init.d/cloudera-scm-agentstart訪問CM地址：://主節(jié)點(diǎn)IP:7180用戶名、密碼：admin安裝CDH登錄后界面承受協(xié)議：選擇CM版本本文檔選擇ClouderaExpress免費(fèi)版。指定主機(jī)在搜索主機(jī)名和IP地址框輸入node[169-172]，這里輸入的內(nèi)容支持正那么表達(dá)式。輸入后點(diǎn)【搜索】按鈕，出現(xiàn)機(jī)器列表：