ttps://open.work.weixin.qq.com/wwopen/mpnews?mixuin=YzSbDgAABwAK1s0mAAAUAA&mfid=WW0323-hRA3mAAABwD1yRtefXtVOA1eiR-85&idx=0&sn=d3c3545c205515328a96b99c8e15f65a&version=4.1.22.8031&platform=win
安裝并配置hadoop-2.6.4(在master中)
(1)將hadoop-2.6.4.tar.gz安裝包復制到hadoop文件目錄下(與windows環(huán)境下類似)。
(2)解壓hadoop-2.6.4.tar.gz
cd
cd hadoop
tar -zxvf hadoop-2.6.4.tar.gz
添加環(huán)境變量: vim /etc/profile
在最后添加
export HADOOP_HOME=/usr/hadoop-2.6.2
export PATH=HADOOP_HOME/bin:$HADOOP_HOME/sbin
生效:source /etc/profile
配置Hadoop (cd /usr/hadoop-2.6.2)
1、vi core-site.xml
hadoop.tmp.dir
/usr/hadoop-2.6.2/tmp
Abase for other temporary directories.
fs.defaultFS
hdfs://master:9000
io.file.buffer.size
4096
2、vi hadoop-env.sh 在開頭加上 export JAVA_HOME=/usr/jdk1.7.0_79
3、vi yarn-env.sh 在開頭加上 export JAVA_HOME=/usr/jdk1.7.0_79
4、vi hdfs-site.xml
dfs.namenode.name.dir
file:///usr/hadoop-2.6.2/dfs/name
dfs.datanode.data.dir
file:///usr/hadoop-2.6.2/dfs/data
dfs.replication
2
dfs.nameservices
hadoop-cluster1
dfs.namenode.secondary.http-address
master:50090
dfs.webhdfs.enabled
true
5、vi mapred-site.xml
mapreduce.framework.name
yarn
true
mapreduce.jobtracker.http.address
master:50030
mapreduce.jobhistory.address
master:10020
mapreduce.jobhistory.webapp.address
master:19888
mapred.job.tracker
http://master.Hadoop:9001
6、vi yarn-site.xml
yarn.resourcemanager.hostname
master
yarn.nodemanager.aux-services
mapreduce_shuffle
yarn.resourcemanager.address
master:8032
yarn.resourcemanager.scheduler.address
master:8030
yarn.resourcemanager.resource-tracker.address
master:8031
yarn.resourcemanager.admin.address
master:8033
yarn.resourcemanager.webapp.address
master:8088
7、vi slaves
slave1
slave2
8、復制到slave上 scp -r /usr/hadoop-2.6.2/ root@slave1:/usr/
9、配置slave上的Hadoop環(huán)境變量
添加環(huán)境變量: vim /etc/profile
在最后添加
export HADOOP_HOME=/usr/local/hadoop-2.8.4
export PATH=HADOOP_HOME/bin:$HADOOP_HOME/sbin
生效:source /etc/profile
10、啟動
先啟動hadoop集群start-all.sh