说明:这里我们以本地模式和伪分布模式伪列,为大家介绍如何搭建hadoop环境。有了这个基础,大家可以自行搭建hadoop的全分布模式。
需要使用的安装介质:
systemctl stop firewalld.service
systemctl disable firewalld.service
bigdata111 192.168.157.111
ssh-keygen -t rsa
ssh-copy-id -i .ssh/id_rsa.pub root@bigdata111
tar -zxvf jdk-8u181-linux-x64.tar.gz -c /root/training/
vi /root/.bash_profile
java_home=/root/training/jdk1.8.0_181
export java_home
path=$java_home/bin:$path
export path
source /root/.bash_profile
tar -zxvf hadoop-2.7.3.tar.gz -c ~/training/
hadoop_home=/root/training/hadoop-2.7.3
export hadoop_home
path=$hadoop_home/bin:$hadoop_home/sbin:$path
export path
source ~/.bash_profile
export java_home=/root/training/jdk1.8.0_181
hadoop jar hadoop-mapreduce-examples-2.7.3.jar wordcount /root/temp /root/output/wc
mkdir /root/training/hadoop-2.7.3/tmp
cd /root/training/hadoop-2.7.3/etc/hadoop
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<!--配置namenode的地址-->
<!--9000是rpc通信的端口-->
<property>
<name>fs.defaultfs</name>
<value>hdfs://bigdata111:9000</value>
</property>
<!--hdfs对应的操作系统目录-->
<!--默认值是linux的tmp目录-->
<property>
<name>hadoop.tmp.dir</name>
<value>/root/training/hadoop-2.7.3/tmp</value>
</property>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<!--配置resourcemanager的地址-->
<property>
<name>yarn.resourcemanager.hostname</name>
<value>bigdata111</value>
</property>
<!--mapreduce运行的方式是洗牌-->
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
hdfs namenode -format
start-all.sh
http://192.168.157.111:50070
http://192.168.157.111:8088
如对本文有疑问, 点击进行留言回复!!
去 HBase,Kylin on Parquet 性能表现如何?
如何找到Hive提交的SQL相对应的Yarn程序的applicationId
如何在 HBase Shell 命令行正常查看十六进制编码的中文?哈哈~
网友评论