导入压缩包，解压改名

三台都配置spark环境变量

 #SPARK_HOME export SPARK_HOME=/opt/softs/spark3.1.2 export PATH=$PATH:$SPARK_HOME/bin export PATH=$PATH:$SPARK_HOME/sbin

修改配置文件

#!/usr/bin/env bash  export JAVA_HOME=/opt/softs/jdk1.8.0 export HADOOP_HOME=/opt/softs/hadoop3.1.3 export HADOOP_CONF_DIR=/opt/softs/hadoop3.1.3/etc/hadoop export SPARK_DIST_CLASSPATH=$(/opt/softs/hadoop3.1.3/bin/hadoop classpath) export SPARK_MASTER_HOST=192.168.111.54 export SPARK_MASTER_PORT=7077 export SPARK_HISTORY_OPTS="-Dspark.history.ui.port=18080 -Dspark.history.retainedApplications=50 -Dspark.history.fs.logDirectory=hdfs://192.168.111.54:8020/spark-eventlog"

spark.master spark://192.168.111.54:7077 spark.eventLog.enabled true spark.eventLog.dir hdfs://192.168.111.54:8020/spark-eventlog spark.serializer org.apache.spark.serializer.KryoSerializer spark.driver.memory 2g spark.eventLog.compress true

配置worker里的hostname

将修改传给其他虚拟机

 scp -r spark3.1.2/ root@bigdata005:/opt/softs/

启动hdfs

启动spark

启动命令在/opt/softs/spark3.1.2/sbin 目录下的

sh start-all.sh

启动成功

 关闭

webui

http://192.168.111.54:8080

​​​​​​​