如何在代码中执行spark

Question

如何在代码中执行spark

Answer 1

写Linux脚本
#!/bin/sh
if [ $# -lt 2 ] ;then
echo "Useage:$0 <hdfs_input_file> $1 <hdfs_output_file>"
exit
else
echo "wordcount is starting!"
spark-submit \
--master spark://spark:7077 \
--class WordCount \
--executor-memory 512m \
--total-executor-cores 6 \
~/input/WordCount.jar \
"$1" \
"$2"
echo "wordcount is stopped!"
fi