如何停止在火花控制台上显示INFO消息？

Question

如何停止在火花控制台上显示INFO消息？我想停止火花壳上的各种消息。我试图编辑该log4j.properties文件以阻止这些消息。这是内容 log4j.properties# Define the root logger with appender filelog4j.rootCategory=WARN, consolelog4j.appender.console=org.apache.log4j.ConsoleAppenderlog4j.appender.console.target=System.errlog4j.appender.console.layout=org.apache.log4j.PatternLayoutlog4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{1}: %m%n# Settings to quiet third party logs that are too verboselog4j.logger.org.eclipse.jetty=WARNlog4j.logger.org.eclipse.jetty.util.component.AbstractLifeCycle=ERRORlog4j.logger.org.apache.spark.repl.SparkIMain$exprTyper=INFOlog4j.logger.org.apache.spark.repl.SparkILoop$SparkILoopInterpreter=INFO但是消息仍在控制台上显示。以下是一些示例消息15/01/05 15:11:45 INFO SparkEnv: Registering BlockManagerMaster15/01/05 15:11:45 INFO DiskBlockManager: Created local directory at /tmp/spark-local-20150105151145-b1ba15/01/05 15:11:45 INFO MemoryStore: MemoryStore started with capacity 0.0 B.15/01/05 15:11:45 INFO ConnectionManager: Bound socket to port 44728 with id = ConnectionManagerId(192.168.100.85,44728)15/01/05 15:11:45 INFO BlockManagerMaster: Trying to register BlockManager15/01/05 15:11:45 INFO BlockManagerMasterActor$BlockManagerInfo: Registering block manager 192.168.100.85:44728 with 0.0 B RAM15/01/05 15:11:45 INFO BlockManagerMaster: Registered BlockManager15/01/05 15:11:45 INFO HttpServer: Starting HTTP Server15/01/05 15:11:45 INFO HttpBroadcast: Broadcast server star我该如何阻止这些？

Answer 1

刚开始spark-shell打字后;

sc.setLogLevel("ERROR")

在Spark 2.0中：

spark = SparkSession.builder.getOrCreate()spark.sparkContext.setLogLevel("ERROR")

Answer 2

建议修改.conf文件。

以下代码解决了我的问题：

1）import org.apache.log4j.{Level, Logger}在导入部分添加

2）在创建spark上下文对象后添加以下行，即val sc = new SparkContext(conf)：

val rootLogger = Logger.getRootLogger()rootLogger.setLevel(Level.ERROR)