获取仅包含 JSON 的日志

Question

我有一个包含日志 log4j 的文件，我无法为 logstash 编写过滤器，它将返回仅包含 JSON 的日志input {   file {      path => "C:/logs/inlog.log"   }} filter {  #A lot of what I tried to write}output {   file {      path => "C:/logs/outlog.log"   }}因此，我们需要一个过滤器来显示包含 json 的日志。Json 日志被标记为 trace示例日志：[2019/08/02 11:21:52.472 MSK] [ 4] INFO  ru.dmko.logstash.util.Utils - Initialized timer 'MyProject' with interval - 60000 [2019/08/02 11:21:53.769 MSK] [ 4] INFO  ru.dmko.logstash.EventHandlerBean - EventHandler started [2019/08/02 11:21:56.535 MSK] [21] INFO  ru.dmko.logstash.processors.MessageProcessorBean - {"glossary": {"title": "example glossary", "GlossDiv": {"title": "S", "GlossList": {"GlossEntry": {"ID": "SGML", "SortAs": "SGML", "GlossTerm": "Standard Generalized Markup Language", Acronym": "SGML", "Abbrev": "ISO 8879:1986", "GlossDef": { "para": "A meta-markup language, used to create markup languages such as DocBook.", "GlossSeeAlso": ["GML", "XML"]}, "GlossSee": "markup"}}}} [2019/08/02 11:21:56.551 MSK] [21] INFO  ru.dmko.logstash.processors.MessageProcessorBean - Equal messages [2019/08/02 11:21:56.613 MSK] [21] INFO  ru.dmko.logstash.processors.MessageProcessorBean - {"glossary": {"title": "example glossary", "GlossDiv": {"title": "S", "GlossList": {"GlossEntry": {"ID": "SGML", "SortAs": "SGML", "GlossTerm": "Standard Generalized Markup Language", Acronym": "SGML", "Abbrev": "ISO 8879:1986", "GlossDef": { "para": "A meta-markup language, used to create markup languages such as DocBook.", "GlossSeeAlso": ["GML", "XML"]}, "GlossSee": "markup"}}}} 其中，只需要看到两个包含json

Answer 1

我会用它来挑选 JSON

grok { match => { "message" => "%{JAVACLASS} - (?={)%{GREEDYDATA:json}" } }

如果您愿意，可以删除 [tags] 中具有“_grokparsefailure”的 {} 消息

如果您将 JSON 修复为有效（将开头“添加到首字母缩略词并添加尾随}），您可以使用解析它

json { source => json }

另一种（更便宜的）可能性，取决于“ - ”是否曾经出现在事件的其他上下文中

    dissect { mapping => { "message" => "%{} - %{json}" } }

    if [json] =~ /^{/ {

        json { source => json }

    } else {

        drop {}

    }