1

Logstash를 사용하여 JSON 메시지를 API로 출력하고 있습니다. 로그 파일에서 로그를 읽습니다. 구성이 정상적으로 작동하고 있으며 모든 메시지를 API로 보내고 있습니다. 내 구성 파일logstash 발송인을 통해 오류 로그를 보내는 방법

TID: [-1234] [] [2016-06-07 12:52:59,862] INFO {org.apache.synapse.core.axis2.ProxyService} - Successfully created the Axis2 service for Proxy service : TestServiceHttp {org.apache.synapse.core.axis2.ProxyService} 
    TID: [-1234] [] [2016-06-07 12:59:04,893] INFO {org.apache.synapse.mediators.builtin.LogMediator} - To: /services/TestServiceHttp.TestServiceHttpHttpSoap12Endpoint********* Sending Message to the Queue*****WSAction: urn:mediate********* Sending Message to the Queue*****SOAPAction: urn:mediate********* Sending Message to the Queue*****MessageID: urn:uuid:d1bbe24a-2ce3-497f-8224-d260b0632506********* Sending Message to the Queue*****Direction: request********* Sending Message to the Queue*****Envelope: <?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope"><soapenv:Body><name> Omer</name></soapenv:Body></soapenv:Envelope> {org.apache.synapse.mediators.builtin.LogMediator} 
    TID: [-1234] [] [2016-06-07 12:59:04,925] INFO {org.apache.synapse.core.axis2.TimeoutHandler} - This engine will expire all callbacks after : 120 seconds, irrespective of the timeout action, after the specified or optional timeout {org.apache.synapse.core.axis2.TimeoutHandler} 
    TID: [-1234] [] [2016-06-07 12:59:04,933] ERROR {org.apache.axis2.description.ClientUtils} - The system cannot infer the transport information from the jms:/Customer.01.Request.Queue.01?transport.jms.ConnectionFactoryJNDIName=QueueConnectionFactory&java.naming.factory.initial=org.apache.activemq.jndi.ActiveMQInitialContextFactory&java.naming.provider.url=tcp://localhost:61616&transport.jms.DestinationType=queue URL. {org.apache.axis2.description.ClientUtils} 
    TID: [-1234] [] [2016-06-07 12:59:04,949] ERROR {org.apache.synapse.core.axis2.Axis2Sender} - Unexpected error during sending message out {org.apache.synapse.core.axis2.Axis2Sender} 
    org.apache.axis2.AxisFault: The system cannot infer the transport information from the jms:/Customer.01.Request.Queue.01?transport.jms.ConnectionFactoryJNDIName=QueueConnectionFactory&java.naming.factory.initial=org.apache.activemq.jndi.ActiveMQInitialContextFactory&java.naming.provider.url=tcp://localhost:61616&transport.jms.DestinationType=queue URL. 
     at org.apache.axis2.description.ClientUtils.inferOutTransport(ClientUtils.java:81) 
     at org.apache.axis2.client.OperationClient.prepareMessageContext(OperationClient.java:288) 
     at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172) 
     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
     at java.lang.Thread.run(Thread.java:745) 
    TID: [-1234] [] [2016-06-07 12:59:05,009] INFO {org.apache.synapse.mediators.builtin.LogMediator} - To: /services/TestServiceHttp.TestServiceHttpHttpSoap12Endpoint, WSAction: urn:mediate, SOAPAction: urn:mediate, MessageID: urn:uuid:d1bbe24a-2ce3-497f-8224-d260b0632506, Direction: request, MESSAGE = Executing default 'fault' sequence, ERROR_CODE = 0, ERROR_MESSAGE = Unexpected error during sending message out, Envelope: <?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope"><soapenv:Body><name> Omer</name></soapenv:Body></soapenv:Envelope> {org.apache.synapse.mediators.builtin.LogMediator} 
    TID: [-1234] [] [2016-06-07 13:00:04,890] INFO {org.apache.axis2.transport.http.HTTPSender} - Unable to sendViaPost to url[http://Omer-PC:8280/services/TestServiceHttp.TestServiceHttpHttpSoap12Endpoint] {org.apache.axis2.transport.http.HTTPSender} 
    java.net.SocketTimeoutException: Read timed out 
     at java.net.SocketInputStream.socketRead0(Native Method) 
     at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) 
     at java.net.SocketInputStream.read(SocketInputStream.java:170) 
     at java.net.SocketInputStream.read(SocketInputStream.java:141) 
     at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) 
     at java.io.BufferedInputStream.read(BufferedInputStream.java:265) 
     at org.apache.commons.httpclient.HttpParser.readRawLine(HttpParser.java:78) 
     at org.apache.commons.httpclient.HttpParser.readLine(HttpParser.java:106) 
ent.ServiceClient.sendReceive(ServiceClient.java:530) 
     at org.apache.jsp.admin.jsp.WSRequestXSSproxy_005fajaxprocessor_jsp._jspService(WSRequestXSSproxy_005fajaxprocessor_jsp.java:294) 
     at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70) 
     at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) 
     at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:432) 
     at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:395) 
     at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:339) 
     at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) 
     at org.wso2.carbon.ui.JspServlet.service(JspServlet.java:155) 
     at org.wso2.carbon.ui.TilesJspServlet.service(TilesJspServlet.java:80) 
     at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) 
     at org.eclipse.equinox.http.helper.ContextPathServletAdaptor.service(ContextPathServletAdaptor.java:37) 
     at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61) 
     at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128) 
     at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68) 
     at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) 
     at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68) 
     at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) 
     at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) 
     at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:57) 
     at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) 
     at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421) 
     at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1074) 
     at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) 
     at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1739) 
     at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1698) 
     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
     at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) 
     at java.lang.Thread.run(Thread.java:745) 
    TID: [-1234] [] [2016-06-07 13:01:40,447] INFO {org.wso2.carbon.core.init.CarbonServerManager} - Shutdown hook triggered.... {org.wso2.carbon.core.init.CarbonServerManager} 
    TID: [-1234] [] [2016-06-07 13:01:40,464] INFO {org.wso2.carbon.core.init.CarbonServerManager} - Gracefully shutting down WSO2 Enterprise Service Bus... {org.wso2.carbon.core.init.CarbonServerManager} 
    TID: [-1234] [] [2016-06-07 13:01:40,477] INFO {org.wso2.carbon.core.ServerManagement} - Starting to switch to maintenance mode... {org.wso2.carbon.core.ServerManagement} 
    TID: [-1234] [] [2016-06-07 13:01:40,481] INFO {org.apache.axis2.transport.jms.JMSListener} - JMS Listener Shutdown {org.apache.axis2.transport.jms.JMSListener} 

다음입니다 :

구성 파일 :

input { 
stdin {} 

    file { 
     path => "C:\WSO2Environment\wso2esb-4.9.0\repository\logs\wso2carbon.log" 
     type => "wso2" 
     start_position => "beginning" 
     codec => multiline { 
      pattern => "(^\s*at .+)|^(?!TID).*$" 
      negate => false 
      what => "previous" 
    } 

    } 
} 

filter { 

    if [type] == "wso2" { 
     grok { 
      match => [ "message", "TID:%{SPACE}\[%{INT:SourceSystemId}\]%{SPACE}\[%{DATA:ProcessName}\]%{SPACE}\[%{TIMESTAMP_ISO8601:TimeStamp}\]%{SPACE}%{LOGLEVEL:MessageType}%{SPACE}{%{JAVACLASS:MessageTitle}}%{SPACE}-%{SPACE}%{GREEDYDATA:Message}" ] 
      add_tag => [ "grokked" ]   
     } 
     mutate { 
      gsub => [ 
      "TimeStamp", "\s", "T", 
      "TimeStamp", ",", "." 
      ] 
     } 


    } 
    if !("_grokparsefailure" in [tags]) { 
     grok{ 
       match => [ "message", "%{GREEDYDATA:StackTrace}" ] 
       add_tag => [ "grokked" ] 
     } 
     date { 
       match => [ "timestamp", "yyyy MMM dd HH:mm:ss:SSS" ] 
       target => "TimeStamp" 
       timezone => "UTC" 
      } 
    } 

    if ("multiline" in [tags]) { 
     grok { 
      match => [ "message", "%{GREEDYDATA:StackTrace}" ] 
      add_tag => [ "multiline" ] 
      tag_on_failure => [ "multiline" ]  
     } 
     date { 
       match => [ "timestamp", "yyyy MMM dd HH:mm:ss:SSS" ] 
       target => "TimeStamp" 

      } 
    } 

} 

output { 
    stdout { } 
    http { 
     url => "http://localhost:8086/messages" 
     http_method => "post" 
     format => "json" 
     mapping => ["TimeStamp","%{TimeStamp}","MessageType","%{MessageType}","MessageTitle","%{MessageTitle}","Message","%{log_EventMessage}","SourceSystemId","%{SourceSystemId}","StackTrace","%{log_StackTrace}"] 

    } 
} 

문제 설명

로그 파일 : 다음은 샘플 로그 파일입니다 :

구성 파일이 올바르게 작동하고 모든 로그 항목을 API로 보내지 만 오류 로그 만 API로 보내려고합니다. 그래서 "MessageType"에 대한 확인을하고 싶습니다. 그 값이 "ERROR"이면이고 메시지가 API로 보내 져야합니다. 그렇지 않으면 logstash가 메시지를 버려야합니다.

답변

1

필터 섹션의 logstash 구성에서 if 조건에 따라 add tag를 사용할 수 있습니다. 그리고 태그 오류가 존재 하는지를 검사하는 출력 add if 문에서 그렇지 않으면 보낼 것입니다. 그렇지 않으면 무시됩니다. 다음 if 문 후

:

if [type] == "wso2" { 
    grok { 
     match => [ "message", "TID:%{SPACE}\[%{INT:SourceSystemId}\]%{SPACE}\[%{DATA:ProcessName}\]%{SPACE}\[%{TIMESTAMP_ISO8601:TimeStamp}\]%{SPACE}%{LOGLEVEL:MessageType}%{SPACE}{%{JAVACLASS:MessageTitle}}%{SPACE}-%{SPACE}%{GREEDYDATA:Message}" ] 
     add_tag => [ "grokked" ]   
    } 
    mutate { 
     gsub => [ 
     "TimeStamp", "\s", "T", 
     "TimeStamp", ",", "." 
     ] 
    } 


} 

이 필터에 다음 문을 추가

하여 출력 메이크업 다음과 같이 변경에 그런
if "grokked" in [tags] { 
     grok { 
      match => ["MessageType", "ERROR"] 
      add_tag => [ "loglevelerror" ] 
     } 
    } 

는 :

output { 

    if "loglevelerror" in [tags] { 
     stdout { } 
     http { 
     url => "http://localhost:8086/messages" 
     http_method => "post" 
     format => "json" 
     mapping => ["TimeStamp","%{TimeStamp}","MessageType","%{MessageType}","MessageTitle","%{MessageTitle}","Message","%{log_EventMessage}","SourceSystemId","%{SourceSystemId}","StackTrace","%{log_StackTrace}"] 

     } 
    } 
} 

나는 그것을 테스트 내 컴퓨터에서 stdout을 사용하여. 그것은 잘 작동합니다. 희망이 도움이됩니다!

+0

감사합니다. –

관련 문제