Skip to content

Regexp doesn't work #190

@jazzl0ver

Description

@jazzl0ver

Hi and thanks for the awesome app!

I'm trying to use my own haproxy log format. The log entry has some captured request headers which are enclosed with braces:

... {||573|Delivery%20Map/11 CFNetwork/758.5.3 Darwin/15.6.0} ...

To catch that part I use the following regexp (check captured_request_headers line):

    def self.compose_regexp(millisecs, backends, counters, connections, queues)
      %r{
        (#{ip_address}):\d+\s # client_ip ':' client_port
        \[(#{timestamp('%d/%b/%Y:%H:%M:%S')})#{millisecs}\]\s # '[' accept_date ']'
        (\S+)\s # frontend_name
        #{backends}
        #{counters}
        (\w|-)(\w|-)(\w|-)(\w|-)\s # termination_state
        #{connections}
        #{queues}
        "([^"]*)"\s # '"' http_request '"'
        (\d+)\s # status_code
        (\S+)\s # captured_request_cookie
        ({[\w\W]*})\s # captured_request_headers
        (\S+)\s # captured_response_cookie
        (\S*)\s? # captured_response_headers
        ireq_size\=(\d+)\s # bytes_sent
        resp_size\=(\d+)\s # bytes_read
      }x
    end

Online regexp checker successfully works for that part: https://regex101.com/r/fM4mW5/1

But your app throws an exception:

$ /opt/rh/ruby193/root/usr/local/bin/request-log-analyzer test -f myhaproxy --reject query "/hastats;csv/" --after 2016-09-02 --before 2016-09-03 --debug
Request-log-analyzer, by Willem van Bergen and Bart ten Brinke - version 1.13.4
Website: http://railsdoctors.com

/opt/rh/ruby193/root/usr/local/share/gems/gems/request-log-analyzer-1.13.4/lib/request_log_analyzer/aggregator/echo.rb:12:in `+': can't convert Enumerator into String (TypeError)
        from /opt/rh/ruby193/root/usr/local/share/gems/gems/request-log-analyzer-1.13.4/lib/request_log_analyzer/aggregator/echo.rb:12:in `aggregate'
        from /opt/rh/ruby193/root/usr/local/share/gems/gems/request-log-analyzer-1.13.4/lib/request_log_analyzer/controller.rb:316:in `block in aggregate_request'
        from /opt/rh/ruby193/root/usr/local/share/gems/gems/request-log-analyzer-1.13.4/lib/request_log_analyzer/controller.rb:316:in `each'
        from /opt/rh/ruby193/root/usr/local/share/gems/gems/request-log-analyzer-1.13.4/lib/request_log_analyzer/controller.rb:316:in `aggregate_request'
        from /opt/rh/ruby193/root/usr/local/share/gems/gems/request-log-analyzer-1.13.4/lib/request_log_analyzer/controller.rb:336:in `block in run!'
        from /opt/rh/ruby193/root/usr/local/share/gems/gems/request-log-analyzer-1.13.4/lib/request_log_analyzer/source/log_parser.rb:332:in `handle_request'
        from /opt/rh/ruby193/root/usr/local/share/gems/gems/request-log-analyzer-1.13.4/lib/request_log_analyzer/source/log_parser.rb:304:in `update_current_request'
        from /opt/rh/ruby193/root/usr/local/share/gems/gems/request-log-analyzer-1.13.4/lib/request_log_analyzer/source/log_parser.rb:225:in `parse_line'
        from /opt/rh/ruby193/root/usr/local/share/gems/gems/request-log-analyzer-1.13.4/lib/request_log_analyzer/source/log_parser.rb:182:in `parse_io_19'
        from /opt/rh/ruby193/root/usr/local/share/gems/gems/request-log-analyzer-1.13.4/lib/request_log_analyzer/source/log_parser.rb:130:in `block in parse_file'
        from /opt/rh/ruby193/root/usr/local/share/gems/gems/request-log-analyzer-1.13.4/lib/request_log_analyzer/source/log_parser.rb:130:in `open'
        from /opt/rh/ruby193/root/usr/local/share/gems/gems/request-log-analyzer-1.13.4/lib/request_log_analyzer/source/log_parser.rb:130:in `parse_file'
        from /opt/rh/ruby193/root/usr/local/share/gems/gems/request-log-analyzer-1.13.4/lib/request_log_analyzer/source/log_parser.rb:76:in `each_request'
        from /opt/rh/ruby193/root/usr/local/share/gems/gems/request-log-analyzer-1.13.4/lib/request_log_analyzer/controller.rb:334:in `run!'
        from /opt/rh/ruby193/root/usr/local/share/gems/gems/request-log-analyzer-1.13.4/bin/request-log-analyzer:133:in `<top (required)>'
        from /opt/rh/ruby193/root/usr/local/bin/request-log-analyzer:23:in `load'
        from /opt/rh/ruby193/root/usr/local/bin/request-log-analyzer:23:in `<main>'

If I remove that regexp and the appropriate log part, everything works like a charm. There's also no exception if I remove * (the star) from "({[\w\W]*})".

Can you please help me with that?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions