ngxtop, a log monitoring tool for nginx

Looking at nginx log files in tail? out!

Tail-f log path

ngxtop:

ngxtop is a python-based program that can be installed on Python. ngxtop parses nginx access logs in real time and outputs the results (useful indicators of nginx servers) to terminals

Main functions:

• Current valid request

• Summary of total request count

• Total number of requests provided by status code (2xx, 3xx, 4xx, 5xx)

• Send average bytes

• Top-level remote address

Not only can the access of Nginx log be monitored in real time, but also the previous log can be sorted out.

Installation:

Examples are: centos7 system with Python and pip installed

wget http://dl.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-6.noarch.rpm
rpm -ivh epel-release-7-6.noarch.rpm

Then execute the following command to install pip
yum install python-pip

Finally, install ngxtop using the following
pip install ngxtop

ngxtop usage:

ngxtop [option]
ngxtop [option](print | top | avg | sum)<var>
ngxtop info

Options and specifications:

# ngxtop --help

ngxtop - ad-hoc query for nginx access log.

Usage:

ngxtop [options]

ngxtop [options] (print|top|avg|sum) <var> ...

ngxtop info

ngxtop [options] query <query> ...

Options:

-l <file>, --access-log <file>  Access logs that need to be analyzed

-f <format>, --log-format <format>  log_format Instruction specified log format [default: combined]

--no-follow  ngxtop default behavior is to ignore current lines in log

and only watch for new lines as they are written to the access log.

Use this flag to tell ngxtop to process the current content of the access log instead.

-t <seconds>, --interval <seconds>  report interval when running in follow mode [default: 2.0]

-g <var>, --group-by <var>  Grouping by variables [default: request_path]

-w <var>, --having <expr>  having clause [default: 1]

-o <var>, --order-by <var>  sort [default: count]

-n <number>, --limit <number>  Number of bars displayed [default: 10]

-a <exp> ..., --a <exp> ...  add exp (must be aggregation exp: sum, avg, min, max, etc.) into output

-v, --verbose  More output

-d, --debug  print every line and parsed record

-h, --help  Current Help Information.

--version  Output version information.

//Advanced Options:

-c <file>, --config <file>  Function ngxtop analysis nginx configuration file

-i <filter-expression>, --filter <filter-expression>  filter in, records satisfied given expression are processed.

-p <filter-expression>, --pre-filter <filter-expression> in-filter expression to check in pre-parsing phase.

ngxtop example:

Example:

All examples read nginx config file for access log location and format.

If you want to specify the access log file and / or log format, use the -f and -a options.

"top" like view of nginx requests

$ ngxtop

404 Top Ten Requests

$ ngxtop top request_path --filter 'status == 404'

//Top 10 requests for total traffic

$ ngxtop --order-by 'avg(bytes_sent) * count'

//ip addresses in the top ten accesses

$ ngxtop --group-by remote_addr

//Output of more than 400 status requests and source of requests

$ ngxtop -i 'status >= 400' print request status http_referer

Average body bytes sent of 200 responses of requested path begin with 'foo':

$ ngxtop avg bytes_sent --filter 'status == 200 and request_path.startswith("foo")'

//Analysis of remote server Apache access log using common log format
$ ssh remote tail -f /var/log/apache2/access.log | ngxtop -f common

Other examples:

Real-time status viewing:

ngxtop -c /opt/nginx/conf/nginx.conf

Access previous IP:

ngxtop -c /opt/nginx/conf/nginx.conf top remote_addr

Requests showing status code 404:

ngxtop -i 'status == 404' print request status

Display the most frequent requests in the top 20:

ngxtop -n 20

Resolve apache logs from remote servers in normal format:

ssh user@remote_server tail -f /var/log/apache2/access.log | ngxtop -f common