Using php for analyzing apache logs

Apache has a nice feature that is to send the log output through a pipe. This avoid to configure syslog or create a listening server in php for syslog forwarding logs.

The changes in apache are really simple, you just need to write something like:

LogFormat "%v %A %D \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" mylog
CustomLog "|/usr/bin/php5 [PATH_TO_SCRIPT]/apache-stdin.php log" mylog
ErrorLog "|/usr/bin/php5 [PATH_TO_SCRIPT]/apache-stdin.php error"

In apache-stdin.php the flow is very simple, you just need to do:

<?php
$fp = fopen('php://stdin', 'r');
do {
	//read a line from apache, if not, will block until have it
	$data = fgets($fp);
	$data = trim($data); //remove line end

	if (empty($data)) {
		break; //no more data so finish it
	}

	//process the data
} while(true);

fclose($fp);

As you can see it’s basically reading a line and processing.

I’ve built a helper script around this at:
https://github.com/danguer/blog-examples/blob/master/php/syslog/apache-stdin.php

Where you can pass an additional param to specify it’s a normal log, or the error log; like in the apache configuration I’ve posted.

You can also configure the log format you are using in apache to get you a simple description like:

<?php
$format = '%v %A %D \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"';

With this, it will give you an array like:

array(
'hostname' => 'danguer.com',
'local_ip' => 127.0.0.1,
'time_ms' => 0.0002,
'first_line_request' => 'GET / HTTP/1.1',
'status_last' => 200,
'bytes_sent' => 2048
);

From there you can use for storing in a file (it’s the common behavior of this script), insert into a db, use simpledb, etc.

Share