Every once in a while, I’ve found myself looking at a simple, stable, app deployed on a server that hasn’t been touched in months and being asked (or myself wondering) one of these things:

• Why is it getting slower and slower?
• How has this app managed to use all free space on the server?

A common and repeating cause for both problems is: fat log files. There’s a simple solution which is possibly often forgotten and that’s what I’d like to write about today.

How do log files slow your app down?

Appending to a file involves finding the end of the file first. Appending to a big file can mean spending more time trying to find the end of a file than it does to actually write the bytes, which is one cause for an otherwise stable application to gradually grind to a halt.

In my own experience, there are 2 basic causes for log files to get large enough to be a problem:

1. The application is generating a large numbers of error messages
2. The application is nolonger being developed/modified

Both of which hint at the root “crime” being committed: The log files aren’t being read by anybody.

Doesn’t CakePHP take care of that for you?

Since CakePHP version 2.4 the problems caused by (cough) not reading your log files are limited to running out of space. That’s because as of 2.4 the file logger has a basic log rotation feature which will rename log files by appending a timestamp whenever they get too big, and create a new empty log file to write to.

Running out of space doesn’t normally happen overnight, but if nobody is checking for that things will work perfectly right up until there’s no space left at which point things crash hard. If you’re using log files, my recommendation to you is to not rely on the built in log rotation feature.

What can you do? well the logical choice is to not use files for logging at all in production and instead log to Syslog. However the simplicity and convenience of the file log engine means it’s probably still the most common form of logging in any given (CakePHP) application, so on with the next best thing.

Logrotate

If you use linux at all, you are probably already using logrotate.

Logrotate, as described by its own man page:

logrotate is designed to ease administration of systems that generate large numbers of log files. It allows automatic rotation, compression, removal, and mailing of log files. Each log file may be handled daily, weekly, monthly, or when it grows too large.

Take a look inside /etc/logrotate.d/ and you’ll find a few example log rotate config files which will give you an idea of what it is already doing for you - these config files are pretty simple, how about setting up logrotate to automatically clean up your application log files?

If log files are being written to a consistent location, that’s pretty trivial to achieve. Consider there are innumerous CakePHP 3.x applications all deployed into /var/www/:domain that would mean the following pattern describes where all php application log files are:

/var/www/*/tmp/logs

Here’s an example CakePHP application log rotate config file, which will rotate all log files daily, compress them, and keep up to 12 old log files:

# /etc/logrotate.d/php-apps
/var/www/*/tmp/logs/*.log {
   rotate 12
   daily
   missingok
   notifempty
   compress
   delaycompress
}

With this config, a server is safe from unloved log files gradually taking over a server until it collapses.

It’s not necessary to wait for log rotate to run on its normal schedule, to verify that the config file does what’s expected - it can be launched directly via the cli:

$ logrotate --force -v /etc/logrotate.d/php-apps
reading config file /etc/logrotate.d/php-apps

Handling 1 logs

rotating pattern: /var/www/*/tmp/logs/*.log  forced from command line (12 rotations)
empty log files are not rotated, old logs are removed
considering log /var/www/example.com/tmp/logs/debug.log
...

Conclusion

You should probably be using syslog, but if you’re not, don’t leave your log files lying around as a (slow) time bomb. Even on a development machine, I’d recommend to configure logrotate to keep things tidy; it at least eliminates one cause for a server needing urgent attention.