Almost any computer system has log files. Fortunately, many of these systems have a facility in place that rotates or trims these log files. Sometimes, a specific application can even rotate or manage its log files internally.
That having been said, there are many reasons why you, as an administrator, might need to be concerned with log files. For instance, if your system does not rotate its own logs, you will have to set up your own log rotation system. Likewise, you may have applications (custom or third party) on your system that have continuously growing log files and do not manage these logs in any way. If you are lucky, the log rotation facility provided with your system can be extended to handle additional log files. If it can't, you may have to add a custom rotation system just to handle any extra log files on your system.
If you do have to add a new rotation system, you may want to use that system for all of your log rotation needs. Doing so can simplify your life and add more power and flexibility to your log rotation possibilities. A custom rotation system can also be applied to multiple platforms and can support the exact style of log rotation you require.
Fortunately, there are several log rotation programs available that you can deploy on almost any system. I will present a few of the more popular ones in this section. If you are already using (or are thinking about using) GNU cfengine, it can also take care of log rotations for you, thus eliminating the need for an additional program.
Red Hat's logrotate
Red Hat has a log rotation system called logrotate. It has shipped with Red Hat Linux for as long as I can remember and has slowly evolved over the years. It is written in C and can be directly compiled on or easily ported to almost any system. Its source can be found at ftp://ftp.redhat.com/pub/redhat/linux/code/logrotate/.
The program is simple enough. The actual executable is typically installed as /usr/sbin/logrotate. The main configuration file is usually /etc/logrotate.conf (but you can specify any file on the command line). The program also maintains the data file /var/lib/logrotate.status (which can also be specified on the command line with the -s option).
On Red Hat Linux, logrotate is set to execute once per day through the cron daemon. You can specify any number of configuration files on the command line, but only /etc/logrotate.conf is specified in the cron job. That configuration file contains global settings and includes the directory /etc/logrotate.d/, which causes the configuration files in that directory to be read as well. This directory can contain any number of files, usually one file per application or set of log files.
Here is the main portion of Red Hat's default /etc/logrotate.conf:
# rotate log files weekly
weekly
# keep 4 weeks worth of backlogs
rotate 4
# create new (empty) log files after rotating old ones
create
# uncomment this if you want your log files compressed
#compress
# RPM packages drop log rotation information into this directory
include /etc/logrotate.d
These global options tell logrotate to rotate each log file once per week and to keep four rotated files. logrotate will create a new empty file once the rotation has taken place and can optionally compress the rotated files. Finally, the include directive is used to include additional configuration files from the /etc/logrotate.d/ directory. The rest of this file follows:
/var/log/wtmp {
monthly
create 0664 root utmp
rotate 1
}
This block of code rotates the file /var/log/wtmp. Everything within the braces applies to this file only. In this case, the global weekly rotation setting is changed to monthly and the rotation limit is reduced to one. Once the file has been rotated, it is replaced with an empty file with permissions of 0664, an owner of root, and a group of utmp.
There are actually about 40 commands you can use within the configuration files. These commands allow you to rotate files based on their size, mail the log entries, and execute pre and post rotate scripts. They are all discussed in the man page provided with the program. I am not going to discuss all of these commands in this section, but I will provide one more example that is a bit more complicated:
/var/log/samba/*.log {
notifempty
missingok
sharedscripts
copytruncate
postrotate
/bin/kill -HUP 'cat /var/run/samba/smbd.pid \
/var/run/samba/nmbd.pid 2> /dev/null' 2> /dev/null || true
endscript
}
This configuration file is placed in the /etc/logrotate.d/ directory and is part of the Samba package. The first thing you should notice is that a wildcard is used in the log file specification. This helps make the configuration files much less complex. A file will not be rotated if it is empty (notifempty). If no files are found, Logwatch will not complain (missingok). Any scripts within this block are run once after all files have been rotated (sharedscripts) and not after each individual rotation. Finally, when the files are rotated, they are copied to a new filename and then the original file is truncated (copytruncate).
This example also has a postrotate script. This script sends the HUP signal to the two Samba processes after the rotation has taken place. As is the case with many daemons, this signal causes them to close and reopen any log files. If this is not done, they will keep writing to the log file after it has been truncated, causing file corruption.
For more information on the commands available within the logrotate configuration files, see the man page provided with the program.
Rotating Logs with Spinlogs
The Spinlogs program is a very simple, yet useful, file rotation script that can run on just about any system. Its only requirement, apart from the standard UNIX commands such as egrep, mv, and cp, is the Korn shell (ksh).
You can download Spinlogs from
http://isle.wumpus.org/cgi-bin/pikie?SpinLogs. Its archive contains the shell script, a sample configuration file, and its license file (the "Artistic License"). Here is an example configuration file:
# log_filename owner:group mode num size time flags pid_file signal
# | | | | | | | |
/var/log/messages root:root 0600 14 * D Z
This simply says to rotate /var/log/messages every time the program is executed (because no maximum size is specified). The meanings for each field are as follows:
log_filename: The file to be rotated, surprisingly enough. No wildcards are allowed.
owner:group: The ownership to be applied to the rotated log files as well as the new blank log file.
mode: The permissions for the rotated and original log files.
num: How many rotated log files to keep. In this case, if the program is being run daily, two weeks worth of logs will be stored. If the program was run hourly, you'd only have 14 hours of log archives.
size: The maximum size of a file, in kilobytes. A file will only be rotated if it is larger than this limit. If set to *, the file will always be rotated.
time: This is only here to maintain the same file format as the FreeBSD's newsyslog log rotation utility. The Spinlogs program does not support time-based rotations.
flags: This field can contain any number of the following flags:
D: Do not create an empty log file in its place after rotation.
Z: Use this flag to compress files after rotation.
0: Don't compress the .0 rotated log (the most recent archive).
B: This says the log file is binary and a rotation message should not be placed in the file.
pid_file: If specified, this file should contain a PID for a program to signal after the rotation has completed.
signal: This is the signal to be sent to the process (such as HUP).
Once you have created your configuration file, you should run the spinlogs command on a regular basis (usually daily) from your system's cron daemon. A typical invocation would be:
% /usr/local/sbin/spinlogs \ -c /usr/local/etc/spinlogs.conf \
-p /var/run/syslogd.pid
This specifies a configuration file of /usr/local/etc/spinlogs.conf and identifies /var/run/syslogd.pid as the file that contains the process ID for the syslog program. The HUP signal will be sent to this process ID after Spinlogs has completed execution.
Log Rotation with cfengine
If you have read the discussions of GNU cfengine thus far (particularly section 6.4), you should realize that cfengine can do just about anything on your systems automatically. When it comes to log rotation, it can get the job done just about as well as any other program, and it can do so on any of your systems.
Unfortunately, the section of cfengine that can perform these rotations—the disable section—is not exactly obvious. The disable section can be placed in cfagent.conf along with the rest of your configuration directives. Like many other sections, the disable section was originally created to delete or rename unwanted files, but it is flexible enough to handle other tasks, such as log rotation.
Like many log rotation systems, you can rotate based on time or file size, and you can store as many rotated files as you desire. This first example will cause cfengine to rotate your web server's access log every Sunday:
disable:
Sunday::
/var/log/httpd/access_log rotate=52
The Sunday class will only be defined on Sundays, so the rotation will only take place on that day. The rotate argument specifies that (at most) 52 rotated files should be present. Here, since the rotation is performed once per week, one year's worth of logs will be stored (52 weeks). One problem with this section is that it assumes cfagent is only executed one time on each Sunday. If it is executed more than once, the rotation will be performed multiple times for that week (and some rotated files will contain less than a day's worth of data).
Another (perhaps better) option is to rotate based on the log file's size. Here is another example:
disable:
/var/log/httpd/access_log size=>100mb rotate=4
This only keeps four previous log files, and the rotation only takes place if the file is greater than 100Mb. It is important to remember that, if you run cfagent only once per day, the file will only be rotated at most once per day. The rotated file can, however, be much larger than 100Mb if your daily traffic is significant. Even if the file grows to one gigabyte in a day, it will only be rotated the next time cfagent executes.
You can also perform certain actions (like sending the HUP signal to a daemon) by defining a class when a rotation occurs. For example:
disable:
/var/log/httpd/access_log size=>100mb rotate=4 define=http_rotated
processes:
http_rotated::
"httpd" signal=hup
Whenever the log file is rotated, the http_rotated class is defined. Then, in the processes section, the HUP signal is sent to all httpd processes only if this class is defined.
No comments:
Post a Comment