Comprehensive guide to Log file management with Logrotate on Ubuntu 20.04/22.04

In any production server environment, managing log files is a crucial aspect of system administration. Log files contain valuable information about system events, application behavior, and potential issues, making them indispensable for troubleshooting and monitoring purposes. However, if left unchecked, log files can grow rapidly, consuming significant disk space and potentially leading to performance degradation or even system crashes.

Logrotate is a powerful utility designed to address this issue by automatically rotating, compressing, and pruning log files based on predefined rules and schedules. By regularly compressing and archiving old log files, and removing obsolete ones, Logrotate ensures that disk space is used efficiently and that log files remain manageable.

In this comprehensive guide, we will explore Logrotate’s functionality, its default configuration on Ubuntu 20.04 and 22.04, and walk through the process of setting up custom log rotation rules for a fictional application.

Prerequisites

Before proceeding, ensure that you have the following:

• An Ubuntu 20.04 or 22.04 server with a non-root user account with sudo privileges.
• Familiarity with basic command-line operations and text editing.

Logrotate is pre-installed on Ubuntu, but if you need to install it manually, you can do so by running the following commands:

$ sudo apt update
$ sudo apt install logrotate

Step 1: Verifying the Logrotate Version

Although Logrotate comes pre-installed on Ubuntu, it’s a good practice to verify the installed version and its default settings. You can do this by running the following command:

$ logrotate --version

This command will display the version number, as well as information about default settings such as the mail command, compression command, and state file path. Here’s an example output:

logrotate 3.19.0
    Default mail command:       /usr/bin/mail
    Default compress command:   /bin/gzip
    Default uncompress command: /bin/gunzip
    Default compress extension: .gz
    Default state file path:    /var/lib/logrotate/status
    ACL support:                yes
    SELinux support:            yes

If you’re using a non-Ubuntu distribution or have a significantly different version of Logrotate, some configuration options covered in this guide may not apply. In such cases, consult the man pages (man logrotate) or online documentation for your specific Logrotate version.

Step 2: Exploring the Default Configuration

On Ubuntu systems, Logrotate’s configuration is primarily managed through two locations:

1. /etc/logrotate.conf: This file contains the default settings and sets up log rotation for a few system logs that are not owned by any installed packages. It also includes configuration files from the /etc/logrotate.d directory using an include statement.
2. /etc/logrotate.d/: This directory contains configuration files for various installed packages that require log rotation. For example, you’ll find configuration files for core system tools like apt, dpkg, rsyslog, and others.

Let’s examine the default configuration in the /etc/logrotate.conf file:

$ sudo cat /etc/logrotate.conf

This file sets up weekly log rotations for log files owned by the root user and the syslog group. It keeps four rotated log files (rotate 4) and creates new empty log files after rotation (create). Additionally, it includes configuration files from the /etc/logrotate.d directory.

Next, let’s look at an example configuration file from the /etc/logrotate.d directory for the apt package:

$ sudo cat /etc/logrotate.d/apt

/var/log/apt/term.log {
  rotate 12
  monthly
  compress
  missingok
  notifempty
}
/var/log/apt/history.log {
  rotate 12
  monthly
  compress
  missingok
  notifempty
}

This file contains configuration blocks for two log files: term.log and history.log, located in the /var/log/apt/ directory. The configuration options used here are:

• rotate 12: Keep 12 old rotated log files (overriding the default of 4).
• monthly: Rotate the logs once a month (overriding the default of weekly).
• compress: Compress the rotated log files using gzip.
• missingok: Don’t generate an error if the log file is missing.
• notifempty: Don’t rotate the log file if it’s empty.

Any options not specified in these configuration blocks will inherit the default values from /etc/logrotate.conf.

Step 3: Setting Up a Custom Configuration

While the default configuration is sufficient for most system logs, you may need to set up custom log rotation rules for your applications. Logrotate offers two main approaches for this:

1. Adding Configuration to /etc/logrotate.d/
2. Creating an Independent Configuration

Adding Configuration to /etc/logrotate.d/

This approach involves creating a new configuration file in the /etc/logrotate.d/ directory. The configuration defined in this file will be run daily as the root user, along with all the other default Logrotate jobs.

Let’s set up a configuration for a fictional web application called “your-app” that generates access.log and error.log files in the /var/log/your-app/ directory. The application runs as the www-data user and group.

First, create a new configuration file:

$ sudo nano /etc/logrotate.d/your-app

Add the following configuration to the file:

/var/log/your-app/*.log {
    daily
    missingok
    rotate 14
    compress
    notifempty
    create 0640 www-data www-data
    sharedscripts
    postrotate
        systemctl reload your-app
    endscript
}

Here’s what each option does:

• daily: Rotate the logs daily (overriding the default weekly rotation).
• missingok: Don’t generate an error if the log file is missing.
• rotate 14: Keep 14 old rotated log files.
• compress: Compress the rotated log files using gzip.
• notifempty: Don’t rotate the log file if it’s empty.
• create 0640 www-data www-data: Create a new empty log file with permissions 0640, owned by the www-data user and group.
• sharedscripts: Run any scripts defined in the configuration (e.g., postrotate) only once per run, instead of for each log file.
• postrotate to endscript: This block contains a script that will run after log rotation. In this case, it reloads the your-app service to ensure it starts writing to the new log file.

Save and close the file. You can test the configuration by running a dry run:

$ sudo logrotate /etc/logrotate.conf --debug

This command executes Logrotate with the standard configuration file and enables debug mode, printing information about which log files would be handled and the actions that would be taken.

Creating an Independent Configuration

In some cases, you may need to run Logrotate as a non-root user or rotate logs more frequently than the default daily schedule. In such situations, you can create an independent Logrotate configuration and set up a cron job to run it at the desired interval.

Suppose you have an application running as the bob user, generating logs in the /home/bob/logs/ directory. You want to rotate these logs hourly.

First, create a configuration file in your home directory:

$ nano /home/bob/logrotate.conf

Add the following configuration:

/home/bob/logs/*.log {
    hourly
    missingok
    rotate 24
    compress
    create
}

This configuration will rotate the logs hourly, keeping 24 old rotated log files, compressing them, and creating a new empty log file after rotation.

Save and close the file.

Next, create a log file for testing purposes:

$ cd ~
$ mkdir logs
$ touch logs/access.log

Now, run Logrotate with the new configuration and specify a state file location:

$ logrotate /home/bob/logrotate.conf --state /home/bob/logrotate-state --verbose

The --state option specifies the location of the state file, which records information about the logs processed during each run. The --verbose flag displays detailed output about Logrotate’s actions.

You should see output similar to the following:

reading config file /home/bob/logrotate.conf
Handling 1 logs
rotating pattern: /home/bob/logs/*.log  hourly (24 rotations)
empty log files are rotated, old logs are removed
considering log /home/bob/logs/access.log
  log does not need rotating

Logrotate recorded information about the logs it encountered in the state file. You can view the contents of the state file with:

$ cat /home/bob/logrotate-state

If you run the same command an hour later, the log file should be rotated as expected.

To force Logrotate to rotate the log file immediately (for testing purposes), use the --force flag:

$ logrotate /home/bob/logrotate.conf --state /home/bob/logrotate-state --verbose --force

Finally, set up a cron job to run Logrotate hourly. Open your user’s crontab with:

$ crontab -e

Add the following line to the crontab file:

14 * * * * /usr/sbin/logrotate /home/bob/logrotate.conf --state /home/bob/logrotate-state

This will run the logrotate command on the 14th minute of every hour, using the full path to the logrotate binary and specifying the configuration file and state file locations.

Save and exit the crontab file. The cron job will now run at the specified schedule.

Advanced Configuration Options

While the examples covered in this guide demonstrate some common Logrotate options, there are many more advanced options available for fine-tuning log rotation behavior. Here are a few notable options:

• dateext: This option appends a date extension to the rotated log file names, making it easier to identify the time period covered by each log file.
• dateformat: Specifies the date format to be used with dateext.
• mail: Sends a log file rotation report to the specified email address.
• olddir: Moves rotated log files to a different directory instead of keeping them in the same directory as the current log file.
• prerotate and postrotate: These options allow you to run scripts before and after log rotation, respectively.
• size: Rotates log files based on their size, instead of a fixed schedule.

You can explore these and other options by consulting the Logrotate manual page (man logrotate) or the online documentation.

Conclusion

Effective log file management is crucial for maintaining a healthy server environment and ensuring that valuable log data is preserved and accessible when needed. Logrotate is a powerful tool that simplifies this task by automating log rotation, compression, and pruning based on customizable rules and schedules.

By following the examples and best practices outlined in this guide, you can effectively manage log files for your applications, ensuring optimal disk space utilization and easy access to historical log data when needed.