Mastering Log Rotation in Linux with Logrotate

Logging to files is a recommended approach for managing application logs. Even if you’re eventually streaming them to a logging backend, writing them to local files first decouples log generation from transmission which can be done more reliably through a telemetry pipeline.

However, large log files present several challenges. They are difficult to search, consume significant CPU and memory to open or parse, and can even cause application failures if disk space is exhausted.

Log rotation solves these problems by periodically archiving active log files and creating new ones for incoming entries.

This process typically involves renaming the current log file (such as by adding a timestamp or number) and then starting a new file with the original name. Older, rotated files can be compressed to save disk space and are eventually deleted based on a configured retention period.

On Linux systems, Logrotate is the standard utility for managing log file rotation. This guide will explain its operation and how to customize it for your specific logging requirements.

Getting started with Logrotate

Logrotate is typically pre-installed on most Linux distributions and runs automatically via the system’s cron scheduler. To verify its installation and check the version, use:

1
logrotate --version

You will see the installed version and its default settings, such as:

13456789
logrotate 3.21.0
    Default mail command:       /usr/bin/mail
    Default compress command:   /bin/gzip
    Default uncompress command: /bin/gunzip
    Default compress extension: .gz
    Default state file path:    /var/lib/logrotate/status
    ACL support:                yes
    SELinux support:            yes

This output reveals the tools Logrotate uses for tasks like compression and mail notifications, along with the path to its state file, which tracks rotation history.

While Logrotate’s behavior is generally consistent across distributions, this guide will reference default configurations found in Ubuntu 24.04 LTS.

Configuring Logrotate

Logrotate’s configuration follows a hierarchical structure. Global default settings are defined in the main configuration file at at /etc/logrotate.confCopy, while application-specific configurations are placed in the /etc/logrotate.d/Copy directory.

Here’s a look at the default /etc/logrotate.confCopy in Ubuntu:

/etc/logrotate.conf
12456891112141517182021
# rotate log files weekly
weekly
# use the adm group by default, since this is the owning group
# of /var/log/.
su root adm
# keep 4 weeks worth of backlogs
rotate 4
# create new (empty) log files after rotating old ones
create
# use date as a suffix of the rotated file
#dateext
# uncomment this if you want your log files compressed
#compress
# packages drop log rotation information into this directory
include /etc/logrotate.d

Based on this configuration:

• Logs are rotated weekly.
• New, empty log files are created (createCopy) after the old ones are rotated.
• Up to four weeks of backed-up logs (rotate 4Copy) are retained.
• The su root admCopy directive ensures rotations execute with rootCopy user and admCopy group permissions, preventing issues with system log files.
• Compression (compressCopy) is disabled by default but can be enabled to save disk space.

The final include /etc/logrotate.dCopy line instructs Logrotate to load all configuration files from the /etc/logrotate.dCopy directory. This allows for a modular approach where packages define their own log rotation rules without altering global settings.

We’ll cover many useful options in this article, but ensure to consult the manual page for a comprehensive list of all available directives:

1
man logrotate

Customizing application-specific log rotation policies

The global configuration file only sets default directives that apply to any log files defined in included configurations (such as /etc/logrotate.dCopy).

Inspecting this directory will reveal configuration files for various installed services, such as rsyslogCopy, nginxCopy, or dpkgCopy.

The structure of these application-specific files is similar to the global configuration but is scoped to particular log files:

/etc/logrotate.d/nginx
123456789
/var/log/nginx/*.log {
    # Directives here apply only to Nginx log files
    daily
    missingok
    rotate 14
    compress
    notifempty
    # other Nginx-specific settings
}

Settings within these files override the global defaults from /etc/logrotate.confCopy. If a specific directive isn’t set in an application’s configuration file, it inherits the value from the global configuration.

For instance, if /etc/logrotate.confCopy specifies rotate 4Copy and an application-specific file in /etc/logrotate.d/Copy does not include a rotateCopy directive, that application’s logs will also be rotated keeping four backups.

Now that you understand Logrotate’s configuration structure, let’s explore some key directives to manage common rotation scenarios.

Understanding log rotation triggers

By default, Logrotate uses a time-based strategy where log files are rotated on a fixed schedule. The following intervals are supported: hourlyCopy, dailyCopy, weeklyCopy, monthlyCopy, and yearlyCopy.

When you specify one of these intervals, Logrotate will check whether the time since the last rotation matches the specified interval. If so, the log is rotated. This works well for applications with predictable log volumes.

Important note on hourly rotation

Logrotate is often configured system-wide to run once per day via /etc/cron.daily/logrotateCopy. If you use the hourly option for a specific log file, you must ensure Logrotate itself runs more frequently. This involves moving the corresponding cron job to an hourly execution path with:

1
sudo mv /etc/cron.daily/logrotate /etc/cron.hourly # On Ubuntu

If you need Logrotate to run even more frequently, you may need to set up a custom cronjob.

Size-based rotation

For services generating unpredictable log volumes, like web servers experiencing traffic spikes, a purely time-based schedule might lead to excessively large log files or, conversely, numerous small files during quiet periods.

That’s where size-based rotation comes in. The size directive tells Logrotate to trigger rotation only when a log file reaches a specific size, regardless of the time schedule. For example:

/etc/logrotate.d/app
1234
/var/log/app/*.log {
  daily # This no longer applies
  size 50M # the last specified option takes the precedence
}

This setup ensures that the specified log file only rotates whenever it exceeds 50 megabytes, even if the daily schedule has been met. You can specify sizes in bytes (default), kilobytes (k), megabytes (M), or gigabytes (G).

Note that size and the time-based options are mutually exclusive. This means the last specified option will always take precedence. So if the sizeCopy position above is swapped with dailyCopy, then the files will rotate daily and the specified size will be ignored.

Combining time and size constraints

For more nuanced control, Logrotate offers minsizeCopy and maxsizeCopy directives, which work in conjunction with time-based schedules:

• minsizeCopy ensures that rotation happens only if the time schedule is met and the log file has reached a minimum size.

/etc/logrotate.d/app
1234
/var/log/app/*.log {
  daily
  minsize 50M
}

Here, the log rotates daily, but only if its size is at least 50 megabytes. If it’s smaller than 50M when the daily check occurs, it won’t be rotated until it meets this size criterion on a subsequent daily check.

• maxsizeCopy enforces rotation as soon as a log file exceeds the specified size, even if the scheduled time hasn’t arrived. It also rotates at the scheduled time if the maxsizeCopy hasn’t been breached.

/etc/logrotate.d/app
1234
/var/log/app/*.log {
  daily
  maxsize 50M
}

This configuration means logs are rotated daily. However, if a log file grows beyond 50 megabytes before the next scheduled daily rotation, it will be rotated immediately during the next run.

These options ensure logs don’t rotate too frequently if they are small (minsizeCopy) or grow too large before their scheduled rotation time (maxsizeCopy).

create vs. copytruncate for log file handling

Logrotate defines how the active log file is archived and a new one is initiated with two primary methods: createCopy and copytruncateCopy.

12345
/var/log/app/*.log {
  create
  # Optionally specify the file permissions, owner and group like this:
  # create 644 <appuser> <group>
}

The create directive is the default method, and it operates by renaming the active log file (app.logCopy to app.log.1Copy) and creates a new empty file with the original name.

The new file inherits the attributes (like permissions, owner, and group) of the old log file by default, but you can explicitly define these for the newly created file as shown in the commented example.

This method is generally preferred as it minimizes the risk of log loss, provided the application creating the logs can immediately start writing to the new file descriptor.

The copytruncateCopy method takes a different approach by copying the current log file to create the rotated version, and then truncating the original file in place. Using our previous example, the app.logCopy data will be copied over to app.log.1Copy, and then the app.logCopy file is immediately emptied.

/etc/logrotate.d/app
123
/var/log/app/*.log {
  copytruncate
}

This method allows applications to continue writing to the same log file using their existing file handle, as the original file is never renamed or replaced, only emptied. However, copytruncateCopy has notable drawbacks:

• There’s a brief window between the copy operation and the truncate operation. Any log entries written during this very short interval might be written to the original file before it’s truncated but after it was copied, leading to those entries being lost.
• During the copy phase, both the original log file and its copy exist simultaneously for a brief moment which consumes more disk space.

In most scenarios, createCopy is the recommended method. You only need to ensure your applications are designed to handle log file rotation gracefully, meaning they can detect that the log file has been renamed and automatically open and start writing to the newly created log file.

copytruncateCopy should only be used as a fallback for legacy applications or services that cannot be easily modified to close and reopen their log files, and where the small risk of log loss is acceptable.

Logrotate also offers a copyCopy directive which copies the log file without truncating the original file. This means the original log file continues to grow, and the copied file serves as a snapshot.

This may be useful for niche scenarios but isn’t a direct alternative for managing ongoing application logs in the same way as createCopy or copytruncateCopy.

Organizing rotated log files with timestamps

As you’ve already seen, rotated files are given incrementing numbers by default (e.g., app.log.1Copy, app.log.2Copy). While this maintains order, it doesn’t provide context about when the file was created or rotated.

Using timestamps in filenames can significantly simplify locating logs from a specific period, and Logrotate supports this through the dateextCopy and dateformatCopy directives:

/etc/logrotate.d/app
1234
/var/log/app/*.log {
    dateext
    dateformat # defaults to -%Y%m%d for daily rotations, and -%Y%m%d%H for hourly rotations.
}

With this configuration, a log file rotated on May 25, 2025, will be named app.log-20250525Copy, clearly indicating the rotation date.

If logs are rotated multiple times within the same day (such as when using size-based rotation or hourly schedules), you can include more granular time information, like a Unix timestamp (%sCopy), to ensure unique filenames:

123
/var/log/app/*.log {
    dateformat -%Y%m%d-%s # produces files like app.log-20250525-1748181385
}

This creates file names such as app.log-20250525-1748181385Copy, where -1748181385Copy is the Unix timestamp at the moment of rotation.

Another relevant directive is dateyesterdayCopy. When used with dateextCopy, it stamps the rotated log file with the previous day’s date instead of the current date. It may come in handy if files are typically processed or rotated shortly after midnight but their contents pertain to the preceding day.

Compressing rotated logs to save disk space

Compressing rotated log files is a straightforward and effective method for conserving disk space, especially when retaining logs locally for extended periods.

Logrotate provides several directives to manage compression. The primary one is compress which uses gzip compression on rotated files. For example, app.log.1Copy becomes app.log.1.gzCopy.

/etc/logrotate.d/app
123
/var/log/app/*.log {
    compress
}

If you prefer a different compression utility, you can specify it through the compresscmdCopy and uncompresscmdCopy directives:

/etc/logrotate.d/app
123456
/var/log/app/*.log {
    compress
    compresscmd /usr/bin/bzip2
    uncompresscmd /usr/bin/bunzip2
    compressext .bz2
}

In this configuration, compressed files now will have the .bz2Copy extension, and Logrotate will use bzip2Copy and bunzip2Copy to handle them.

The delaycompressCopy option is useful to prevent the most recently rotated log file from being compressed immediately. This is beneficial if a program might continue writing to its old log file for a short time after rotation, even after the file has been renamed.

/etc/logrotate.d/app
123456
/var/log/app/*.log {
    compress
    compresscmd /usr/bin/bzip2
    uncompresscmd /usr/bin/bunzip2
    compressext .bz2
}

With delaycompressCopy, the app.log.1Copy file (the most recent backup) will not be compressed during the current rotation cycle. Instead, its compression will be deferred until the next rotation cycle (when app.log.1Copy becomes app.log.2.gzCopy). This leaves the latest rotated log uncompressed for immediate access or to allow processes to gracefully release their file handles.

To fine-tune the compression process, you can use the compressoptionsCopy directive. For gzip, you can specify a compression level from -1Copy (fastest, least compression) to -9Copy (slowest, highest compression):

/etc/logrotate.d/app
1234
/var/log/app/*.log {
    compress
    compressoptions -9  # Use highest gzip compression level
}

Hooking into the rotation process with custom scripts

Logrotate allows you to execute custom scripts at various stages of the rotation process to enable actions like notifying services to switch to new log files or archiving rotated logs to remote storage.

The following directives facilitate this:

• prerotateCopy and postrotateCopy scripts are executed before and after each individual log file is rotated respectively. If a configuration block matches multiple log files, these scripts run once for each log file being rotated.
  
  However, if you use the sharedscriptsCopy directive, prerotateCopy and postrotateCopy scripts will run only once for the entire set of matching log files. For instance, Nginx’s default configuration includes a postrotateCopy script with sharedscriptsCopy which instructs Nginx to reopen its log files after all of them have been rotated:

/etc/logrotate.d/nginx
1234567
/var/log/nginx/*.log {
    # ... other directives
    sharedscripts
    postrotate
        invoke-rc.d nginx rotate >/dev/null 2>&1
    endscript
}

• The firstactionCopy script executes once before any log files matched by the configuration block are rotated. It’s suitable for preparatory tasks that need to run just once for an entire batch of logs defined in that block.
• lastactionCopy executes once after all files have been rotated and compressed. A common use case is automatically uploading rotated logs to a remote service, such as Amazon S3:

/etc/logrotate.d/app
12345
/var/log/app/*.log {
   lastaction
        aws s3 sync /var/log/app/ "s3://<my-bucket>/" --exclude "*" --include "*.gz"
   endscript
}

Diagnosing and resolving Logrotate issues

When log rotation isn’t behaving as expected, several diagnostic steps can help you pinpoint the cause. Common issues often stem from incorrect permissions, configuration errors, or timing conflicts.

The Logrotate status file is the first place to look. It records the state of log files managed by Logrotate, including the last time each file was rotated.

1
sudo cat /var/lib/logrotate/status

12345
logrotate state -- version 2
[...]
"/var/log/user.log" 2025-4-9-0:0:0
"/var/log/nginx/access.log" 2025-5-25-15:0:57
"/var/log/cron.log" 2025-4-9-0:0:0

If a log file you expect to be rotated isn’t listed here, or its last rotation date is incorrect, it could indicate that Logrotate isn’t aware of it.

This might happen if there’s no configuration file for the service in the /etc/logrotate.d/Copy directory, or if the main /etc/logrotate.confCopy doesn’t include the specific log file path.

For a more detailed view of Logrotate’s operations, run it in debug mode . This will simulate the rotation process and print verbose information about which configuration files are loaded and how Logrotate evaluates each log file:

1
sudo logrotate --debug /etc/logrotate.conf

The output will reveal syntax errors, reasons for skipping files (“log does not exist,” “log does not need rotating”), permission issues, and the actions it would take:

1234578910111213
[...]
considering log /var/log/cron.log
  log /var/log/cron.log does not exist -- skipping
not running postrotate script, since no logs were rotated
switching euid from 0 to 0 and egid from 4 to 0 (pid 13518)
rotating pattern: /var/log/ubuntu-advantage*.log  monthly (6 rotations)
empty log files are not rotated, old logs are removed
considering log /var/log/ubuntu-advantage-apt-hook.log
  Now: 2025-05-25 16:55
  Last rotated at 2025-04-09 00:00
  log does not need rotating (log is empty)
[...]

For ongoing monitoring, you can redirect Logrotate output to a dedicated log file by modifying the appropriate cron job to capture both success and error messages:

/etc/cron.daily/logrotate
1
/usr/sbin/logrotate --log /var/log/logrotate.log /etc/logrotate.conf

The --logCopy flag directs Logrotate’s output to /var/log/logrotate.logCopy each time it runs. Reviewing this file can help you track successful rotations and identify any errors that occurred.

For instance, an OpenTelemetry Collector can be configured to parse these logs and send them to a observability service like Dash0 for monitoring and alerting:

otelcol.yaml
123456791011121314151617192021222324262728293031
receivers:
  filelog: 
    include:
      - /var/log/logrotate.log
    start_at: beginning
    multiline:
      line_start_pattern: ^rotating pattern
processors:
  batch:
  transform/severity:
    error_mode: ignore
    log_statements:
      - context: log
        statements:
          - set(log.severity_number, 9)
          - set(log.severity_number, 17) where IsMatch(log.body, "error:")
exporters:
  otlphttp/dash0:
    endpoint: <your_dash0_endpoint>
    headers:
      Authorization: Bearer <your_dash0_token>
      Dash0-Dataset: <your_dash0_dataset>
service:
  pipelines:
    logs:
      receivers: [filelog]
      processors: [batch, transform/severity]
      exporters: [otlphttp/dash0]

This setup allows for centralized monitoring of Logrotate’s activity, and alerting on key events or errors.

Final thoughts

In this guide, we covered many Logrotate configurations and how to customize its behavior for your log rotation strategy.

However, effective log management doesn’t stop at rotation. To truly harness the value of your logs, they need to be actively utilized and correlated with other telemetry signals like traces and metrics.

By ingesting them into a capable observability platform like Dash0, you’ll unlock a range of powerful tools for searching, alerting, visualization, correlation, and in-depth analysis.

If you’d like to explore a platform with these features, Dash0 offers a free 14-day evaluation.