Search our site for answers
Help Docs Server Administration Linux Server Administration Apache Web Server Troubleshooting Apache startup error: no space left on device

Troubleshooting Apache startup error: no space left on device

Learn to fix the Apache startup error "Failed to set a watch for httpd.service's PID file: No space left on device" by checking disk space or increasing inotify watches.
Troubleshooting
4 min read

If you’re encountering an error message like:

systemd[1]: Failed to set a watch for httpd.service's PID file /run/apache2/httpd.pid: No space left on device

Or:

(28)No space left on device: AH00023: Couldn't create the fcgid-pipe mutex


…when attempting to start Apache (httpd), it can be perplexing. While the message “No space left on device” often points to a full disk, it can also indicate that the system has run out of available inotify watches. This article will guide you through troubleshooting both potential causes.

Understanding the error

The error message indicates that the system’s systemd service manager tried to monitor the Apache PID file (/run/apache2/httpd.pid) for changes. This monitoring often uses the kernel’s inotify subsystem, which allows applications to be notified of filesystem events. The “No space left on device” part can mean either:

  • The storage partition where /run or / resides is genuinely full.
  • The system has exhausted its limit of inotify watches, which are resources used to monitor files and directories.

Step 1: Check for full disk or partition

First, let’s rule out the most common cause: a full disk partition. You can check your server’s disk space usage using the df command.

Open your server’s command line interface (CLI) via SSH and run the following command:

df -h

This command will display disk space usage in a human-readable format (e.g., MB, GB). Examine the output, paying close attention to the Use% column for partitions like /, /run, or any other relevant mount points. If you see any partition at or near 100% capacity, you’ve found the problem. You’ll need to free up disk space by removing unnecessary files, archiving old data, or resizing the partition if possible.

If your disk space appears to be adequate, proceed to the next step.

Step 2: Increase the inotify watch limit

If disk space is not an issue, the error likely stems from the system running out of available inotify watches. Inotify is a Linux kernel subsystem that notices changes to the filesystem. Services like Apache (via systemd) might use inotify watches to monitor important files. If too many files are being watched system-wide by various applications, this limit can be reached.

You can temporarily increase the maximum number of inotify watches allowed per user by executing the following command as the root user or with sudo:

echo 1048576 > /proc/sys/fs/inotify/max_user_watches

This command sets the limit to 1,048,576, a significantly higher value than the default on many systems (often 8192). This change takes effect immediately.

After running this command, try starting Apache again:

systemctl start httpd

Or, if your system uses apache2 as the service name:

systemctl start apache2

If Apache starts successfully, the inotify watch limit was indeed the issue.

Step 3: Make the inotify fix persistent

The change made in Step 2 by echoing to /proc/sys/fs/inotify/max_user_watches is temporary and will be reset upon a system reboot. To make this change permanent, you need to add the setting to the sysctl.conf file.

First, open the sysctl.conf file in a text editor. Common editors include nano or vi:

nano /etc/sysctl.conf

Add the following line to the end of the file:

fs.inotify.max_user_watches=1048576

Save the file and exit the text editor. (In nano, you can do this by pressing Ctrl+X, then Y to confirm, and then Enter).

To apply the changes from /etc/sysctl.conf without rebooting, run the following command:

sysctl -p

This command loads the sysctl settings from the default file, including the new max_user_watches value you just added. Your increased inotify watch limit will now persist across reboots.

Conclusion

The “Failed to set a watch” error for Apache’s PID file can be misleading, but by checking disk space and then the inotify watch limit, you can typically resolve it quickly. Ensuring the inotify limit is set appropriately for your server’s needs will help maintain Apache’s stability and performance.

If you’ve followed these steps and Apache still fails to start, or if you have any concerns about making these changes, please don’t hesitate to contact our support team for assistance.

Was this article helpful?