Understanding IOwait

When your server seems slow or unresponsive, one common culprit might be IOwait. This article will explain what IOwait is, why it matters, and how you can track down the processes responsible for it using simple command-line tools.

What Is IOwait?

IOwait is a measure of the time the CPU spends waiting for disk input/output (I/O) operations to complete. When this number is high, it means your server’s processor is idling while it waits on the disk to read or write data — which often results in slower performance for your applications or websites.

Why Is IOwait a problem?

High IOwait indicates that your disk is a bottleneck. This could be due to:

• An application reading or writing large amounts of data
• A backup or virus scan running in the background
• Inefficient disk usage by a process
• Hardware issues with the disk

How to find what’s causing IOwait

Step 1: Confirm IOwait is happening

Use the iostat command to check your current I/O statistics. If the %iowait value is consistently high (generally above 10%), you may have an issue worth investigating.

iostat -x 1

If you don’t have iostat installed, you can usually get it with:

yum install sysstat

Step 2: Use block_dump to log disk activity

This method logs which processes are doing disk I/O to the system’s kernel ring buffer (dmesg):

1. Turn on logging:

echo 1 > /proc/sys/vm/block_dump

2. Wait a few minutes while the system logs disk activity.

3. Turn off logging:

echo 0 > /proc/sys/vm/block_dump

4. Parse the results to see which processes are most active:

dmesg | egrep "READ|WRITE|dirtied" | egrep -o '([a-zA-Z]*)' | sort | uniq -c | sort -rn | head

• You might see a process like kjournald at the top — this is normal and simply indicates the system is writing to disk.
• The real culprit is usually the next process in the list.

4. Clean up the ring buffer after you’re done:

dmesg -c

Optionally: use iotop for real-time monitoring

If you’d prefer a more interactive view, iotop provides a real-time summary of I/O usage by process.

1. Install iotop:

yum install iotop

2. Run it:

iotop

3. This will show a live view of:
   • Which processes are reading/writing to disk
   • Swap usage
   • IOwait percentages per process

Use the arrow keys to sort columns and identify the biggest I/O consumers.

Summary

High IOwait can seriously impact your server’s performance, but with tools like iostat, block_dump, and iotop, you can quickly identify the cause and take action. Whether it’s optimizing your applications, rescheduling backups, or upgrading your storage solution, pinpointing the root of the issue is the first step toward a faster, more stable system.

If you’d like help interpreting your results or need assistance reducing disk I/O, don’t hesitate to contact our support team.