Troubleshooting When Your Node is Running Out of Disk Space
A full disk can render a node unresponsive — services that need to write to the disk fail to start, log writes fail, and in extreme cases the node can drop into Powered Off. This guide explains the most common sources of disk consumption and how to identify and free space.
Common Causes of Disk Consumption
Log Files
A server log file is a text record of activity over time. Logs are usually written to /var/log/, are created and rotated by the server automatically, and grow steadily — especially when applications are noisy or are emitting errors. If no rotation or purge policy is in place, log files will keep growing until the disk fills up.
Use a log-rotation tool like logrotate to keep log files within a known size. If you do not want to keep certain logs at all, truncate them directly:
cat /dev/null > /var/log/messages
This wipes the contents of /var/log/messages without affecting any process that has the file open.
Backup Files
Storing regular backups locally on the same node is one of the fastest ways to fill the disk. Make sure backup retention is enforced — old backups must be deleted on a schedule, or moved off the node to Object Storage.
Temporary Files and Uploads
Many applications write temporary state into /var/tmp/. These files often outlive the process that created them. Audit /var/tmp/ and clear what is no longer needed.
Also check directories that accept regular file uploads. Old uploads accumulate over time and can occupy a large share of disk without being noticed.
How to Identify What is Using the Disk
Step 1: Find the Full Partition
If you received a "root partition full" alert, confirm which mount is full:
df -h
The output shows every mounted filesystem with Use% for each. Focus on whichever partition is at or near 100%.
Step 2: Walk the Directory Tree
For a quick listing of which subfolders inside a path are heaviest, use du:
du -sh /root/user/*
The output looks like:
1.5GB /root/user/backupfile
158MB /root/user/text
1MB /root/user/textfolder
Step 3: Use ncdu for Faster Analysis
ncdu (NCurses Disk Usage) is an interactive version of du and is the fastest way to navigate a large directory tree by size.
Install on RHEL / CentOS:
yum install ncdu
Install on Ubuntu / Debian:
sudo apt-get install ncdu
Scan a path:
ncdu /
ncdu scans the path and presents an interactive tree sorted by size. Use arrow keys to navigate, d to delete a selected file, and q to quit.
Disk Space is "Missing" — df Disagrees with du
A common puzzle: df reports the disk is full, but du does not account for all the space. This happens when a running process is still writing to a file that has been deleted from the filesystem but not yet closed by the process. The file disappears from du but its blocks are still held by the kernel.
Find these files:
lsof | grep '(deleted)'
You can either restart the process holding the deleted file, or — if rebooting is acceptable — reboot the node. After a reboot, the kernel releases the blocks and the disk shows the freed space.
If You Cannot SSH In
If the disk is full to the point that you cannot SSH into the node, log in over the VNC console from MyAccount and free space from there. If even the console is unresponsive, see Recovery Mode — recovery mode lets you mount the disk read/write and delete files without needing the OS to boot normally.