Services are down due to /tmp full

[John Stanley]

Hello,

Came across an issue today I am receiving emails about /tmp on his CPanel server being full. Websites are not responding either. I logged in, and df-h indeed showed 100% usage, but looking inside the directory there are no large files. Please, would you get me a solution for this? I have attached the df results below.

root@server[/tmp]# df -h /tmp
Filesystem Size Used Avail Use% Mounted on
/usr/tmpDSK 485M 460M 0 100% /tmp
root@server[/tmp]# ls -lah /tmp/
total 23K
drwxrwxrwt 3 root root 1.0K May 6 15:12 ./
drwxr-xr-x 28 root root 4.0K Apr 22 01:37 ../
-rwxr--r-- 1 root root 6.0K Jan 26 09:28 aquota.user*
drwx------ 2 root root 12K Jan 20 18:25 lost+found/
lrwxrwxrwx 1 root root 30 May 6 15:12 mysql.sock -> ../../var/lib/mysql/mysql.sock=
root@server[/tmp]#

There are no files in lost+found either.

[Aaron]

Hello John,

Sometimes this happens that at some point the files taking up all the space in /tmp were deleted from the system, but weren’t actually deleted because some process had a lock on them. The deleted files will continue to take up disk space until the process that’s using them exits. You can find open deleted files with lsof:

Code: Select all

lsof | grep tmp

[John Stanley]

Hello,

Thanks, Aaron for your prompt reply. I have checked the command you have provided and could find the below things. Sorry, I don't have enough technical knowledge, please would you let me know how can I completely remove the deleted ones which still exist on the server.

root@server[~]# lsof | grep tmp
queueproc 5133 root 0r REG 7,0 16817 6103 /tmp/sh-thd-1300762559 (deleted)
cpdavd 5170 root 0r REG 7,0 16817 6103 /tmp/sh-thd-1300762559 (deleted)
mysqld 28569 mysql 4u REG 7,0 0 12 /tmp/ib5AISFk (deleted)
mysqld 28569 mysql 5u REG 7,0 0 13 /tmp/ibSyq2LH (deleted)
mysqld 28569 mysql 6u REG 7,0 0 14 /tmp/ib3ZkcS4 (deleted)
mysqld 28569 mysql 7u REG 7,0 0 15 /tmp/ibIOXl0r (deleted)
mysqld 28569 mysql 11u REG 7,0 0 16 /tmp/ibKMMLdP (deleted)
root@server[~]#

[Aaron]

Hello John,

It seems tmp files related to MySQL service holding the disk space. Restarting the service that’s holding onto the deleted files frees up the disk space. I would also recommend creating a larger /tmp directory on CPanel machines as it’s quite small.

[John Stanley]

Hello Aaron,

That's saved me. I have restarted the MySQL service and /tmp is back to normal. Thank you very much for your help.

root@server[/tmp]# /etc/init.d/mysql restart
Shutting down MySQL........................................[ OK ]
Starting MySQL. [ OK ]
root@server[/tmp]# df -h /tmp
Filesystem Size Used Avail Use% Mounted on
/usr/tmpDSK 485M 11M 449M 3% /tmp
root@server[/tmp]#