can't open or create /var/run/crond.pid: No space left on device

This error message told us that there is no disk space left on /var

When we run "df -h" on the system we get:

[root@A5AFETTASB pats]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/cciss/c0d0p3     3.0G  1.7G  1.2G  59% /
/dev/cciss/c0d0p1      99M   15M   80M  16% /boot
/dev/cciss/c0d0p6      24G  5.9G   17G  27% /opt
none                  1.3G     0  1.3G   0% /dev/shm
/dev/cciss/c0d0p5     2.0G  1.1G  790M  59% /var

Seems like only 59% disk space is used but that is not true. As the difference here is MB of space versus the almighty inode. So if we try using "df -i", we get:

[root@A5AFETTASB pats]# df -i
Filesystem            Inodes   IUsed   IFree IUse% Mounted on
/dev/cciss/c0d0p3     393600   95644  297956   25% /
/dev/cciss/c0d0p1      26104      40   26064    1% /boot
/dev/cciss/c0d0p6    3123232   12620 3110612    1% /opt
none                  322278       1  322277    1% /dev/shm
/dev/cciss/c0d0p5     262144  262144       0  100% /var

Further investigation showed, there are huge number of files created under /var/spool/clientmqueue that caused the problem:

[root@A5AFETTASB pats]# du -h /var/spool/*
4.0K    /var/spool/at/spool
8.0K    /var/spool/at
1.1G    /var/spool/clientmqueue
12K     /var/spool/cron
4.0K    /var/spool/cups/tmp
8.0K    /var/spool/cups
4.0K    /var/spool/lpd
8.0K    /var/spool/mail
4.0K    /var/spool/mqueue
4.0K    /var/spool/repackage
4.0K    /var/spool/rwho
4.0K    /var/spool/samba
4.0K    /var/spool/up2date
4.0K    /var/spool/vbox

If you don't care about the content you can simply delete the files, but you should ask yourself why you have tons of messages in that directory.

# rm -rf /var/spool/cilentmqueue
# mkdir /var/spool/cilentmqueue
# chown smmsp:smmsp /var/spool/cilentmqueue

If the emails are the result of crontab, below will explain how to disable it:
http://www.cyberciti.biz/faq/disable-the-mail-alert-by-crontab-command/


In this case the problem caused by sendmail. If you don't use it we can just disable the service


# svcs sendmail
STATE          STIME    FMRI
disabled       Sep_15   svc:/network/smtp:sendmail

Or you should probably enable it just to make sure that at and cron job output is delivered.  You can configure sendmail such that it doesn't accept remote SMTP connections.