Managing Your Disk Usage

The Main Problem - Backups

All users' home directories are backed up 2-3 times daily, and these ``snapshots'' are kept on a backup server which has finite disk capacity. The larger the average size of a snapshot, the fewer backups can be kept. Hance if the total disk usage increases too much, it may be necessary to take backups less regularly or keep them for a shorter time in order to fit them all on the backup server. This is the main reason why we have to set limits on disk usage.

It is also possible for the disk partition containing your home directory to fill up completely, leading to serious problems including lost data and email. To find out how full it is, type ``df ~'', which will give output like this. Note that the figures apply to the whole /users1 filesystem which contains almost all Statslab users' home directories.

Filesystem           1K-blocks      Used Available Use% Mounted on
spaces:/users1/admin/eva
                     456701760 391439232  41689216  91% /home/eva

The alert may notice that the ``Used'' and ``Available'' figures do not add up to the total in the ``1k-blocks'' column. This is because Unix slows down considerably when partitions are more than about 90% full, so a partition which is actually 90% full is reported as being 100% full, and ordinary users are not allowed to write to it. (The system itself is, hence it is possible for the reported disk usage to go above 100%.)

Measuring Your Disk Usage

To see how much space your home directory is taking up, go there (type ``cd'') and then type ``du -sk .'' (the full stop is part of the command). This will tell you the total number of kilobytes you are using. If it's over 5 million, you are using more than 5GB of disk space and should try to cut down. (A few users have permission to exceed this limit as they need more space for their work.)

To find out where your large files and directories are, run the following command from your home directory:

du -sk * | sort -n

which will produce a sorted list of your files and directories in increasing order of size. To include ``hidden'' files and directories in the sorted list, use the command

du -sk * .??* | sort -n

Of course, you can also run either of these commands in a subdirectory to find out which of its subdirectories are taking up most space.

How To Cut Down

Easy Steps Everyone Can Take

A significant amount of disk space is wasted in mostly useless computer-generated files. While you should not delete these files if you don't understand their purpose, there are two hidden disk space sinks you should be aware of.

The .Trash directory. When you delete files by dragging them to the wastebasket icon instead of using the rm command, the files are not deleted but only moved to your .Trash subdirectory. To delete them properly, you must right-click on the wastebasket and select ``Empty Trash'' on a regular basis.
If I have set up spam filtering for you, the spam it detects is filtered into a "spam" folder, which may be in your mail or Mail directory depending on your setup. (While it is possible to delete all incoming spam, it is not recommended in case a genuine email is mislabelled as spam.) Make sure that this folder is emptied regularly. If you haven't checked it for a while, I suggest you look at it with pine or your normal mail reader (in the same way as you would change to any other folder) to check for misclassified emails. Then close the mail reader and delete the spam folder with the rm command.

Ex-Users, Please Transfer Your Files

If you are no longer at the Statslab, but have been given permission to keep your account, please transfer your files and email to your account at your current job if at all possible. We do not have enough space for everyone who has ever been here. If you manage to clear out your Statslab account completely, so that you no longer need it, please let me know so that I can close it. I realise that some ex-users need to keep their websites, but your other files and your email should be moved away. (We can forward email to your new address even after your account is closed.)

Similarly, if you have been allowed to keep your Statslab account because you are a regular visitor to the Statslab or because you are collaborating with Statslab staff, please give some consideration to whether any of your large files could be stored in your home university account rather than your Statslab account.

Delete or archive large files

To reduce the space taken by your home directory, delete any large files you no longer need. If you have large files and directories which you need to keep around for reference but rarely refer to, you can burn them to a recordable CD or DVD. When you insert a blank CD/DVD, the graphical CD creator will launch automatically.

Large files which do not need to be backed up can be kept in /scratch/missmuffet, which is NFS-mounted like your home directory and therefore accessible from any Statslab Linux desktop, or in /usr/local/work on your office PC if you are happy with only being able to access those files from that PC. You are advised to create a subdirectory (e.g. ``mkdir /scratch/missmuffet/myusername'') and keep your files in there so they don't get mixed up with everyone else's.