Data Storage Space
Groups beginning new funded projects are advised to purchase their
own disk space for analysis and storage of data. You can get disk space
for data in one of two ways: buy space on the Marintos Center storage
servers, OR buy workstations with large terabyte-sized
disks and have the IT group partition and export them.
Storage on the Marintos Center storage services cost $26.66/TB/month
and includes a weekly disk-to-disk mirror backup.
Or when you buy your workstation(s), you can
make sure to buy lots of disk space internal to the box to use.
This can be a bit cheaper than the storage cluster but has several
drawbacks. You will need to supply your own backup (usually by buying
twice the disk you need and setting up weekly mirror jobs).
Lots of disk in your workstation will make your office much hotter
and noisier and may trip your power breakers. Performance will be
horrible or even unusable from jobs you run on the compute cluster.
Stability will be as stable as your workstation (no where near the
stability of the compute cluster).
All data storage spaces will be in Linux under the /space and /cluster
name spaces. Anything under /cluster is on the storage cluster. Some
things in /space have been moved to /cluster and just point there now.
Checking Disk Free/Usage
The normal command in UNIX for checking how much space is free on a disk
is the 'df' command. This works fine for the /space volume tree where
we place all the workstations disks and some of the older storage servers.
Temporary Scratch Areas
Limited areas of temporary disk space are available to all for data
analysis. Users must remove data from these disks when
analyses are complete. These spaces are cleaned weekly and are not
backed up by the Martinos Center.
Any data over 7 days old left on the day-of-the-week spaces will be removed
early in the morning of the named day (that is, /space/monday is
cleaned on Monday morning, etc).
Please run 'clusterquota scratch' to see how much space is available (shared by all day-of-week dirs) before you copy data to any of these partitions.
All other /space and /cluster filesystems are project specific and
can be used only with permission of the owner of that space.
Home directories are NOT to be used for image data. They are for
code, documents, summarized results, etc. This covers anything under
the /homes namespace.
Home directory disk space on central UNIX/linux based systems is
limited to a 2GB quota for each user. To determine how much is currently being
used, run the quota command. To figure out what files/dirs are using
all the space in your home directory, type the following command at a UNIX/linux command line from
your home directory
du -sk .??* * | sort -n
The resulting numbers will be in KiloBytes. The sort will order the
results so the biggest space hogs are last.
See the Backups page for information
on storage volume backups.
See the Understanding Group Permissions in UNIX
page for information on general file permissions in UNIX and how
to set up group areas for shared write access by the whole group.
See the Security page for information
about data file security issues.
It is important to note that NO NETWORK DISK SPACE in the Martinos
center is safe for HIPAA sensitive files.