SISSA's Computing FAQ: Space disks usage

This page is reserved for the users who have an account on the SISSA main cluster and use:

  • a linux workstations at Santorio or
  • a terminal connection (for example using the ssh command) on the access server ssh.sissa.it .


Your main directory (home) on the SISSA main cluster:

After login (when your username and password are accepted) you have access to your main directory (home):
/u/yoursector/yourusername For example: /u/ap/atena
If you change your working directory, you can return to your main directory using the cd command without any additional option or name.
You can use the pwd command to view which is your main directory (the pwd command prints the path of your working directory, the directory where you are currently in).
The main directories (of the SISSA main cluster) of all users are stored on a central filesystem.
diskusageimage

Your main directory:
  • is regularly saved ( “ backed up ” );


  • has a space limit: You can check your space usage and limit using the quota command;


  • is reachable from outside SISSA using the ssh command (a Secure Shell remote terminal connection) on ssh.sissa.it (for example: ssh ssh.sissa.it );


vpnimage

  • is also reachable from outside SISSA using a VPN connection on your workstation.

Using the SFTP service, the path of your main directory (home) is a bit different. For more information, please read the section on SFTP access: sftp.sissa.it of the Common Access server.



Quota

Using the quota command you can check your space usage on your main directory (or use quota -s to see the values in MB).
For example:

Disk quotas for user yourusername (uid yourUIDnumber):
Filesystem  blocks   quota   limit   grace  files quota limit grace
filesrv-ap.sissa.it:/u/ap
                     64100  950000 1000000         2381     0     0

- The first number of the last line is the space which you are using.

Soft limit

- The second number (in the example 950000) is your soft limit. If your usage exceeds this limit, you will receive an e-mail notification. You have 7 days to reduce your disk space occupation back under the soft limit.
If you do not bring your usage back within the soft limit within 7 days:

  • you will not be able to create or save files…
  • you will not be able to login in graphical mode or …
  • your graphic settings could be damaged and could appear different, so you will not be able to run programs and command activated through the top bar (panel).
Hard limit

- The third number (in the example 1000000) is your hard MAXIMUM limit, if you overtake this limit:

  • you cannot create or save files…
  • you cannot login in graphical mode or …
  • your graphic settings could be damaged and could appear different, so you will not be able to run programs and command activated through the top bar (panel).

When you are over the quota set by the hard limit - message: Disk quota exceeded - you have to clean your main directory from unneeded files. To avoid this situation, we suggest to regularly compress the files which you are not using and to remove any unneeded files. Be careful, “ Haste makes waste ”.



Backup

Your home directory is automatically backed up, and all recent backups are now freely accessible for reading. You can find them in

/u/your_sector/.backup/

Note that the .backup folder begins with a dot, so in a unix system it is hidden. In that folder you can find some other folders, with names in the form

YYYY-MM-DD.HH

Where YYYY is the year, MM is the month, DD is the day of the month, and HH is the hour of the day (24h format) of the backup. Inside the folder you can find the home directories of the users of your sector. Please note that you can't write in those folders.

Usually you can find 6 backups at different hours for the last day, one week of daily backups, and some older backups up to a couple of weeks. Older backups are available via a special helpdesk request and can take some time to get your data.

On a windows client, all you need is to right click on the file, select Properties and then the Previous versions tab. From there you can restore the file to its original content.



Check files and directory sizes

To see how many blocks are used by your directories and files (start first from your main directory):

  • including only normal directories and files: du -sk *
  • including also hidden directories and files: du -sk .[a-z]*
  • and then du -sk .[A-Z]*

Alternatively, to see the space occupied by all subdirectories of your current “position” (please note, there are two “ - ” before max-depth):
du –max-depth=1 -k .

If you have a lot of files and directories, you can add at the end of the command “ | more ”                 for example: du -sk * |more

Have you checked the content of your .Trash directory? Use the command:
ls -laR ~/.Trash |more

Change directory

To “go inside” a sub-directory to check the size of its subsequent sub-directories and files, hidden or not, use the command:
cd subdirectoryname

To return in you main directory use cd

Remove files

Use carefully the rm command to remove your files especially if used in conjunction with the wildcard characters such as (*) and (?) .
Please note:
The asterisk sign “ * ” can be used to substitute zero or more characters, and question mark (?) a single character.
If you are not sure, PLEASE try using the ls command. Some examples:
To remove a file: rm filename
To remove files which start with omega: ls omega           then, ONLY if you are sure: rm omega*
If you are very sure about the files you wish to remove, you can use “ -f ” option to remove all files in a directory without prompting confirmation before removing any files. For example: rm -f omega*



Local SCRATCH area on your linux workstation

I need to use the local scratch area, may I? How? Is it backed up?

Generally each linux workstation should have a local disk area called “ /scratch
You can write your data here.
Put your files in /scratch/yourusername directory and NOT in /scratch. Therefore, firstly you have to create a subdirectory in the /scratch directory using the following command from a linux shell:
mkdir /scratch/yourusername

Do not put important files in scratch areas, because if the local disk is damaged, we will not be able to recover its contents.
Local directories (for example: /scratch /data /tmp) are never saved (“ backed up ”)!!

For I/O intensive applications (i.e. huge data on input/output files), you are strongly suggested to use the local disk of the workstation where your executable is running and NOT your home directory. Since the latter is accessed by the workstation through the network, the reading and writing procedure could slow down terribly and lower the performance of your application.


You can not access to workstations local scratch areas from outside SISSA through the server ssh.sissa.it
vpnimage Please note, there is a NEWS. From the external network you can reach your linux workstation and so reach its /scratch area using the VPN connection.
To use the VPN connection, you should install the VPN client on your laptop, so please read the information on Cisco VPN client installation page.

2011/07/05 23:00

The common SCRATCH area on ssh.sissa.it is reachable from outside SISSA through ssh.sissa.it and sftp.sissa.it

The scratch area on ssh.sissa.it is useful when:

  • you have not enough space on your main directory of the SISSA main cluster due to your quota, and
  • you can not use a VPN connection to reach the local scratch area of your workstation and
  • you need to copy some files to this area (for example from the local scratch area of your workstation) in order to use those files outside the internal SISSA network.

An image with the previous explanation.
This is an example of a typical usage of the ssh.sissa.it scratch area:

  • 1) do not put your files on /scratch of ssh.sissa.it directly; create first a subdirectory with your username using this command: mkdir /scratch/yourusername on ssh.sissa.it
  • 2) then copy your files from the local scratch area of your workstation to the common scratch area /scratch/yourusername on ssh.sissa.it using the scp command.
Only keep copies on /scratch directory of ssh.sissa.it, do not keep original files because files more than 30 days old will be deleted. None of the files in /scratch areas are ever saved!!!!!!!!


This website uses cookies for visitor traffic analysis. By using the website, you agree with storing the cookies on your computer.More information