Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
disk:disksusage [2014/10/07 13:13] delise |
disk:disksusage [2023/05/19 10:10] (current) zanello |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| ====== SISSA's Computing FAQ: Space disks usage====== | ====== SISSA's Computing FAQ: Space disks usage====== | ||
| - | **<color teal>This page is reserved for the users who have an account on the SISSA main cluster and use:</color>**\\ | + | This page is reserved to the users who have an account on SISSA and use: |
| - | * **<color teal>a linux workstations at Santorio </color> **or | + | |
| - | *** <color teal>a terminal connection</color> **(for example using the ssh command)** <color teal> on the access server </color>//<color teal> ssh.sissa.it </color>//** .\\ | + | |
| + | * a **linux workstation** at Santorio; | ||
| + | * an **ssh connection** on a linux workstation; | ||
| + | * an **sftp connection** on **sftp.sissa.it**; | ||
| + | * an [[services:computing:hpc|HPC access]]. | ||
| \\ | \\ | ||
| - | ===== YOUR MAIN DIRECTORY (home) ===== | + | ===== HOME directory ===== |
| - | ==== Your main directory (home) on the SISSA main cluster: ==== | + | ==== Your HOME directory in SISSA: ==== |
| - | After login (when your username and password are accepted) you have access to your main directory (home): \\ | + | After login (when your username and password are accepted) you have access to your HOME directory: \\ |
| - | **/u///yoursector/yourusername//** For example:// /u/ap/atena // \\ | + | **/u///username_initial/username//** (e.g., ///u/a/atena//) \\ |
| - | If you change your working directory, you can return to your main directory using the **cd** command without any additional option or name.\\ | + | |
| - | You can use the **pwd** command to view which is your main directory (the **pwd** command prints the path of your working directory, the directory where you are currently in).\\ | + | |
| - | The main directories (of the SISSA main cluster) of all users are stored on a central filesystem. \\ | + | |
| - | {{ :disk:diskusageimage-a.jpg?270x150|diskusageimage}}\\ | + | |
| - | == Your main directory: == | + | If you change your working directory, you can return to your HOME directory using the **cd** command without any additional option or name.\\ |
| + | You can use the **pwd** command to view which is your HOME directory (the **pwd** command prints the path of your working directory, the directory where you are currently in).\\ | ||
| + | |||
| + | |||
| + | == Your HOME directory: == | ||
| * is regularly saved ( " backed up " ); | * is regularly saved ( " backed up " ); | ||
| + | * **has a space limit**: each user has 50GB of disk space; | ||
| + | * is reachable from outside SISSA using the **sftp** protocol on **sftp.sissa.it** (e.g.: ''ssh //username//@sftp.sissa.it''); | ||
| + | * is also reachable from outside SISSA using a [[services:network:internal:vpnclient|VPN connection]] on your workstation. Your workstation name is: <hostname>.<area_name>.sissa.it (e.g.: host.phys.sissa.it or host.maths.sissa.it); | ||
| + | * is reachable from inside SISSA using a CIFS/SAMBA connection: **\\nas-users.sissa.it\USERNAME** or **[[smb://]]nas-users.sissa.it///USERNAME//** | ||
| \\ | \\ | ||
| - | * **has a space limit**: You can check your space usage and limit using the **quota** command; | ||
| - | \\ | ||
| - | * is reachable from outside SISSA using the **ssh** command (a Secure Shell remote terminal connection) on **//ssh.sissa.it//** (for example: // ssh //ssh.sissa.it//// ); | ||
| - | \\ | ||
| - | |||
| - | {{ :disk:vpn4.jpg?270x150|vpnimage}}\\ | ||
| - | |||
| - | * is also reachable from outside SISSA using a [[services:network:vpnclient|VPN connection]] on your workstation. | ||
| - | |||
| ---- | ---- | ||
| - | Using the **//SFTP service//**, the path of your main directory (home) is a bit different. For more information, please read the section on [[on-line_resources:faq:server-ssh-sftp#SFTP access: sftp.sissa.it|SFTP access: sftp.sissa.it]] of the [[on-line_resources:faq:server-ssh-sftp|Common Access server]]. | + | Using the **//SFTP service//**, the path of your HOME directory is a bit different. For more information, please read the section on [[on-line_resources:faq:server-ssh-sftp#SFTP access: sftp.sissa.it|SFTP access: sftp.sissa.it]] of the [[on-line_resources:faq:server-ssh-sftp|Common Access server]]. |
| - | ---- | ||
| - | ---- | ||
| - | ==== Quota ==== | + | ==== Backup (snapshot) ==== |
| - | Using the **quota** command you can check your space usage on your main directory (or use **quota -s** to see the values in MB).\\ | + | |
| - | For example:\\ | + | |
| - | Disk quotas for user yourusername (uid yourUIDnumber): | + | |
| - | Filesystem blocks quota limit grace files quota limit grace | + | |
| - | filesrv-ap.sissa.it:/u/ap | + | |
| - | 64100 950000 1000000 2381 0 0 | + | |
| - | - The //first number// of the last line is the space which you are using.\\ | + | Your HOME directory is automatically backed up, and all recent backups are now freely accessible for reading. |
| - | == Soft limit == | + | |
| - | - The //second number// (in the example //950000//) is your soft limit. If your usage exceeds this limit, you will receive an e-mail notification. You have 7 days to reduce your disk space occupation back under the soft limit. \\ | + | |
| - | If you do not bring your usage back within the soft limit within 7 days:\\ | + | |
| - | * you will not be able to create or save files... | + | |
| - | * you will not be able to login in graphical mode or ... | + | |
| - | * your graphic settings could be damaged and could appear different, so you will not be able to run programs and command activated through the top bar (panel). | + | |
| - | == Hard limit == | + | To look for a previous version of a file, simply go via terminal or by forcing the location to the file manager to the subdirectory .snapshot. |
| - | - The //third number// (in the example //1000000//) is your hard MAXIMUM limit, if you overtake this limit: | + | |
| - | * you cannot create or save files... | + | |
| - | * you cannot login in graphical mode or ... | + | |
| - | * your graphic settings could be damaged and could appear different, so you will not be able to run programs and command activated through the top bar (panel). | + | |
| + | You can use the command | ||
| + | <code> | ||
| + | ls .snapshot | ||
| + | </code> | ||
| - | When you are over the quota set by the hard limit - message: **Disk quota exceeded** - you have to clean your main directory from unneeded files. | + | Note that the .snapshot is completely hidden, so you can't see even if you enable hidden files. But if you change to that directory, you can do and you'll find the previous versions of the parent directory. From a file manager typing CTRL+L and adding /.snapshot to the current location might help. |
| - | To avoid this situation, we suggest to regularly compress the files which you are not using and to remove any unneeded files. Be careful, " Haste makes waste ". | + | |
| - | + | ||
| - | ---- | + | |
| - | ---- | + | |
| - | ==== Backup ==== | + | |
| - | + | ||
| - | Your home directory is automatically backed up, and all recent backups are now freely accessible for reading. You can find them in | + | |
| - | + | ||
| - | /u/your_sector/.backup/ | + | |
| - | + | ||
| - | Note that the .backup folder begins with a dot, so in a unix system it is hidden. In that folder you can find some other folders, with names in the form | + | |
| - | + | ||
| - | YYYY-MM-DD.HH | + | |
| - | + | ||
| - | Where YYYY is the year, MM is the month, DD is the day of the month, and HH is the hour of the day (24h format) of the backup. Inside the folder you can find the home directories of the users of your sector. Please note that you can't write in those folders. | + | |
| Usually you can find 6 backups at different hours for the last day, one week of daily backups, and some older backups up to a couple of weeks. Older backups are available via a special helpdesk request and can take some time to get your data. | Usually you can find 6 backups at different hours for the last day, one week of daily backups, and some older backups up to a couple of weeks. Older backups are available via a special helpdesk request and can take some time to get your data. | ||
| - | On a windows client, all you need is to right click on the file, select **Properties** and then the **Previous versions** tab. From there you can restore the file to its original content. | + | On a Windows client, all you need is to right click on the file, select **Properties** and then the **Previous versions** tab. From there you can restore the file to its original content. |
| ---- | ---- | ||
| ---- | ---- | ||
| - | ===== Useful COMMANDS to CLEAN YOUR AREA ===== | + | ===== Useful commands to CLEAN your area ===== |
| ==== Check files and directory sizes ==== | ==== Check files and directory sizes ==== | ||
| - | To see how many blocks are used by your directories and files (start first from your main directory): | + | To see how many blocks are used by your directories and files (start first from your HOME directory): |
| * including only normal directories and files: **du -sk * ** | * including only normal directories and files: **du -sk * ** | ||
| * including also hidden directories and files: **du -sk .[a-z]* ** | * including also hidden directories and files: **du -sk .[a-z]* ** | ||
| Line 104: | Line 73: | ||
| **cd //subdirectoryname//** \\ | **cd //subdirectoryname//** \\ | ||
| - | To return in you main directory use **cd **\\ | + | To return in you HOME directory use **cd **\\ |
| \\ | \\ | ||
| === Remove files === | === Remove files === | ||
| Line 113: | Line 82: | ||
| To remove a file: **rm //filename//**\\ | To remove a file: **rm //filename//**\\ | ||
| To remove files which start with //omega//:// ls omega//<html>   </html> then, ONLY if you are sure: //rm omega*// \\ | To remove files which start with //omega//:// ls omega//<html>   </html> then, ONLY if you are sure: //rm omega*// \\ | ||
| - | If you are very sure about the files you wish to remove, you can use " **-f** " option to remove all files in a directory without prompting confirmation before removing any files. For example: //rm -f omega*// | + | If you are very sure about the files you wish to remove, you can use " **-f** " option to remove all files in a directory without prompting confirmation before removing any files. E.g.: //rm -f omega*// |
| ---- | ---- | ||
| Line 119: | Line 88: | ||
| - | ===== SCRATCH AREAS ===== | + | ===== SCRATCH areas ===== |
| Line 128: | Line 97: | ||
| - | ===== SCRATCH area on ssh.sissa.it is reachable from outside SISSA ===== | ||
| - | The common SCRATCH area on ssh.sissa.it is reachable from outside SISSA through [[on-line_resources:faq:server-ssh-sftp|ssh.sissa.it and sftp.sissa.it ]]\\ | ||
| - | |||
| - | The scratch area on ssh.sissa.it is useful when: | ||
| - | * you have not enough space on your main directory of the SISSA main cluster due to your quota, **and** | ||
| - | * you can not use a VPN connection to reach the local scratch area of your workstation **and** | ||
| - | * you need to copy some files to this area (for example from the local scratch area of your workstation) in order to use those files outside the internal SISSA network. | ||
| - | |||
| - | {{ :disk:scratchshannonusage-a.jpg?270x150|An image with the previous explanation.}}\\ | ||
| - | This is an example of a typical usage of the ssh.sissa.it scratch area:\\ | ||
| - | * **1)** do not put your files on /scratch of //ssh.sissa.it// directly; create first a subdirectory with your username using this command: **mkdir /scratch///yourusername//** on ssh.sissa.it | ||
| - | * **2)** then copy your files from the local scratch area of your workstation to the common scratch area /scratch///yourusername// on //ssh.sissa.it// using the **scp** command.\\ | ||
| - | <note i> | ||
| - | Only keep copies on /scratch directory of ssh.sissa.it, **do not keep original files because files more than 30 days old will be deleted. None of the files in /scratch areas are ever saved**!!!!!!!! | ||
| - | </note> | ||
| \\ | \\ | ||