See File System and Disk Quotas for important information about project directories and where to upload large files to.
Note: Beamline projects should use the Australian Synchrotron's Secure File Transfer process
Command Line :
Use the "scp" (Secure Copy command) for transferring files to MASSIVE. The following will copy a local file in your current directory to the destination folder on MASSIVE. You will be asked for you password before it proceeds.
scp somefile usernamem2.massive.org.au:~/destinationfolder/
The above can be reversed to copy files from MASSIVE. The following will copy a file on MASSIVE into your current directory (specified in the command line using the "."). You will be asked for you password before it proceeds.
scp usernamem2.massive.org.au:~/destinationfolder/ somefile
You can also copy from MASSIVE to another system. From one of the login nodes, following will copy a local file in your current directory to the destination folder on another system (assuming you are also able to ssh login to it).
scp somefile usernameanothersystem.org.au:~/destinationfolder
For more details type "man scp" on MASSIVE. Common commands are:
Copy recursively (-r) all files in a directory:
scp -r somedir usernamem2.massive.org.au:~/destinationfolder/
Note: The above assumes you have ssh'd previously into the server and accepted the host as being correct (e.g. ssh firstname.lastname@example.org)
Use rsync to synchronise file system and to transfer large amounts of files, with the ability to stop and restart the file transfers. rsync will replicate all files in a folder from one spot to another. It first analyses both files systems to find the difference and then transfers only the changes.
A typical command to synchronise files from MASSIVE to a local folder is:
rsync -auv -e ssh somedir usernamem2.massive.org.au:~/destinationfolder/
rsync is very powerful and has many options to help transfer data. For example it can delete unwanted files (--delete), compress data before transfer (-z) or can you let you see what command options might do without actualling executing them (--dry-run). For more info on rsync try "man rsync".
While Cyberduck is our recommended client for MacOS X other clients (the command line client above, or FileZilla) may be faster. If you choose to use FileZilla, be aware that some download links try to get you to install additional (unnecessary) software. These download links should be safe from additional installs.
Using GUI client Cyberduck See: Client Software Requirements
While WinSCP is our recommended client for Windows other clients (the cygwin command line client, or FileZilla) may be faster. If you choose to use FileZilla, be aware that some download links try to get you to install additional (unnecessary) software. These download links should be safe from additional installs.
Using WinSCP See: Client Software Requirements
The steps are similar to the above but WinSCP hides the commands behind a GUI. You are presented with a similar interface to Windows Explorer or a FTP client.
1) The first step is to start WinSCP and save a connection with m2.massive.org.au and your username.
2) Next login using the above saved connection
3) Start transferring files, left is your local file system and right is the MASSIVE file system. Be careful as it is very easy to drag and drop files to new locations, it is also easy to make mistakes.
Mount Using "Places"
From the MASSIVE desktop you can create a mount of remote file systems using "places", such as between m1 and m2. Below is an example of mounting m1's file system on m2.
From the top Menu: Places > Connect to Server.
Select Service type: "SSH", Server "m1.massive.org.au", Folder: "/home/<your username>"" and User Name: <your username> (replacing <your username>)
rclone allows you to transfer files on the command line to cloud services such as Google Drive
module load rclone
Config a drive from the desktop (the first time you need a webbrowser to get a token)