Computational Cluster Programs

Transferring files to/from an ATS-Hosted Cluster

The preferred way to access an ATS-hosted cluster is through the UCLA Grid Portal at http://grid.ucla.edu. The grid portal makes ATS-hosted clusters accessible from any web browser.

You can use the UCLA Grid Portal to:

• upload files from your local machine to the cluster.
• download files and job output from the cluster to your local machine.
• transfer files between the cluster and other clusters at UCLA.

You can use the UC Grid Portal to:

• transfer files between the cluster and clusters at other campuses of the University of California.

If you are going to use an ATS-hosted cluster from the one of these grid portals, you will not need to know any of the information presented here.

Using scp to Transfer Files

For security reasons, ATS-hosted clusters allow file transfer only with scp or sftp. scp and sftp transfer files using Secure Shell (ssh) protocol, in which data is encrypted during transfer. The use of scp requires that an scp client be run on the machine that you use to initiate the transfer and that it communicate with a server or daeman run on any other machines which participate in the transfer. ATS-hosted clusters, like most Linux and Unix systems run both a client and an appropriate server. Mac OS X systems can also run a server for ssh; Windows systems usually do not run a server.

What does this mean to you? Suppose you want to transfer a file from your local machine to an ATS-hosted cluster and do so from your local machine. Since all ATS-hosted clusters run scp servers, you can do this. However, if you want to initiate the same transfer from the ATS-hosted cluster, you will probably not be able to do so if your local machine is running Windows.

There is an scp client command on Linux/Unix systems and on Macs (use Terminal).

(On Windows, you usually have to install an GUI scp client like the one that comes with SSH Secure Shell.)

The syntax of the scp command is very similar to that of the cp command. For complete scp syntax, enter:

man scp

Here is a symplified scp syntax that accomplishes most transfers:

scp [-r] source target

Normally one of source or target will be on the machine from which you are issuing the scp command and the other will be on some other machine. For a source on the local machine you can specify an absolute or relative file path or that of a directory. Specify -r to transfer a directory. For a target on the local machine, you can specify the directory where the file is to be placed, for example, ., or an absolute or relative path including a new file name, to have the file renamed.

A source or target file on a remote machine is specified identically to that on a local system except that the login id on the remote machine and its address are appended as follows:

loginid@address:path

For example:

scp cluster-loginid@cluster.address:myfile .

will transfer the file named myfile from your home directory on an ATS-hosted cluster to your current directory on the machine from which you issue the command.