Computational Cluster Programs

Express queue

The purpose of the express queue is to increase the use of shared resources for users and the utilization of the cluster. Although no active users are excluded from using the express queue, it is general campus users which benefit by being able to concurrently run an increased number of jobs. A user belongs to the campus user group if his/her sponsor has not contributed nodes to the Hoffman2 Cluster.

Characteristics

  • Using the express queue does not consume your resource group's slot quota.
  • The express queue is limited to serial jobs, or array jobs, or shared memory parallel jobs that run on a single node and use the shared parallel environment.
  • Due to the short time limit, these jobs make better use of the back-fill feature of the job scheduler and usually start running within 5-10 minutes.
  • Parallel multi-node jobs will not be able to use this queue.
  • You cannot use this queue to run on your resource group's own nodes (highp).
How to request the express queue
  1. Make sure that each of your jobs will finish in 2 hours. Jobs submitted to this queue will be unconditionally terminated after 2 hours.

  2. To submit jobs to this queue, add "-l express" in addition to other parameters you typically use. Request time of no more than 2 hours.

  3. To direct an existing pending job to this queue, use
    qalter -l express JOB_ID
    where JOB_ID is its job id as shown by qstat -u $USER or the myjobs script.
Sample job submission scenarios

For a simple serial job requesting one hour

qsub -l express,h_rt=1:00:00 myjob.cmd

For a simple array job requesting one hour per jobtask

qsub -l express,h_rt=1:00:00 -t 1:1000 myjobarray.cmd

If you do not request time, your job will run for maximum 2 hours set by the queue and will then be terminated.

January 2011