Computational Cluster Programs
Queue Scripts
Name
queue or its aliases job.queue job.q etc.
Purpose
Prepare a job scheduler command file. Interactive menu interface to native job scheduler commands. Command line interface. Resource file for persistent option values.
Table of Contents
How to Begin
At the shell prompt, enter the name of a queue script, for example job.queue. From the interactive menu interface, enter Build or B to tell queue to build a job scheduler command file.
Input
Queue needs the following information about your job. If you do not specify a value, queue will use application specific default values or values from a .queuerc resource file that you have previously saved.
- Name of your executable program or script or application control file (no default). Some applications require a particular extension, for example ".in", and if this is the case you can enter either myprogram.in or myprogram. In both cases myprogram.in will be used.
- Amount of memory your job requires, in megabytes.
- Amount of time your job requires, in hours.
- For serial jobs which request 2 hours or less time, queue may ask if you want to use the express option.
- For parallel jobs, number of tasks your job uses. Each task will use a processor.
- For array jobs, the low and high index numbers and interval value. Each array jobtask will run as an independent job.
Some applications allow or require you to enter arguments for the program or script, or other application specific information.
Resource Group
If you belong to a resource group which has contributed nodes to the Hoffman2 Cluster, you may also request to limit your job to using only your resource group's nodes. For most jobs you should not use this option. However in these cases it may be necessary:
- If your job requires more time than is available from the shared cluster queues. In this case queue will automatically add this option to the command file that it builds for you. (-u and -t time command line options).
- If your parallel job requires a large number of processors which necessitates a job scheduler reservation. If the number of parallel tasks that you request is larger than the default reserve value (currently 32), the reservation will automatically be made. (-u and -n parallel-tasks command line options).
- If you belong to more than one resource group and you want to direct your job to use a particular group's nodes (-u and -rg resource-group command line options). Default is for the job scheduler to choose which of your resource group's nodes to use (-rg any).
If your resource group has nodes in more than one data center, the job scheduler will choose the data center where your job will run.
Output Files
Two files are generated by queue for all applications: jobname.cmd and jobname.joblog. These files are saved in the output directory. The output directory is usually the same as the directory where your input program or control file is located, unless you have used the -o output-directory option or have specified an output directory in your .queuerc resource file.
The jobname.cmd file is the command file that is submitted to the job scheduler.
The jobname.joblog file is the output file produced by the job scheduler. It contains job information, statistics and error messages. Most applications will append the job's JOB_ID to this file name.
Most applications will also generate a jobname.output file which contains the results of running your program. Most applications will append the job's JOB_ID to this file name.
In addition, each application may generate other files. For example, if you run gamess.q it will generate an .output file and a .dat file and might additionally generate an .irc file.
Interactive Menu Mode
When you enter a queue script name at the shell prompt without entering your executable program or script or application control file name, you will see a menu similar to this:
------------------------------------------------------------ Serial job script Functions (acceptable abbreviations are shown in CAPS) Menu: Display this menu Info: Display help information Build: Build a .cmd file for Serial Submit: Submit a .cmd file for execution STatus: Display the status of jobs for userid SYsstat: Display the status of jobs for the system Hold: Hold a job RELease: Release a job that is held RESet: Reset the priority of a job Cancel: Cancel a job Quit: Exit this script Command: ------------------------------------------------------------
Enter Build or B to begin the process of creating a job scheduler command file for your program. Queue will prompt you for values for the amount of memory, number of hours and possibly other parameters related to the application you are using. It will use any ~/.queuerc file options you may have saved.
After queue has created your cmd file, it will ask whether you want to submit the command file to the job scheduler. If not, you will be returned to the menu and prompted for another command. If so, queue will check that jobname.joblog and jobname.output do not already exist. If one or more output files does exist, you will be asked if you want to overwrite these files and continue with the job submission. Some applications or programs do not allow output files to be overwritten.
You may want to not submit your job at that time and instead view or edit the command file that queue has built. You can later use either queue's Submit menu function, or a native job scheduler command (qsub) to submit your command file.
Menu Interface to Job Scheduler Commands
The following interactive queue menu functions implement native job scheduler commands as follows:- Submit
- qsub jobname.cmd
- STatus
- qstat -u userid
- SYsstat
- qstat
- Hold
- qhold jobid
- RELease
- qrls jobid
- RESet
- qalter -p priority jobid
- Cancel
- qdel jobid
Command Line Mode
Command mode can only be used to build and submit a .cmd file. It will not usually prompt for any option values. It will use application specific defaults which may be overridden by command line options. It will use any ~/.queuerc file options you may have saved. It will not save the .cmd file that it creates unless you specify the -k (keep) option.
It may write messages to stdout unless you have used the -q (quiet) option, in which case it may still write error messages to stdout.
The syntax for command line mode is:
queue [queue-options] control-file [program-arguments]
Example to build, submit and keep a .cmd file for a file myprogram
job.q -k myprogram
It is possible to use interactive menu mode with command line options. When you enter queue at the shell prompt, also enter command line options but do not enter a control file name or program arguments. This will trigger queue's interactive menu mode. If you use any required options (e.g., -t time or -d memory) interactive menu mode will not prompt you for new values for them.
Example of invoking interactive menu mode with command line options:
job.q -o ~/output -m complete
Command Line Options
syntax:
queue [-e] [-f] [-h] [-k] [-q] [-ns] [-u] [-//]
[-p priority] [-d memory] [-t time]
[-n parallel-tasks] [-mt threads]
[-jl lower-index] [-jh higher-index] [-ji interval]
[-o output-directory] [-m messaging]
[-rg resource-group | any] [-dc data-center | any]
control-file [program-arguments]
- -e
- Use express complex. Limited to serial jobs or jobs using the shared parallel environment. Jobs must request 2 hours or less time. Cannot be used with the -u option. There is no advantage to using this option if you belong to a resource group which has contributed nodes. See References for more information.
- -f
- Does not use ~/.queuerc file.
- -h
- Brief syntax list of these options. queue exits after this option.
- -k
- Keeps the .cmd file that is generated.
- -q
- Quiet mode, suppresses job submission messages in command line mode (useful when queue is used in a script). The default is to write messages to stdout.
- -ns
- Build but do not submit command (.cmd) file. Implies the -k (keep) option.
- -u
- Limit your job to using only your resource group's nodes. Cannot be used with the -e option.
- -//
- Uses application.queue.parallel
- -p priority
- Replace priority with the priority of this batch job relative
to others you have submitted. The default is 50. Maximum is 100.
This priority is only used to order your current jobs for selection for execution. It will not affect the priority for selection among all jobs. Within your jobs it is possible for a lower priority job to be selected for execution before a higher priority job, if the jobs will be in different queues and a processor in the lower priority job's queue becomes available before there is a processor available in the queue for the higher priority job.
- -d memory
- Replace memory with amount of memory in megabytes.
- -t time
- Replace time with amount of time in hours. This is a limit; your job will end after this amount of time whether or not it has otherwise completed.
- -n parallel-tasks
- Replace parallel-tasks with the number of processors for parallel jobs.
- -mt threads
- Replace threads with the number of processors for threaded jobs.
- -jl lower-index
- Replace lower-index with the low index for the job array.
- -jh higher-index
- Replace higher-index with the high index for the job array.
- -ji interval
- Replace interval with the job interval for the job array.
- -o output-directory
- Replace output-directory with the directory where the .cmd, .joblog and other output files are created. The default is to use the same directory as the control-file. output-directory must already exist; queue will not create it for you.
- -m messaging
- Replace messaging with one the following:
never no messages are sent. start a message is sent when the job starts complete a message is sent when the job completes always messages are sent when the job starts and completes, and for errors. error a message is sent only if the job aborts, or is rescheduled or suspended. or, with one or more of the qsub command -m values:
b Mail is sent at the beginning of the job. e Mail is sent at the end of the job. a Mail is sent when the job is aborted or rescheduled. s Mail is sent when the job is suspended. n No mail is sent. Messages are sent to your userid and automatically forwarded to the email address you supplied when you applied for your Hoffman2/UCLA Grid account. The default is application specific.
- -rg resource-group | any
- Replace resource-group with the name of a job scheduler access list, or the word any. Does not affect your job unless you also request -u command line option. This option is ignored if you belong to only one resource group. any means the job scheduler will choose one of your resource groups. any is the default.
- -dc data-center | any
- Replace data-center with the name of the data center where you want your job to run, or the word any. Does not affect your job unless you also request -u and your resource group has nodes in more than one data center. any means the job scheduler will choose the data center where your job will run. any is the default.
- control-file
- Replace control-file with the name of your executable program or script or application control-file.
- program-arguments
- Replace program-arguments with any arguments required by your program.
Resource File
If you have some queue options that you always want to be used, you can put them in the file .queuerc under your home directory.
This file contains lines that set options to be used by queue and may also contain comment lines that begin with a # symbol.
The following options can be specified. Note that qqkeepcmd, qqquiet, qqpriority, qqmemory, and qqtime cannot be overridden by their respective command line options. If you set any of these options, interactive menu mode will not prompt you for new values for them.
- qqnotify
- qqodir
- qqkeepcmd
- qqquiet
- qqpriority
- qqmemory
- qqtime
----------------(a sample .queuerc file)-------------------- # My queue options # Messaging option, overridden by "-m messaging" set qqnotify = complete # Output directory, overridden by "-o output-directory" set qqodir = ~/output ------------------------------------------------------------
References
For more information about available queue application scripts and how to use them, please point your browser at:
Software Installed on ATS-Hosted Clusters and its subsequent links for each package or program to "How to run on ATS-Hosted Clusters".
See the following man pages for further information about job scheduler commands:
qacct qalter qconf qdel qhold qhost qlogin qmake qmod qmon qping qquota qrdel qresub qrls qrsh qrstat qrsub qselect qsh qstat qsub qtcsh
See also the following scripts: myjobs, mygroup
January 2011
home
Disability Resources