Gaussian 09, Revision A.02 is now available on the the Hoffman2 Cluster. It supports parallel distributed memory execution (Linda-parallelism) and shared memory execution (SMP-parallelism). GaussView 5 is also available.

Gaussian provides state-of-the-art capabilities for electronic structure modeling. GaussView is a graphical interface available for Gaussian. With GaussView, you can import or build the molecular structures that interest you, set up, launch, monitor and control Gaussian calculations, retrieve, view and visualize the results, GaussView 5 provides comprehensive support for importing and working with structures from PDB (protein data base) files.

See How to Run Gaussian on ATS-Hosted Clusters There are new queue scripts to invoke Gaussian 09.

See also How to Run GaussView on ATS-Hosted Clusters. There is new information about GaussView and how to run Gaussian from within GaussView.

Gaussian 09 and GaussView 5 documentation is available at http://www.gaussian.com/ Gaussian 09 Release Notes are available at http://www.gaussian.com/g_tech/rel_notes.pdf

Quantum ESPRESSO v 4.1.1 available

Quntum ESPRESSO v 4.1.1 is now available on the Hoffman2 Cluster for SGE batch execution. Quantum ESPRESSO is an integrated suite of computer codes for electronic-structure calculations and materials modeling at the nanoscale. It is based on density-functional theory, plane waves, and pseudopotentials (both norm-conserving and ultrasoft).

Please see How to Run Quantum ESPRESSO on ATS-Hosted Clusters for detailed information.

The Quantum ESPRESSSO documentaion is available on-line at http://www.quantum-espresso.org/user_guide/user_guide.html

Stata 11 available

The Stata 11 is now available on the Hoffman2 Cluster for both SGE batch and interactive applications. Stata is a complete, integrated statistical package that provides a broad suite of statistical capabilities, complete data-management facilities, and publication-quality graphics.

Please see How to Run Stata on ATS-Hosted Clusters for detailed information.

Stata manuals are not available online. They may be borrowed from ATS Statistical Consulting http://www.ats.ucla.edu/stat/books/#Stata

LAMMPS version 7Sep09 is now available

LAMMPS version 7Sep09 is now installed on the Hoffman2 cluster in the /u/local/apps/lammps/current directory. The software can be run as a parallel application.

LAMMPS, Large-scale Atomic/Molecular Massively Parallel Simulator, is a classical molecular dynamics simulation code designed to run efficiently on parallel computers. It was developed at Sandia National Laboratories, a US Department of Energy facility, with funding from the DOE. The developers of LAMMPS are Steve Plimpton, Paul Crozier, and Aidan Thompson who can be contacted at sjplimp, pscrozi, athomps at sandia.gov.

LAMMPS is an open-source code, distributed freely under the terms of the GNU Public License (GPL).

See How to Run LAMMPS on ATS-Hosted Clusters

LAMMPS documentation is available at http://lammps.sandia.gov/doc/Manual.html

Intel compiler upgrade

The default Intel c and fortran compilers on Hoffman2 are now version 11.1

For more information, see Languages and Compilers and the documentation links on that page.

Priority for your jobs

Note: This new job scheduling feature is available to people who are in a sponsored group which has contributed cores to the Hoffman2 Cluster. If you are not in a sponsored group, the new highp complex will not work for you; using it will cause qsub to reject your job.

Now you can tell the Sun Grid Engine to run your job on your own group's allocated cores. This modification, which adds "urgency" to the queue definitions, is designed to ensure that your group's allocated cores are given back to your group within 24 hours.

To use your group's own allocated cores, specify the new SGE complex highp either in your SGE command file or as an argument to qsub. Examples:

#$ -l highp,h_data=1024M,h_rt=24:00:00

or,

qsub -l highp [other arguments]

Currently both the new highp queues and the former queues are available to your jobs. After the old queues are disabled, if you don't specify the highp complex your job will be sent to the shared cluster nodes which have a maximum time limit of 24 hours.

How to decide whether to run your job on your group's own nodes or the shared cluster?

If your job requires more than 24 hours, you must specify the highp complex.

The qquota command will tell you how many cores are currently in use by your group. Example of qquota output:
```
resource quota  rule limit   filter
-----------------------------------------------------------------
queue_limits/3  slots=67/72   users @groupname queues *d_4g_b.q
rulset1/3       slots=85/144  users @groupname hosts @msa-amd_04g
```
In the above example, there are 5 slots available in the group's own allocation and 59 available in the group's shared cluster allocation -- it does not mean there are any cores free. So in case your job specifies highp and requests 72 or fewer cores, the job will be released to run on your group's allocated nodes once your priority becomes first within your group. In case you do not specify highp, your job will go to the shared queue (max. 144 cores in this example) and may run or wait for other nodes to be free before it starts.
If you rebuild your SGE command files with a queue script (e.g., job.q, mpi.q) then if highp is required, the queue script will supply it. The queue scripts will ask you if you want to use your own group's cores for jobs requesting 24 hours or less. For people who use the queue scripts in non-interactive mode, there is a new queue script command line option -u which you can use to request urgency (highp).

Back-filling

There is a back-filling feature in SGE which is enabled. In order for back-filling to work, you should specify your best guess of the time required for your job and not take the default of 24 hours or 14 days.

If you have any questions or problems, please send email to ATS High Performance Computing at atshpc@ucla.edu

GROMACS v4.0.5 installed

GROMACS version 4.0.5 is now installed on the Hoffman2 cluster in the /u/local/apps/gromacs/4.0.5 directory. The software can be run as a serial and/or parallel application.

GROMACS is an engine to perform molecular dynamics simulations and energy minimization. GROMACS is free software. The entire GROMACS package is available under the GNU General Public License.

See How to Run GROMACS on ATS-Hosted Clusters

GROMACS documentation is available at http://www.gromacs.org/Documentation/Manual

R version 2.9.1 is now available

R has been upgraded to version 2.9.1 and the following packages have been installed: MCMCpack, Rserve, abind, ape, coda, degreenet, ergm, gee, igraph, latentnet, network, networksis, rgl, scatterplot3d, shapes, sna, and statnet.

To see all packages which are installed, at the R command prompt, issue the command

> library()

R is GNU S, a freely available language and environment for statistical computing and graphics which provides a wide variety of statistical and graphical techniques: linear and nonlinear modelling, statistical tests, time series analysis, classification, clustering, etc. Please consult the R project homepage http://www.r-project.org/ for further information and documentation.

See How to Run R on ATS-Hosted Clusters

MATLAB 7.7 available

MATLAB has been upgraded to version 7.7 (Release R2008b).

MATLAB is a high-level language and interactive environment that enables you to perform computationally intensive tasks faster than with traditional programming languages such as C, C++, and Fortran.

See How to Run MATLAB on ATS-Hosted Clusters

The Hoffman2 Cluster has licenses for some MATLAB toolboxes. Please see Matlab Toolboxes Available on the Hoffman2 Cluster for a list of available toolboxes.

More MATLAB licenses and more toolboxes are available from the San Diego Super Computer OnDemand service. This is available to you through the UCLA Grid. It is also accessible with the new gapp command at the Hoffman2 Cluster shell prompt.

MATLAB documentation is available on the web at: http://www.mathworks.com/access/helpdesk/help/helpdesk.html

Power Outage Saturday Sept. 12th

All power was lost to the IDRE and engineering buildings Saturday, September 12th, at about 10:30 AM. All nodes in the IDRE Data Center that were on utility power immediately went down. Power was restored 3 hours later, and by 2:30 PM all nodes and filesystems were back online.

Please send any problem reports to atshpc@ucla.edu

Hoffman2 Cluster major queue reconfiguration

All users must rebuild their current SGE command files using the appropriate queue scripts (for example: mpi.q, job.q, gaussian.q). This is a major reconfiguration of the queue structure due to increased compute resources.

The most important reasons are:

Campus group users can only request 1GB per core. If your serial job needs more than 1GB, you must add the following SGE active comment to your SGE command file:
#$ -pe shared 2
There are new parallel environments. We are deprecating some of the old parallel environments and most of the current queues.
We have a new interactive queue with a forced complex to make more interactive sessions available to you. In order to get this queue, campus group users must say:
qrsh -l interactive
or
qrsh -l i

Sponsored group users do not need to use "-l i" on their qrsh commands because it will use their sponsored group queues which do not have the forced complex.
New nodes are available for CNSI group and some sponsored group users. If you are eligible to use the new nodes, you and your sponsor will receive more information.

We also have enabled qsub checking which will tell you immediately when you submit your job if it will never run, and not accept your job. This may happen because the resources it requests no longer exist. If you do not rebuild your SGE command file, you can expect to see this message:

Unable to run job: error: no suitable queues.

If you find any problems, please send email to atshpc@ucla.edu

OpenMPI update

OpenMPI has been upgraded to 1.3.3 and is available in the /u/local/mpi/openmpi/current directory.

The Open MPI Project is an open source MPI-2 implementation that is developed and maintained by a consortium of academic, research, and industry partners. Open MPI is therefore able to combine the expertise, technologies, and resources from all across the High Performance Computing community in order to build the best MPI library available. Open MPI offers advantages for system and software vendors, application developers and computer science researchers.

For further details please refer to: http://www.open-mpi.org/

As per release documentation, users should not be required to recompile their codes.

PLINK installation

PLINK version 1.06 is now installed on the Hoffman2 cluster in the /u/local/apps/plink/1.06 directory.

PLINK is a free, open-source whole genome association analysis toolset, designed to perform a range of basic, large-scale analyses in a computationally efficient manner.

See How to Run PLINK on ATS-Hosted Clusters

PLINK v1.06 PDF documentation is available from http://pngu.mgh.harvard.edu/~purcell/plink/pdf.shtml

Free Abaqus Student Edition download

Abaqus is finite analysis modeling, visualization, and process automation software. Abaqus SE (Student Edition) is available to academic students, professors and researchers from the vendor as a free download to your Windows XP Professional SP2, Windows XP Home Edition SP2, or Windows Vista personal computer.

You must register with the vendor, Dassault Systems, in order to be eligible for the free download (a "DS passport").

free download at http://campus.3ds.com/simulia/freese

Abaqus SE includes the core Abaqus products: Abaqus/Standard, Abaqus/Explicit, and Abaqus/CAE. The maximum model size is set to 1000 nodes for both analysis and postprocessing. Access to features requiring compilers (user subroutines, Abaqus Make, C++ ODB API), parallel execution, or add-on products has not been included. Replay and journal files are not available for Abaqus/CAE.

More information at http://www.simulia.com/academics/student.html

Hoffman2 System Upgrade

During last couple of days the cluster went through a major software as well as key hardware upgrade. The login systems and SGE server have been upgraded by brand new and powerful servers. The modifications were also done in order to allow this cluster to expend beyond the physical limits of current IDRE data center. The cluster is now fully operational and back in service.

Although all of the abovementioned upgrades are transparent to user, yet software stacks like MPI, compiler and OS/run time libraries are new. Therefore you are advised to recompile user codes in order to avoid any unforeseen problems during run time. The codes provided from system side are already been recompiled with the new software stack.

Hoffman2 Cluster Outage May 27 2009

The Hoffman2 cluster will be down for system and storage upgrades from 8:00AM - 8:00PM on Wednesday, May 27, 2009. You will be unable to login or access the cluster during this period. The cluster will be upgraded to Centos 5.3 with many additional application updates, a faster Grid Engine scheduler, and more.

The maximum time allotted to cluster queues will decrease daily as the outage date nears to prevent unexpected job termination. All jobs still running under the Sun Grid Engine, for example jobs you submitted through UCLA Grid or with a command-line queue script, will be terminated when the cluster is taken down. You will need to resubmit your jobs when the cluster comes back.

Spring 2009 Parallel Computing Classes

We are pleased to announce the IDRE high performance computing workshops for this quarter. There are three classes:

Introduction to Parallel Computing
Monday, Apr 27, 10am-noon
5628 Math Sciences (Visualization Portal)
Parallel Programming using MPI
Monday, May 4, 10-noon
5628 Math Sciences (Visualization Portal)
Parallel Computing Lab
Monday, May 11, 1-3pm
320B Powell (CLICC Classroom B)

To sign up, use: http://idre.ucla.edu or http://www.ats.ucla.edu/cfapps/events/classes/schedule.cfm

Description of class content is available at http://www.ats.ucla.edu/classes/classdesc.htm#hpc

GAMESS application Update

The current GAMESS version on Hoffman2 Cluster is "12 JAN 2009 (R1)" and is available in the /u/local/apps/gamess/current directory.

GAMESS is a program for ab initio molecular quantum chemistry. A wide range of quantum chemical computations are possible using GAMESS.

See How to Run GAMESS on ATS-Hosted Clusters.

GAMESS Documentation is available from http://www.msg.chem.iastate.edu/gamess/documentation.html

CPMD Application Update

The current CPMD version on Hoffman2 Cluster is 3.13.2 and is available in the /u/local/apps/cpmd/current/parallel/ib directory.

CPMD (Car-Parrinello Molecular Dynamics) is an ab initio Electronic Structure and Molecular Dynamics Program. The CPMD code is a parallelized plane wave/pseudopotential implementation of Density Functional Theory, particularly designed for ab-initio Molecular Dynamics simulation.

See How to Run CPMD on ATS-Hosted Clusters

Point your browser at http://www.cpmd.org/ for further information about CPMD and the CPMD consortium.

FFTW upgrade

The FFTW library has been upgraded to version 3.2. It is installed on the Hoffman2 cluster under the following directory:

/u/local/apps/fftw3/current

FFTW is a C subroutine library for computing the Discrete Fourier Transform (DFT) in one or more dimensions, of both real and complex data of arbitrary size. There are three versions of the fftw3 library depending on precision.

See How to Use the FFTW Library on ATS-Hosted Clusters. For FFTW documentation, see http://www.fftw.org/fftw3_doc/

HDF5 upgrade

HDF5 has been upgraded to version 1.8.2. HDF5 (Hierarchical Data Format 5) Software Library and Utilities are a suite of data model, file format, API, library, and tools that makes possible the management of extremely large and complex data collections. HDF5 is installed at:

/u/local/apps/hdf5/current

See How to Use the HDF and HDF5 Libraries on ATS-Hosted Clusters. For more information about HDF5, see http://www.hdfgroup.org/HDF5

Trilinos upgrade

Trilinos has been upgraded to version 9.0.1 It is installed on the Hoffman2 cluster under the following directory:

/u/local/apps/trilinos/current

Trilinos is a set of sophisticated software tools, containing more than 20 library packages. The Trilinos Project is an effort to develop algorithms and enabling technologies within an object-oriented software framework for the solution of large-scale, complex multi-physics engineering and scientific problems.

For more information about Trilinos, see http://trilinos.sandia.gov/ See How to Use Trilinos on ATS-Hosted Clusters

Paraview upgrade

Paraview has been upgraded to version 3.4.0. See How to Run ParaView on ATS-Hosted Clusters

ParaView is an open-source, multi-platform data analysis and visualization application. ParaView was developed to analyze extremely large datasets using distributed memory computing resources. See http://www.paraview.org for more information.

ATLAS upgrade

ATLAS has been upgraded to version 3.8.2. The ATLAS library is installed in: /u/local/apps/atlas/current

See How to Use the ATLAS Library on ATS-Hosted Clusters

ATLAS (Automatically Tuned Linear Algebra Software) provides C and Fortran77 interfaces to a portably efficient BLAS implementation, as well as a few routines from LAPACK. When the ATLAS library is built and installed, it automatically optomizes its performance for whatever system it was built on. In this case, these routines are optimized for the cluster compute nodes.

Intel compiler upgrade

The default Intel compiler on Hoffman2 is now version 11.0. For more information, see Compilers and the Documentation links on that page.

October Downtime for Hoffman2 Cluster

The Hoffman2 cluster will be down for an upgrde of the BlueArc file server and maintenance on Thursday, Oct 9, 2008 between 10:00 AM and 3:00 PM. All queues will be turned off 24 hours in advance. Any running jobs at that time will be terminated. We are sorry for any inconvenience it may cause you.

Fall 2008 Parallel Computing Classes

The Institute for Digital Research and Education (IDRE) is offering the following (free) high performance computing workshops this quarter:

Sep. 30, 2-4pm: Introduction to parallel computing

Oct. 7, 2-4pm: Parallel computing using MPI

Oct. 14, 2-4pm: Parallel computing hands-on session

Oct. 21, 2-4pm: Advanced topics of MPI (*)

Oct. 28, 2-4pm: HPC tools and libraries (*)

(*)'s are new ones introduced this quarter.

For more information or to sign up, please see
http://www.idre.ucla.edu/ (under "Upcoming Events")
http://www.ats.ucla.edu/cfapps/events/classes/schedule.cfm

ABAQUS upgrade

ABAQUS has been upgraded to version 6.8. ABAQUS is finite element modeling, visualization, and process automation software. For a description of ABAQUS, download the Abaqus 6.8 brochure pdf file from http://www.simulia.com/products/abaqus_cae.html

The Hoffman2 Cluster has licenses for the following ABAQUS features: abaqus aqua cae design euler_lagrange explicit foundation parallel standard

See How to Run ABAQUS on ATS-Hosted Clusters. ABAQUS documentation is not available on the web. To view ABAQUS documentation, at the shell prompt, enter:

abaqus doc

Tecplot upgrade

Tecplot 360 has been upgraded to version 2008 release 2. Tecplot 360 is numerical simulation and computational fluid dynamics (CFD) visualization software that combines engineering plotting with advanced data visualization. It has a graphical user interface. It is not available in batch mode. It is available from UCLA Grid interactive tab.

To use Teplot 360, at the shell prompt, enter:

tec360

See How to Run Tecplot on ATS-Hosted Clusters. For more information, see www.tecplot.com/support/360/docs.aspx

PETSc available

ATS is pleased to announce that PETSc version 2.3.3-p13 has been installed on the Hoffman2 cluster under the following directory:

/u/local/apps/petsc/current

PETSc is a suite of data structures and routines for the scalable, parallel solution of scientific applications modeled by partial differential equations. It employs the MPI standard for all message-passing communication.

See How to Run PETSc on ATS-Hosted Clusters For more information see www-unix.mcs.anl.gov/petsc/petsc-as

Trilinos available

ATS is pleased to announce that Trilinos version 8.0.8 has been installed on the Hoffman2 cluster under the following directory:

/u/local/apps/trilinos/current

See How to Run Trilinos on ATS-Hosted Clusters. For more information, see trilinos.sandia.gov

HDF5 upgrade

HDF5 has been upgraded to version 1.8.1. HDF5 (Hierarchical Data Format 5) Software Library and Utilities are a suite of data model, file format, API, library, and tools that makes possible the management of extremely large and complex data collections.

HDF5 is installed at:

/u/local/apps/hdf5/current

See "How to Use the HDF and HDF5 Libraries on ATS-Hosted Clusters". For more information about HDF5, see http://hdf.ncsa.uiuc.edu/HDF5/

NetCDF upgrade

NetCDF has been upgraded to version 4.0. NetCDF (network Common Data Form) is a set of software libraries and machine-independent data formats that support the creation, access, and sharing of array-oriented scientific data.

NetCDF is installed at:

/u/local/apps/netcdf/current

See "How to Use the netCDF Library on ATS-Hosted Clusters". For more information about NetCDF, see http://www.unidata.ucar.edu/software/netcdf/

Application Cluster

Over the past few weeks, you may have noticed an increase in the number of slots available in the serial queue. This increase was temporary, while the Infiniband cards were on backorder for the 114 nodes being added to the cluster.

Adding these nodes to the serial queue, allowed Hoffman2 users to get some use out of them and allowed us to do testing and ensure the nodes were running properly prior to adding the Infiniband cards.

These nodes will be shutdown on Tuesday, September 2, 2008 to have their Infiniband cards installed and will be added to the parallel queue that afternoon.

Hoffman2 Cluster Shutdown and Upgrade August 20, 2008

The Hoffman2 Cluster will be shut down for a short time between 7:00AM-7:00PM on Wednesday, August 20th for a major hardware upgrade to the BlueArc file server.

All campus queues will be disabled on August 18th. Shared-cluster queues will only accept requests that will be finished before August 20th. All jobs running on the cluster will be terminated August 20th.

Hoffman 2 Cluster Shutdown and Upgrade July 9-11, 2008

The Hoffman2 Cluster will be shut down between Wednesday, July 9 and Friday, July 11 to have additional electrical power installed. The BlueArc file server will also be reconfigured during this period.

This outage is extended to 11AM Friday, July 11th due to unforeseen problems encounted by UCLA Facilities during the power upgrade.

The power upgrade will allow us to more than double the size of the Hoffman2 cluster. We will be adding 115 nodes or 920 cores to the cluster. These nodes were purchased by 5 separate research groups for inclusion in the Shared Cluster portion of Hoffman 2.

Stata10 available

The multi-processor version of Stata10 is now available on the Hoffman2 Cluster for both SGE batch and interactive applications. Stata10 and Xstata are available as serial applications through the UCLA Grid Portal

Stata is a complete, integrated statistical package that provides a broad suite of statistical capabilities, complete data-management facilities, and publication-quality graphics.

Please see How to Run Stata on ATS-Hosted Clusters for detailed information.

Stata manuals are not available online. They may be borrowed from ATS Statistical Consulting.

Major additions include:

Text strings can be read and manipulated as normal data
New interactive terminal based on wxWidgets, pango and cairo
New 2D plot styles 'histogram' 'labels' 'image' 'rgbimage'
New 3D plot styles 'labels' 'vectors' 'image' 'rgbimage'
User control over color definitions and color use in plots
Improved font handling and text formatting
New syntax to handle string variables and string functions
Creation of animated gif sequences
Support for UTF-8 and other multi-byte font encodings
Japanese language documentation and internal help

Demo plots illustrating these and other features are online at:

http://gnuplot.sourceforge.net/demo_4.2/

(description from: http://www.gnuplot.info)

Hoffman2 Cluster Now Available

The new Hoffman2 Cluster is now installed and available for use.

For Hoffman2 Cluster features and how to transition from the Hoffman Cluster to the Hoffman2 Cluster, see the Hoffman2 Cluster Announcement.
To learn all about the new Hoffman2 Cluster, please see the Hoffman2 Cluster web site.

Computational Cluster Programs

Hoffman2 Cluster News and Announcements

Contents

News Items

How to decide whether to run your job on your group's own nodes or the shared cluster?

Back-filling