Computational Cluster Programs

Hoffman2 Cluster Hardware

Hoffman2 Cluster Overall Design

The following figure shows the overall design of the Hoffman2 Cluster.

Shared Cluster Hosting System Standards for Compute Nodes

In order to maximize the effective management of the Shared Cluster System and to provide the highest level computing services to Shared Cluster customers, compute nodes added to the Hoffman2 Cluster must meet minimum standards. As of October 2007 these standards are:

  • 1U, rack mounted compute nodes
  • High quality, well integrated, hardware in terms of thermal, power, rack mounting capability and parts used
  • Quad-core 2.0GHz AMD Opteron CPUs
  • 4GB of RAM per compute node (1GB per core)
  • 80-120GB hard drive per node
  • Dual Gigabit Ethernet ports
  • PCI-Express slot (required for high-speed interconnect)
  • DDR InfiniBand interconnect HCA
  • PCI-Express slot (required for high-speed interconnect)
    • 3 year warranty

The standards will be evaluated periodically and updated based on best price/performance.

Campus General Purpose Cluster Nodes

The Campus General Purpose Cluster consists of 40 AMD Opteron nodes (160 cores) configured to the same specifications as the rest of the Shared Cluster. This is done to allow, by prior special agreement, the use of the entire cluster by large compute runs.

Base Shared Cluster

The Base Shared Cluster consisting of 24 AMD Opteron nodes (96 cores) provides the base pool of nodes available to Shared Cluster Hosting Program customers. These nodes are available only to those research groups which provide equipment under the program.

Application Cluster Nodes

The Application Cluster consists of 10 Intel nodes each with 2 quad core CPU's (80 cores). The nodes each have 16 GB Memory and a 250 GB hard drive. They do not have InfinBand.

Interconnect

The Hoffman2 Cluster has both InfiniBand and a gigabit Ethernet interconnects. The Ethernet interconnect is dedicated to traffic in and out of the storage system and also handles various administrative functions. InfiniBand is used for inter-node, MPI-type communication. Using these two interconnects maximizes performance within the cluster.

Cisco 6509-E Ethernet Switch

The Cisco 6509-E switch is a 288 port GigE switch with a 10Gb uplink to the campus backbone.

InfiniBand

The Cisco Cisco SFS 7024D DDR IB Server Switch is configured for 288 ports.

Storage

BlueArc

A high-performance, fault-tolerant BlueArc Titan 2200 storage server provides storage for the Hoffman2 Cluster System. The Titan disk space is formatted as RAID 5 (striped with distributed parity) to increase performance. Usable storage is about 80 TB after striping.

The BlueArc Titan's very low latency results in high data throughput. The Titan can hold up to 8 billion total files with up to 4 million files per directory.

On the Hoffman2 Cluster home directories are served from the BlueArc. Users of the General Purpose Cluster Nodes have quotas on their home directories. Users of the Shared Cluster System pay a one-time, per terabyte, fee for storage. This fee pays for storage space, administration of the storage system and backup services for a mutually agreed to amount of data.