SOE HPC Cluster

Announcements

 

Hardware Architecture

The cluster hardware is based on Intel Sandy Bridge 2670 CPUs, 16 cores and 128 GB of RAM per node. 

There are 56 nodes in the cluster available for general use.

The nodes are interconnected over FDR infiniband and Gbit networks.

Each of the two interactive front-end machines have two Nvidia Kepler K20 GPU cards installed.

Software Installed 

At the moment, the following apps have been installed and tested on the
cluster:

  •   PGI 13.7 C/C++/Fortran compilers
  •   GNU GCC/G++/gfortran 4.7.3 compilers
  •   Open MPI (compiled with GNU and PGI compilers)
  •   LAMMPS
  •   GROMACS
  •   OpenFoam
  •   Matlab
  •   CUDA

Environment

The front end hosts and the storage server on the cluster can be accessed via SSH from the RU networks. See the details here.

The cluster resources are managed by SLURM queue system. Users will need to submit all their applications through batch scripts to run on the cluster. Here is a reference to the most common SLURM commands.

The environment for the applications, such as path to the binaries and shared libraries as well as the licenses, is loaded/unloaded through the environment modules. For example, to see what the modules are available on the cluster, run command  module avail, to load a module, say matlab, run command   module load matlab. More information on using the environment modules can be found at this link.

There are three file systems available for computations: local /tmp on the nodes, Lustre, and NFS. Both Lustre and NFS are shared file systems and provide the same file system image to all the computational nodes. If you run a serial or parallel computational job, utilizing only one node, always use the local /tmp file system. For multi-node MPI runs, you need a shared file system, so please use Lustre. Avoid using NFS as its I/O saturates quickly at multiple parallel runs and becomes extremely slow. For details on using the file systems on the cluster, follow this link

Account on the Cluster

Who is eligible for having an account: Engineering faculty, postdocs, graduate students

Account Requirements: EIT/DSV Account

Application for SOE HPC Cluster Account: http://ecs.rutgers.edu/form/soe_hpc_cluster_application (NOTE: Must be on a Rutgers-networked computer/VPN to access! ) Undergraduate and graduate students should have their faculty advisor to send a confirmation e-mail to Alexei about requested computing resources. 

Those with accounts may access the SOE HPC cluster via SSH to soemaster1.hpc.rutgers.edu and soemaster2.hpc.rutgers.edu

ACCESS NOTE: The two aforementioned hosts and the application URL are only accessible from WITHIN Rutgers' networks, so you must be logged in on a Rutgers computer physically or remotely via SSH or VPN to access these hosts.

© 2011 Rutgers, The State University of New Jersey - This site is XHTML 1.0 valid.