Wednesday, November 13, 2013

Booth Activities for SC'13

As much as I try to shy away from shilling for SDSC on my personal blog here, the SDSC official booth (#3313) schedule has been finalized and I wanted to post it online for the world to see.  I've highlighted two major and very cool events:
  1. The LONG TAIL DEATH MATCH (highlighted in red) will be a gaming tournament featuring Meteor, SDSC's Raspberry Pi cluster and display wall.  The grand prize for the gaming tournament is a Raspberry Pi mini-cluster with a four-screen (2x2) display wall sponsored by SDSC and Aeon Computing, which is pretty darned awesome.  Even if you don't win, finalists all get either a Raspberry Pi or a Google Chromecast.
  2. All of my wonderful talks are highlighted in blue.  People who attend one of my talks can get an awesome SDSC I  BIG DATA scarf, also sponsored by SDSC and Aeon.  As far as swag goes, I think this is pretty cool.
In addition, some pretty big names will be making appearances:
  • The official Comet announcement and press briefing on Tuesday at 2:00 PM has a star-studded lineup.  I'll be present for this in an official capacity.
  • Geoffrey Fox, the PI of FutureGrid and a prominent figure in scientific cloud computing from IU, will be speaking on Wednesday at 11:30 AM.
  • John Towns, the PI of XSEDE, will be speaking on Wednesday at 2:00 PM.
  • D.K. Panda, the creator of MVAPICH2, will be speaking on Wednesday at 3:20 PM.
Printed copies of the agenda will be available at SDSC's booth (#3313), and I've uploaded a PDF version, complete with details about the LONG TAIL DEATHMATCH, to my website.

Monday, November 18
7:00 - 9:00 PM
Exhibition Opening Gala
Tuesday, November 19
2:00 - 3:30 PM
Presentation and Q&A followed by a reception
  • Michael Norman, Director, SDSC (Welcome, Comet announcement & overview)
  • Richard Moore, Deputy Director, SDSC
  • Nancy Wilkins-Diehr, Associate Director, SDSC
  • Doug Johnson and Jeff Johnson, Co-Founders, Aeon Computing
  • John Mullen, Vice President & General Manager, Large Institution Sales, Dell
  • Stephen Wheat, General Manager, HPC, Intel
3:45 - 6:00 PM
Opening Rounds, Prizes for Entrants!
Wednesday, November 20
11:00 AM
  1. Janssen R&D Collaboration
    Glenn K. Lockwood, User Services Consultant, SDSC
  2. Computational Finance
    Robert Sinkovits, Interim Director for Scientific Computing Applications, SDSC
  3. Neuroscience Gateway
    Subhashini Sivagnanam, Research Staff Member, SDSC
11:30 AM
  1. FutureGrid Connection
    Geoffrey Fox, FutureGrid Connection to Comet, Indiana University
  2. XSEDE Gateway Program, Utility of Comet
    Suresh Marru, Program Lead, XSEDE Science Gateways
  3. SciGap: Science Gateways Platform as a Service
    Amit Majumdar, Interim Division Director, Data Enabled Scientific Computing, SDSC
11:45 AM
Mellanox Booth 2722

SR-IOV Test Results
Glenn K. Lockwood, User Services Consultant, SDSC
Chaitan Baru, Associate Director, Data Initiatives; Director, Center for Large- scale Data Systems Research (CLDS), SDSC

2:00 PM
  1. CyberGIS Gateway for Data-Rich Geospatial Research and Education
    Shaowen Wang, Professor and Director of CyberGIS Center, UIUC
  2. XSEDE Counting on Comet
    John Towns, PI and Project Director, XSEDE, NCSA/University of Illinois
    Wayne Pfeiffer, Distinguished Scientist, SDSC
2:30 PM
  1. SEEDME.ORG: A Free Infrastructure for Researchers to Share Data
    Amit Chourasia, Senior Visualization Scientist, SDSC
  2. SR-IOV Test Results
    Glenn K. Lockwood, User Services Consultant, SDSC
  3. Data Analytics
    Mahidhar Tatineni, User Services Manager, SDSC
3:00 PM
Robert Sinkovits, Interim Director for Scientific Computing Applications, SDSC
3:20 PM
Dhabaleswar K. (DK) Panda, Professor, Computer Science and Engineering, Ohio State University
3:35 PM
Chaitan Baru, Associate Director, Data Initiatives; Director, Center for Large- scale Data Systems Research (CLDS), SDSC
3:45 - 6:00 PM
Elimination Rounds, Prizes for the Winners!

Thursday, November 21
10:15 - 10:45 AM
Michael Norman, Director, SDSC
10:30 - Noon
Championship Rounds, Grand Prize Award Ceremony!
3:00 PM

Saturday, November 9, 2013

A Photographic Tour of Gordon and Trestles

I took a bunch of photos of Gordon and Trestles, the supercomputers at the San Diego Supercomputer Center, in case I needed some material for background graphics or flash for any of the presentations and activities in which I'll be participating surrounding SC'13.  Upon reviewing the photos, I realized that there is a lot of technological content captured in them that others, either professionals or enthusiasts, might enjoy seeing.

So as to contribute what I can, I uploaded a few photos and annotated them as best I could in a gallery I call "A Photographic Tour of Gordon and Trestles." Here are the thumbnails:

SDSC's Gordon Supercomputer A Gordon IO Node Rack of Gordon IO Nodes Rack of Gordon Compute Nodes Gordon Compute Subrack, Front Fan Modules in Gordon Subrack
Rear of Gordon Compute Subrack Cabling in Gordon Compute Rack Rear of Gordon Compute Rack Networking for Gordon Compute Nodes A QDR InfiniBand Switch in Gordon Gordon Compute Node
Gordon Compute Nodes Gordon Viewed from Rear Gordon Compute Rack Ethernet SwitchesInter-row Cabling in GordonFlash Photo of Gordon Compute SubrackSDSC's Trestles Supercomputer
Trestles's Infiniband SwitchA Rack of Trestles Compute NodesTrestles Nodes CloseupData Oasis OSS

Gordon and Trestles, a set on Flickr.

For those who may not be familiar, SDSC is the home to two "national systems," or supercomputers which are available for anyone in the U.S. to use through the National Science Foundation's XSEDE program.  These two systems were both integrated by Cray (formerly Appro) and are:
  • Trestles, a 324-node, 10,363-core, 100 TF, AMD Opteron (Magny Cours) cluster outfitted with a fat tree QDR InfiniBand interconnect designed to accommodate the computational needs of the common man.  This machine was deployed in 2011.
  • Gordon, a 1024-node, 16,384-core, 341 TF, Intel Xeon (Sandy Bridge) cluster outfitted with two (dual-rail) hybrid 3D torus QDR InfiniBand fabric designed to tackle data-intensive problems with an architecture very rich in IO capabilities.
Most of the gallery details Gordon, since it has a very unique architecture based around a relatively complex basic building block:

Sixteen compute nodes in an 8U subrack share a common 36-port QDR IB switch, and an IO node with sixteen SSDs also hangs off that switch.  The IO node is connected directly to Lustre via a 2×10Gbe bonded link, and it also provides the SSDs to the compute nodes over InfiniBand.  This entire building block represents one torus node on Gordon's overall 4×4×4 torus, and since the whole fabric is really dual-rail, each node (compute and IO) is connected to two of these 36-port switches.

This all makes for a very pretty machine.

Monday, November 4, 2013

On the road to SC'13

It's been about a year since I decided to quit the life of a research scientist and move across the country to get into the supercomputing business, and coincidentally, it's also time for this year's SC conference.  I note the coincidence because hearing about SC'09 from pals in attendance was the first time I realized there was a whole industry surrounding what I considered to be a fun hobby.  I'd argue that hearing about SC antics was a major force in putting me on the path that led me to becoming a part of the supercomputing industry--for example, I specifically recall thinking that my dream job would be one that paid for me to attend SC.

It seems like I've found that job now, and luckier still, I get to attend SC during my first year in the business. I've had this post open as a draft for the past two weeks and initially intended to write a reflective post on where I've come in the last year, but as it turns out, going to SC is a lot of work, and I don't really have time to do all that writing.

My Schedule

Instead, I'm going to do a bit of self-promotion and let people know where I'll be presenting and what I'll be doing.  I am giving three 7-10 minute public booth talks on two different topics, all of which will be on Wednesday, November 20:

  1. At 11:00 AM, I will be presenting some of the work we supported to enable large-scale genomic analysis using Gordon at the SDSC booth (#3313)
  2. At 11:45 AM, I will be talking about our early benchmarks using SR-IOV and Mellanox Virtual-IQ, the key technologies underpinning Comet, the world's first fully virtualized HPC cluster, at the Mellanox booth (#2722)
  3. At around 2:45 PM, I will be presenting the same talk on Mellanox Virtual-IQ and SR-IOV, but this time at the SDSC booth (#3313)
The exact times are a little infirm, and I don't have finalized (read: company-approved) titles for these talks yet, but I will update this post when I do.  Also, a map of the expo floor is already online; both the SDSC and Mellanox booths are in the lower-right quadrant.

In addition to these speaking events, I will definitely be at the SDSC booth (#3313) for these events:
  • On Tuesday at 2:00 PM, the official Comet announcement will be made by SDSC's director, deputy directors, and a bunch of dignitaries from the vendor partners involved.  I will be on hand to discuss the project with whoever is interested in learning more and to consume the free refreshments to be served.
  • On Thursday at 10:00 AM, we will be concluding a special event (not sure how much more I can/should say).  I will be there for the good times.
Until SDSC's final SC'13 agenda is published though, none of this is set in stone so please treat it as such.  Also, if it's of interest to anyone, I will be flying in early (arriving Sunday at 6:30 PM) and leaving late (Friday at 10:00 PM)

SDSC's Schedule

UPDATE: I've posted the exact schedule on a subsequent post.

SDSC has some cool stuff planned for the booth centered around Gordon, SDSC's data-intensive production machine, and Comet, SDSC's upcoming machine designed to address the needs of the 99% of supercomputing users who aren't running ultra-massive hero jobs.

Right now there are two separate "lightning round" talks being given: one on Tuesday morning, and one on Tuesday afternoon.  I am presenting different talks at both, as are a number of my colleagues.  In addition, I think there is an additional set of lightning talks, but I'm not (as far as I know...but I may be wrong) giving one there.  Once the SDSC schedule is available for public consumption, I'll post an update here.

Finally, there is a really neat special event for students spanning the entire exhibition which concludes on Thursday morning.  I'm not sure how secretive this is, so I won't say more until I know more.

My Talks

I'm presenting on two topics, both of which I think are really neat.  

The Gordon Talk

So as to not leave this post without any fun pictures, here is one that describes my talk on large-scale genomic analysis:

Lustre filesystem capacity and number of jobs in flight over time

In brief, a project came to us that involved processing over four hundred complete human genomes straight out of a next-generation sequencer.  The input data came in the form of eight 6-terabyte RAID0 devices which we needed to somehow plug into Lustre to upload, and the ensuing adventure was quite challenging because of the nine-step processing pipeline through which each genome needed to go.  It proved to be a fantastic example of non-traditional uses of HPC that, despite requiring only modest CPU power, is wholly intractable without a lot of high-end compute capability.  

We wound up exploiting almost every aspect of Gordon's flexible architecture:

The fundamental building block of SDSC's Gordon resource

to solve the problem within the parameters necessitated by the project's sponsors.

The Comet Talk

I'm also presenting on some of the early benchmarking work I did with coworkers while architecting Comet, SDSC's upcoming supercomputer.  In particular, Comet will rely heavily upon Mellanox's Virtual-IQ technology and SR-IOV to allow users and gateways to dynamically provision virtual clusters with their own software stacks and operating environments on SDSC's hardware.

SR-IOV is a really interesting technology that greatly simplifies the process of virtualizing cluster interconnects (i.e., Infiniband) without the significant loss of performance that has been characteristic of virtualized clusters.  This talk will be a bit more data-oriented, and I will be presenting hard performance numbers comparing the following types of clusters:

  • Bare metal
  • Virtualized with PCIe passthrough
  • Virtualized with Mellanox's SR-IOV-capable Infiniband adapters
  • Virtualized on Amazon EC2 with cluster compute instances (sound familiar?)
We ran both standard benchmarks as well as real-life applications across these different types of clusters, and the results were quite interesting.