Monday, April 1 • 11:00am - 11:30am
HPC Performance and Private Cloud Agility - Euclid-as-a-Service - Stig Telfer, StackHPC

Sign up or log in to save this to your schedule and see who's attending!

Combining Public and Private Cloud capacity heralds new opportunities for Research Computing Communities. Providing cloud flexibility ensures different classes of workload may be targeted at the most effective federated resources. For private cloud, OpenStack has become the standard IaaS environment. We describe how combining OpenHPC with Scientific OpenStack allowed the rapid deployment of bare-metal and virtualised infrastructure for the Euclid community to tackle short-term simulation needs.

Euclid, an ESA medium class astrophysics mission, aims at a better understanding of dark matter and dark energy. The UK is involved in Euclid on many levels. One of the most important is in the development of the weak lensing analysis of galaxy images by the University of Edinburgh at the Institute for Astronomy. To validate the accuracy of the algorithms, analyses on ~13 full Euclid data sets simulated with different parameters to mimic realistic data scenarios are required. The computational requirements for this are ~16,000 cores for 4 weeks, and ~150TB of intermediate transient data. The coding and execution environment are set at the Science Ground Segment level and do not conform to the default environments deployed by most large HTC sites in the UK. As such a more flexible compute platform was required, where the Euclid execution environment could be installed. This was accomplished with the cloud-based Slurm-as-a-Service (SaaS) platform developed and deployed by StackHPC on resources provided by the University of Cambridge HPC team through the aegis of the STFC IRIS project. The SaaS platform enabled rapid deployment and scale up to the necessary resource levels without requiring a large amount of effort in re-tooling code or changing the operational model.

This talk will describe how infrastructure, platform, and application were integrated into a cloud-native deployment, how it was delivered and how it could be generalised for similar projects

avatar for Stig Telfer

Stig Telfer

Stig has a background in R&D working for various prominent technology companies, particularly in HPC and software-defined networking. Stig is now CTO for StackHPC, a consultancy specialising in the convergence of cloud, HPC and big data. Stig is also co-chair of the OpenStack Scientific... Read More →

Monday April 1, 2019 11:00am - 11:30am
Premium 3 Etc.Venues St. Paul's

Attendees (6)