How to Build a Reliable, Scalable Parallel Filesystem Solution using Cloud - SDC India 2018 скачать в хорошем качестве

How to Build a Reliable, Scalable Parallel Filesystem Solution using Cloud - SDC India 2018

In the last couple of years, the case for HPC in the cloud is growing stronger. But still, the HPC industry lies far behind enterprise IT in its willingness to outsource computational power. One of its reason being storage - as none of the built-in storage solutions available across the public cloud providers are suitable for applications with high bandwidth requirements. A parallel, clustered file system built on top of block storage (Ex: AWS EBS) forms a good solution. However, there exist multiple architectural approaches (such as when & why to use placement groups, autoscaling, auto recovery, replication across availability zones etc.) one can take to build parallel, clustered filesystem. Apart from building, there exists a need for effectively managing the cloud resources used for building the filesystem as it could quickly become expensive. The proposed presentation is aimed to discuss in detail different architectures (weighing Pro’s & Con’s) that can be used to build a reliable parallel filesystem in the cloud (showcasing AWS, IBM Spectrum Scale as an example) and data lifecycle techniques that help reduce OPEX cost by effectively managing parallel filesystem in the cloud. Learning Objectives Understand how the cloud is falling short for HPC workloads Introduction to various AWS compute, storage services Learn different ways (weighing pro's & con's) using which a parallel, clustered filesystem can be built using AWS services Presented by: Sasikanth Eda, IBM