Towards Cloud elasticity and high-level abstraction for HPC-based epidemiological simulations скачать в хорошем качестве

Towards Cloud elasticity and high-level abstraction for HPC-based epidemiological simulations

Yiannis Georgiou CTO Ryax Technologies This presentation outlines the design and implementation of a novel, cloud-elastic platform for executing HPC-based epidemiological simulations in support of the MePreCiSa (Precision Medicine for Healthy Cities) project at BSC-CNS. We introduce a layered software architecture that decouples domain-specific simulation logic from underlying infrastructure, providing high-level abstractions for resource management, workflow orchestration, and data handling. At its core, the architecture is agnostic to the type of cloud (public or on-premise), interfacing via the standard Kubernetes API, and employs techniques to dynamically provision and de-provision elastic virtual HPC clusters empowered with the adapted software stack (EMEWS, Swift/T, etc.) required for epidemiological simulations. To streamline end-to-end operations, we integrate the Ryax workflow-based data automation platform, enabling automated pipelines for data ingestion, preprocessing, and simulation execution across Cloud-HPC infrastructures. The platform incorporates workflows for ingesting heterogeneous data streams (demographics, mobility, clinical records), while interactive dashboards built on Apache Superset provide customizable visualizations, and reporting. Authentication and authorization are managed via Keycloak, ensuring secure, role-based access control and single-sign-on. By combining elasticity, abstraction, integrated data services, and robust security and visualization layers, this system empowers epidemiologists to run large-scale simulations with minimal operational overhead, accelerating the generation of actionable insights for urban health planning and rapid response. More information: https://hpckp.org/