HPC and Big Data Technologies for Global Challenges
2021-01-20
Sergiy Gogolenko
Introduction to the Project HiDALGO and its Services
HiDALGOEU founded project
ℵ824115
@OITTTIO


Introduction to the Project HiDALGO and its Services

Sergiy Gogolenko

High-Performance Computing Center, Stuttgart, DE

HiDALGO workshop @ HiPEAC∘2021-01-20

Introduction

Global Challenges: Media and Pop Culture

Global Challenges: Art

CoE for Global Challenges

  • Consortium:
    • 13 partners
    • 7 countries
  • coordination: ATOS
  • TM: HLRS (DE)
  • Runtime
    • duration: 36M
    • start: Dec 2018
  • Fund:
    • EU Horizon 2020
    • €7'991'500.00

partners.png

HiDALGO Motivation and Main Objectives

Goal: evidence based policy-making for current and upcoming situations via accurate GC simulations

High Level Ambition & Project Targets

  • Benefit from the synergies between HPC, HPDA, AI, and GCs
    • baseline for HPC, HPDA, and AI in the domain of GCs
    • focus on highly accurate models
  • Provide a single entry point for decision makers (and other entities)
    • a multi-domain portal for the GC community
  • Connect & train the different communities

Use Cases

What challenges to choose?

For the first time in history, more people die today from eating too much than from eating too little; more people die from old age than from infectious diseases; and more people commit suicide than are killed by soldiers, terrorists and criminals combined.

— Yuval Harari "Homo Deus: A Brief History of Tomorrow"

Top news of the last months (dominate in media):

Refugee and human migration simulation

  • develop realistic models for simulating refugee streams
  • complete data collection on refugee movements
  • investigate the consequences of a nation closing its borders
  • HPC in migration use case:
    • expensive simulations
    • ensemble runs
  • HPDA in migration use case:
    • pre-process GIS data
    • analyse weather/climate data
    • post-process results
    • synthetic data
  • Usage: CAR, S.Sudan, Mali, Ethiopia

south_sudan_migration.png

COVID and Flu spread simulation

  • simulate epidemics spread across the local area
  • determine people infected, ICU occupation, etc.
  • investigate the effects of applying certain policies (i.e., curfews)

facs-relults.png

  • HPC in epidemiology use case:
    • expensive simulations
    • ensemble runs
  • HPDA in epidemiology use case:
    • pre-process GIS inputs
    • post-process results
    • analyse weather/climate data
    • synthetic inputs
  • Usage: London, Madrid

Urban air pollution simulation

  • simulate pollution in cities based on real-world sensor data
  • couple models for traffic, air flow, and weather simulation
  • design evidence-based decision models to leverage green growth

uap_gyor.png

  • HPC in UAP use case:
    • expensive simulations
    • ensemble runs

  • HPDA in UAP use case:

    • impute the missing data
    • reduce models
    • post-process results
  • Usage: Györ, Graz, Milwaukee

Social network analysis

  • analyze the structure of social networks
  • analyze the stochastic behavior of the message spreading
  • simulate the spread of messages among users
  • study the spread of 'Fake News' and develop countermeasures

sna-clustering.png

  • HPC in SNA use case:
    • expensive network analytics
    • ensemble runs
  • HPDA in SNA use case:
    • features extraction
    • retweet probabilities
    • post-process results
  • Usage: COVID-19 Tweets

HiDALGO Approach and Generalized Workflow

Accurate digital twinning of GCs ⥂ coupled simulations + HPDA

  • models for diverse social and physical phenomena (often multiscale)
  • massive static and streaming data sets

Sorry, your browser does not support SVG.

HiDALGO Tools

hidalgo-tools.png

Services and portal

hidalgo-services.png

HiDALGO Services: Consultancy

  • Analyse problems
  • Analyse systems/codes
  • Propose solutions
  • Adapt modules
  • Customize final offering

HiDALGO Services: Training & Community

  • Public Discussion Forums
  • Events
  • Yellow Pages
  • Matchmaking Tools

HiDALGO Services: Support

  • Expert Support through
  • Online Documentation

HiDALGO Services: Co-Design

Software-Software Co-Design

  • Adapt the codes based on library capabilities
  • Adopt new features of SW libraries

Hardware-Software Co-Design

  • Port codes to benefit from the new machines/architectures
  • Study HPC/HPDA tools on new machines/architectures

HiDALGO Services: Repositories

  • Data for Global Challenges:
    • massive datasets harvested from various sources (CKAN)
  • Efficient data transfer between infrastructure
  • Complementary data services:
    • format conversion
    • visualization

HiDALGO Services: Portal - Single Entry Point

Collaborating

https://hidalgo-project.eu/stakeholder-survey

contact@hidalgo-project.eu

Thank you for your attention!

https://hidalgo-project.eu/consulting

contact@hidalgo-project.eu

https://hidalgo-project.eu/stakeholder-survey

January 20

15:30 – 16:00 Derek Groen, BUL, UK Simulating the Spread of Covid-19 in Urban Areas
16:10 – 16:40 Florian Ziemen, DKRZ, DE Preparing European Weather and Climate Models for Exascale
16:40 – 17:10 Fabian Dembski, CCGSS-BW, DE Resilient Cities: Following the Path Towards Sustainable Development Goals
17:20 – 17:50 Christoph Schweimer, KNOW, AT Route Pruning Algorithm for Location Graph Construction
17:50 – 18:30 Lara López, ATOS, ES Round table: How can we solve Global Challenges through HPC/HPDA/AI?