Towards Accurate Simulation of Global Challenges on Data Centers Infrastructures via Coupling of Models and Data Sources

• Data:
  • ECMWF weather/climate data
  • UNHCR refugee data
  • food security data
  • telecommunication data
• Models and software:
  • macro- and micro-
  • ABM with location network
    • Flee framework (python3)
  • weather/climate forecasts
  • GIS: OSM driven toolkit
• Usage: Burundi, CAR, S.Sudan, Mali

• Models and software:
  • ABMS for traffic
    • SUMO framework
  • CFD for NO_x spread in air
    • OpenFOAM, ANSYS Fluent
    • ⨯ Fenics-HPC, ⨯ NEK5000
  • weather/climate forecasts

• Data:
  • weather/climate data
  • streaming from sensors
  • OpenStreetMaps, other
• Usage: twins for
  • ✓ Györ (HG)
  • EU: Stuttgart(DE), Graz(AT)
  • US: Milwakee(WS)

• Models and software:
  • ABMS for message spread
  • numerical linear algebra
    • PETSc, SLEPc
    • eigenvalues histogram
  • nets: NetworKit, Snap

• Data:
  • streaming from Twitter
  • telecommunication data
  • SNAP datasets
• Usage:
  • COVID-19 tweets

Cloudify:

• Clouds out-of-the-box
• coupling mechanisms:
  • job_depends_on
• OASIS TOSCA standard
• Web GUI
• many extensions

Croupiuer extension:

• workload managers:
  • HPC: ✓ Slurm, ✓ Torque
  • HPDA: ↺ Mesos
• coupling mechanisms:
  • job_mpi_coupled_with
  • job_data_coupled_with
    • streaming data with Kafka
• data catalogues: ↺ CKAN

• Notation
  • ↦ acyclic coupling
  • cyclic coupling:
    • ⥂ sequential
    • ⇆ concurrent
• SNA:
  • sim↦verify/validate

• Migration:
  • conflict model ↦ migration model
  • migration model ↦ validation activities
  • coarse-grained national ⇆ refined local
• UAP:
  • traffic model ↦ CFD model of NO_x flows
  • WCD ↦ CFD model of NO_x flows

CKAN (DMS/DC):

• consistency in harvesting
• adequate level of security
• extensible via plugins
• data delivery methods:
  • ✓ files
  • ✓ links to external sources
  • ✓ profiled harvester

Apache Kafka:

• real-time data pipelines
• streaming data in HiDALGO
  • ↺ Twitter (with tweepy)
  • camera based traffic
  • monitor based pollution

• specialized data center:
  • ECMWF: WCDF
• vision:
  • bring users to the data
  • use data while it is hot
  • access using metadata
• software: Polytope

• goal:
  • enable coupling to build a workflow
• implementation:
  • ✓ Step 1: Static coupling
    • static reanalysis data (calibration)
  • ↺ Step 2: Dynamic coupling
    • forecast data via a REST API