When Models Query Models
2022-07-15 , Liffey Hall 1

The design of large-scale engineering systems, including but not limited to aerospace, particle accelerators, nuclear power plants, is carried out by a wide range of numerical models such as CAD files, finite-element models, and machine learning surrogate models to name a few. In order to provide a uniform modelling interface, we encapsulate numerical models in notebooks. A notebook is controlling model creation, execution, and query of results. Numerical solvers are embedded into Docker containers and provide an isolated and reproducible environment exposing a language-agnostic REST API. A model registry enables efficient queries of models. The overall system is represented as a collection of models that exchange data. Then, the design optimization involves execution of a dependency tree of models to study the impact of a parameter change and perform its optimization. In this contribution, we present a model query mechanism allowing notebook models to query one another. The model dependencies are represented with a graph with suitable processing algorithms. In order to ensure that only affected models are executed we derive and cache a model resolution order. The presented modelling framework relies on open source-technologies (packages: pydantic, Fast API, Jupyter, papermill, scrapbook, containers: Docker and Openshift as well as databases: MongoDB and Redis) and the talk will focus on good practices and design decisions encountered in the process.


Expected audience expertise: Domain:

some

Expected audience expertise: Python:

some

Abstract as a tweet:

When models query models presents a package for cached model query for notebook based hierarchy of numerical models for modelling of complex physical systems.

Enthusiastic, hard-working software engineer with contagious energy and passion for working on interdisciplinary projects. Focused on delivering high-quality products in a timely manner. Servant leader always eager to learn from and with team members while instilling the ownership, autonomy, and sense of responsibility.

Over ten years of experience as a software developer for modelling and analysis of complex systems. Passionate about model-based system engineering solutions improving existing analysis workflows while ensuring reproducibility. Currently, the lead developer of an MLOps framework to support the design of superconducting accelerator magnets. Formerly, a technical lead for an agile team of experts and scientists creating signal monitoring framework for the Large Hadron Collider.