A range of software toolkits are available that address various facets of UQ. The present team of investigators has, over the past three years, built on this work, and honed it to provide a powerful, interoperable, open source and open development environment into which existing and new UQ capabilities can be inserted. These tools are intended to be general-purpose, cross-cutting, and highly flexible, our priority demonstration applications cover fusion energy, weather, biomedicine, advanced materials, human migration, and geosciences. Ease of use is a central design feature, by which existing applications can be quickly instrumented with VVUQ software, and rapidly deployed on supercomputers. As a testament to these capabilities, several of our VVUQ software items have been downloaded thousands of times.

Will establish the SEAVEA toolkit which aims to ensure that we can orchestrate full VVUQ on the NEPTUNE code. Official releases will be in M12 and M24 and a final release in M30, with the remaining months used to promote and harden the full version.

The SEAVEA toolkit establishes a platform for verification, validation and uncertainty quantification (VVUQ), building on the work VECMAtk and UQ codes from the Alan Turing Institute (ATI). The goal is to provide tools that can be combined to capture complex scenarios, applied to applications in disparate domains, and used to run multiscale simulations on any desktop, cluster or supercomputing platform.

March 2023 Release of SEAVEAtk

Figure 1: Graphical overview of the components in the SEAVEA toolkit release and their integrations.

The March 2023 release is made up of the following tools:

• EasyVVUQ
• FabSim3
• QCG PilotJob
• Muscle3
• EasySurrogate
• MOGP Emulator

EasyVVUQ – v1.2

Documentation: https://easyvvuq.readthedocs.io/en/latest/

Code: https://github.com/UCL-CCS/EasyVVUQ/releases/tag/v1.2

New features:

• New Simplex Stochastic Collocation sampler for irregular outputs, e.g. with discontinuities or high gradients in the stochastic input space. Works for scalar QoI only thus far.
• Grid-Search sampler, (e.g. for neural-network hyper parameter tuning).
• HDF5 decoder to allow for reading HDF5 output files, useful when dealing with outputs of different size.

Tutorials:

• SSC tutorial: https://github.com/UCL-CCS/EasyVVUQ/blob/dev/tutorials/simplex_stochastic_collocation_tutorial.ipynb
• Hyperparameter tuning tutorial, local sampling: https://github.com/UCL-CCS/EasyVVUQ/blob/dev/tutorials/hyperparameter_tuning_tutorial.ipyn
• Hyperparameter tuning tutorial, remote sampling with FabSim3: https://github.com/UCL-CCS/EasyVVUQ/blob/dev/tutorials/hyperparameter_tuning_tutorial_with_fabsim.ipynb

Usability updates:

• Make it more obvious how to import a pandas dataframe containing cases to be considered
• Make it more obvious how to massage the results from the runs before performing the PCE/SC/MC analysis

FabSim3 – v3.71

Documentation: https://fabsim3.readthedocs.io/en/latest/ 

Code and tutorials: https://github.com/djgroen/FabSim3/releases/tag/v3.71

New features:

• New VVP tutorials in the readthedocs site.
• More HPC access modes for automated access.
• Extended multiplexing documentation.
• Improved support and documentation for QCG-Pilotjob installation on remote machines.
• Add support for continuing replicated simulation runs by using the replica_start_number argument in run_ensemble.
• Plugin developments
  ○ Initial version of FabParticleDA.
  ○ Initial version of FabDynamics
  ○ FabFlee: added built-in visualization tools, and updated to support Flee 3.0

QCG-PilotJob – v0.14.0

Documentation: https://qcg-pilotjob.readthedocs.io

Code: https://github.com/psnc-qcg/QCG-PilotJob/releases/tag/v0.14.0

New features:

• Support for QCG-PJ on the ARCHER2 supercomputer.
• High level interface with EasyVVUQ for novice users.
• Improved handling of custom arguments for LocalManager in QCGPJExecutor
• Fixed dependencies of zmq package
• Improved script element handling of in various parallel modes
• Changed the method of gathering available CPU identifiers
• A range of minor improvements

Bugfixes:

• Improved handling of custom arguments for LocalManager in QCGPJExecutor
• Fixed dependencies of zmq package
• Improved script element handling of in various parallel modes
• Fixed bug undefined variable
• Typo in service arguments

Other Improvements

• Changed the method of gathering available CPU identifiers

MUSCLE3 – v0.6.0

Documentation: https://muscle3.readthedocs.io/en/latest/

Code: https://github.com/multiscale/muscle3/releases/

Updates:

•  Connecting multiple conduits to outgoing ports
• Initial checkpointing support (Python only preview)
• Support for Intel compilers & clang.
• TCP latency improvements
• Small improvements to error messages and documentationEasySurrogate

EasySurrogate

Updates:

• Added Dropout regularization. Tutorial and documentation on this feature is forthcoming

MOGP – v0.72

Documentation: https://github.com/alan-turing-institute/mogp-emulator/tree/main/mogp_emulator/demos

Code: https://github.com/alan-turing-institute/mogp-emulator/releases/tag/v0.7.2

Updates:

• Bugfix release to fix a few outstanding issues. Fixes issues #241 and #242. Changes in this release:
  o Update to Dimension Reduction to improve memory usage with some additional tests
  o Update to documentation to reflect new dependency requirements
  o Improvements to GPU build script in setup.py to remove outdated shared object library files.

Previous Releases

Month 12 Release of SEAVEAtk

EasyVVUQ v1.1.2

• Documentation: https://easyvvuq.readthedocs.io/en/latest/
• Code: http://www.github.com/UCL-CCS/EasyVVUQ

Updates

Overhaul of SC sampler / analysis class:

• Made isotropic sparse-grid subroutines more scalable to higher input dimensions. Reused dimension-adaptive subroutines for this purpose, instead of having (slower) separate isotropic routines.
• Rewrote dimension-adaptive SC expansion as a standard PCE expansion with generalized PCE coefficients. See adaptive sparse-grid tutorial.

Documentation

• Extensive methodological sparse-grid tutorial: https://www.researchgate.net/publication/359296270_Adaptive_sparse-grid_tutorial
• New tutorial on using mathematical expressions involving parameters in template files using the Jinja encoder: https://github.com/UCL-CCS/EasyVVUQ/blob/dev/tutorials/jinja_tutorial.ipynb

QCG PilotJob v0.13.1

• Documentation: https://qcg-pilotjob.readthedocs.io/en/latest/
• Code: https://github.com/vecma-project/QCG-PilotJob

New features

• Additonal environment variables for launching srun jobs from inside a pilot job parallel task

Bugfixes

• Support for different argument names between Slurm versions (Lourense Veen)
• Race condition in wait4all method (LourenseVeen)
• Reporting version in debug logs

Other Improvements

• Enhanced tests (Lourense Veen)
  • support for tests with different Slurm versions
  • updated documentation for tests
  • bugfixes within the testing scripts

EasySurrogate v0.19

• Tutorial: https://github.com/wedeling/EasySurrogate/tree/master/tutorials
• Code: https://github.com/wedeling/EasySurrogate

Updates

Some minor changes have been introduced in the ANN / Deep active subspace methods:

• made it possible to select varying numbers of neurons, to create “constrained” hidden layers. This is used to make a deep-active subspace type of surrogate without orthonormality embedded in network architecture. Instead, orthonormality is computed in a post-processing procedure. Explained in updated tutorials.

Tutorials

• 2 epidemiological deep-active subspace Jupyter notebook tutorials (HIV and COVID 19): https://github.com/wedeling/deep_active_subspace_data

FabSim3 v3.2

• Documentation: https://fabsim3.readthedocs.io/en/latest/
• Code: http://www.github.com/djgroen/FabSim3

New Features

• Two new plugins: FabSCEMa and FabNEPTUNE
• Range of fixes and improvements to the FabMOGP plugin
• Inclusion of the FabSim3 Automated Validation tutorial.
• Added support for multiplexing in the SSH layer, and tutorial documenting it. This greatly reduces the number of times users have to type in passwords.
• (We also submitted a paper describing FabSim3 to a journal)

Updates

• A range of improvements to the documentation.

Tutorials

This version of FabSim3 has been used to enable the following application tutorials:

• • FabFlee (migration modelling) link to tutorial
  • FabMD (molecular dynamics) link to tutorial
  • FabCovid (Covid-19 simulation) link to tutorial
  • FabUQCampaign (basic CFD models) link to tutorial
  • fabmogp (Multi-Output Gaussian Process Emulator) link to tutorial

Muscle3 v0.5.0 (made possible through the Dutch e-Science Center)

• Documentation: http://muscle3.readthedocs.io
• Code: https://github.com/multiscale/muscle3/releases/tag/0.5.0

New Features

• MUSCLE3 now starts submodels and other components (using QCG-PilotJob)
• Automatic resource management for components on HPC

Incompatible changes

• compute_elements are not called components in .ymmsl files

Improved

• Integration with QCG-PilotJob
• Extension of yMMSL language
• Simplification of the installation process by removal of gRPC dependency and automatic detection of MPI availability
• Use latest OpenSSL library when installing it automatically
• Cleaner logging output
• Improved communication performance and scalability.

MOGP Emulator v0.7.0

• Documentation: https://github.com/alan-turing-institute/mogp-emulator/tree/main/mogp_emulator/demos
• Code: https://github.com/alan-turing-institute/mogp-emulator/tree/main/mogp_emulator

New Features

New release incorporating recent work. New features in this release:

• Maxi-min LHC Experimental Designs
• Mulit-output history matching
• Emulator validation
• GPU implementation now supports Priors
• Additional fixes and improvements to documentation and examples

SEAVEAtk V0.1

EasyVVUQ v1.1.1

• Documentation: https://easyvvuq.readthedocs.io/en/latest/
• Code: http://www.github.com/UCL-CCS/EasyVVUQ

QCG PilotJob v0.13.0

• Documentation: https://qcg-pilotjob.readthedocs.io/en/latest/
• Code: https://github.com/vecma-project/QCG-PilotJob

EasySurrogate v0.18

• Tutorial: https://github.com/wedeling/EasySurrogate/tree/master/tutorials
• Code: https://github.com/wedeling/EasySurrogate

FabSim3 v3.6

• Documentation: https://fabsim3.readthedocs.io/en/latest/
• Code: http://www.github.com/djgroen/FabSim3

EQI (formerly EasyVVUQ-QCGPilotJob) v0.5.2

• Documentation: https://easyvvuq-qcgpj.readthedocs.io/en/latest/
• Code: https://github.com/vecma-project/easyvvuq-qcgpj

MUSCLE 3 v0.4.0

• Documentation: http://muscle3.readthedocs.io
• Code: https://github.com/multiscale/muscle3/releases/tag/0.4.0

MOGP Emulator v0.6.1

• Documentation: https://github.com/alan-turing-institute/mogp-emulator/tree/main/mogp_emulator/demos
• Code: https://github.com/alan-turing-institute/mogp-emulator/tree/main/mogp_emulator