ICHEC Common Tools

ICHEC Common Tools#

The ICHEC Common Tools (ictools) are a collection of software libraries that are intended as a concrete example of the Continous Improvement and Software Engineering best practices introduced in previous Sections.

../_images/common_tools_overview.png

Fig. 21 An overview of the elements of ICHEC common tools.#

With reference to Fig. Fig. 21, the ‘common tools’ stem from the idea of trying to have a common set of processes for Projects and Activities undertaken at ICHEC. Some processes or areas where having standardized or template-based approaches may be useful include:

  • Data Management: How we store, transfer and publish data at different stages in its lifecycle

  • Analytics workflows: How we do data processing, parameters sweeps and machine learning

  • Bechmarking: How we assess the performance of codes or hardware systems

  • Troubleshooting: How we help users identify and fix issues in their simulations

  • CI/CD Environment: How we test our code for correctness and release it for use by others

  • Dev Environment: How we set up our development machines to take advantage of modern tooling

  • Reporting Templates: How we report our results in a consistent and professional way

  • Handbook: How we collect and transfer knowledge in the centre

Software, templates and automations can help use standardize and improve our processes in these areas. The ictools, collected here is one collection of software and templates to help us do this.

The idea of these tools is that as we work on Projects or on certain Activities we gradually refactor our code and workflows so that they start moving them into these common tools. For example, the Hestia project involved building a sophisticated CI/CD pipeline - which would be useful for many other projects. This has been gradually refactored into the ictool module - meaning it can be used by others. Some goals of this approach are:

  • It encourages us to work together - reducing siloing

  • It allows knowledge transfer through seeing other’s code and doing code reviews

  • It avoids waste when projects finish

  • It avoid duplicating work in projects - they can re-use previously built tooling more easily

  • Common software engineering standards can be adopted - including testing and documentation

  • We get practice working in shared codebases and can transfer between projects more easily.

  • We can standardize our approaches and workflows - since we can see what everyone else is doing.

Standarization of this type can come with some downsides and challenges however, so it should be evaluated on a case-by-case basis whether relying on these tools is suitable for a particular project. Some things to keep in mind include:

  • APIs and quality levels are not stable, since the concept is at an early stage

  • Some packages will include dependencies not needed by all projects

  • There can be a tendency to re-invent existing libraries and software - although integrating existing software through interfaces or plugins is encouraged.

Overall, as covered in the Continous Improvement section in this Handbook - adopting standarized tooling and automation can lead to worse outcomes if the steps involved aren’t fully understood and there isn’t a long-term determination to adopt these processes. So, if you are interested in adopting these tools in their current state it is likely you will need to get your hands dirty contributing and fixing broken functionality or awkward APIs.

Package Overviews#

This section overviews the packages in the common tools, which are all available here. Domain agnostic tooling includes:

  • iccore: This has low level utilities that are useful across many modules. Examples include reading and writing json or yaml, setting up logging or making http requests.

  • icsystemutils: This has functionality for interacting with real system resources, such as querying CPU and memory use. It also has networking functionality, such as running commands on remote machines.

  • icplot: This has functionality for generating plots and graphics and converting between common image formats. It is used to generate the graphics in this handbook.

  • icreports: This has code and templates for generating technical documents, such as reports or this handbook.

  • iccicd: This has functionality for building CI/CD pipelines and packages for specific languages. For example it handles version incrementing Python projects and deployment to PyPI.

  • ictasks: This supports running many small tasks in batch, similar to taskfarm - however it allows more customization.

  • icflow: This supports building data analytics and compute workflows. Concrete examples include a machine learning workflow and parameter sweep.

Domain specific tooling includes:

  • icquantum: This package is for building quantum circuit simulations - it is particularly focused on benchmarking studies.

Example Applications#

This section has some example applications of using ictools in ICHEC projects.

Quantum Simulation Benchmarking#

The first example, sketched in Fig. Fig. 22 is the DQUL Project - an element of which involves running batches of quantum circuits in Qiskit. We wanted to understand the best parameters for running circuit batches on a HPC system based on circuit characteristics.

../_images/common_tooling_dqul.png

Fig. 22 Use of common tools in the DQUL project.#

Work was originally done in an internal DQUL repo. We then moved generic functionality for performing parameter sweeps into icflow and let ictasks handle launching of individual circuit simulations. We pulled functionality for running benchmark quantum circuits into a common domain-specific package icquantum - since it may be useful in other projects. Plotting code was moved into icplot. We also created a tutorial in the Quantum Recipes repo allowing ICHEC or external uses to replicate the quantum circuit benchmarks themselves. In the end, the DQUL repo was left with little project-specifc code - mostly just configuration files for performing benchmarks and postprocessed results. Most code can be re-used by other projects and is executed in CI/CD driven tests.

Fig. Fig. 23 shows the quantum circuit benchmark in more detail - with classes used from the various ictools packages. A YAML config file is passed to a ParameterSweep from icflow. This generates a collection of Tasks, defined in ictasks. The ictasks package launches these tasks on Workers, which correspond to a resource such as a CPU - the number of which can be controlled by the input config or detected from the system environment using icsystemutils. Tasks are assigned to a worker from a queue - using code refactored from ICHEC’s taskfarm - and launched. In this case the config specifies that the benchmark feature in icquantum will be called for each task, with arguments from the parameter ranges defined in the config. A directory is created for each task, with a structured output format for results.

../_images/common_tooling_dqul_details.png

Fig. 23 Specific classes and functionality used in the DQUL-ictools refactor.#

Once results are generated an analysis stage can be run. This takes a ‘postprocessing’ config and passes it with the results location to the benchmark-postprocess feature in icquantum. This uses the Task object from ictasks and plotting tools from icplot to generate plots or other results as specified in the postprocessing config.

In summary - the code developed in the DQUL projects was gradually refactored and made more generic. This allows its use in other projects and gives an example format for structured analysis output.

Machine Learning#

The next example, sketched in Fig. Fig. 24 is the SEODA project. The SEODA project involves research toward the development of a scalable data analytics pipeline for the Earth Observation domain. An element of this project involves training and benchmarking a variety of machine learning models.

../_images/common_tooling_seoda.png

Fig. 24 Use of common tools in the SEODA project.#

Work has been carried out in the SEODA repo. Reference EO machine learning models were originally given in the form of Jupyter notebooks - in various formats and from various sources. A common structure was identified for all notebooks and included in the icflow project. This allows training and running inference on different machine learning models and for different datasets with relatively little code modification. It also allows for more structured definition of model configuration and model outputs - although not to the degree possible in the DQUL project yet. The icflow:MachineLearningSession is the core functionality moved from SEODA, and is sketched in Fig. Fig. 25.

../_images/common_tooling_seoda_details.png

Fig. 25 Overview of the MachineLearningSession class in icflow.#

The use of ictools in SEODA is not as far along as for the DQUL project. Some outstanding steps that would be useful from a standardization perspective include:

  • creation of a public machine_learning recipes repo - similar to the quantum one with basic tutorials

  • creation of an iclearn project similar to icquantum that removes domain specific dependencies (particularly pytorch) from icflow.

  • support for a fully structured input config via YAML and structured output directories and content

This would allow common code and approaches from both projects to be adopted and for contributors to switch between projects or work on both more easily.

Contents