Guix for scientific computing
--------------------------------------------------------------------------------------------------------------------
14h00 - 14h45 Ricardo Wurmus (Max Delbrück Center, Germany):
A decade of Guix - a subjective retrospective
Almost a decade ago a Berlin research institute looked for help in
compiling scientific software for bioinformatics researchers. A
systems engineer from Shanghai with a yearning for a simpler life
answered the call. Little did he know that his long repressed
infatuation with a quaint programming language would soon resurface,
sparking his fantastically deterministic journey away from the
traditions of the sysadmin tribe and just beyond the brink of the
cutting edge.
This is the story of a quest for predictability, reproducibility, and
stability through unpredictable means, ad-hoc hacks, and an embrace
of the quirky. Based on actual events.
Keywords: research and HPC; Guix project infrastructure; round PiGx
and square holes; Guix Workflow Language; the unreasonable allure of
simple abstractions.
--------------------------------------------------------------------------------------------------------------------
14h45 - 15h30 Arun Isaac (Department of Genetics, Evolution & Environment, University College London, United Kingdom):
Why we deploy and develop using Guix and why you should too - an experience report from GeneNetwork
Scientific software is increasingly complex, but is developed on a shoestring budget. Maintaining a reproducible
development environment for all developers and running robust deployments is challenging, to say the least.
Wouldn't it be nice to have a tool that does it all and does so correctly?
The traditional Unix way to deploy complex web applications and provision servers is to manually mutate
configuration files on the server. Such an approach is brittle, time consuming and hard to migrate to new machines.
Tools as varied as Ansible and Docker have been developed to ease this process, but these tools are still
mutation based and their abstractions leak in unexpected ways. Guix, with its "functional" package deployment
provides the watertight abstractions necessary to express complex deployments with precision.
In this talk, I will present how we deploy development and production environments using Guix at genenetwork.org.
I will show how we use Guix channels to distribute our own packages and services; and how we run continuous
integration and deployment (CI/CD) using Guix. I will explain how this enables us to further software quality in
science, and will hopefully be able to convince you to use more Guix in your team.
--------------------------------------------------------------------------------------------------------------------
15h30 - 16h15 Lars-Dominik Braun:
PsychNotebook is intended as…
PsychNotebook was a web platform for students and scientists providing
access to shareable and reproducible programming environments including
RStudio and JupyterLab. It was developed and operated by Leibniz Institute
for Psychology between 2019 and 2023.
In this talk I will review why PsychNotebook was built, which components
we used and built ourselves, why we chose them and how they interacted
with each other as well as how the platform was kept running. I will
also discuss why, ultimately, the service was shut down and what can be
learned from its technical and organizational design.
--------------------------------------------------------------------------------------------------------------------
16h30 - 17h15 Yann Dupont (GLiCID, France):
Reproducible VM management with Guix
--------------------------------------------------------------------------------------------------------------------
17h15 - 18h00 Emmanuel Agullo (Inria, France):
Reconciling high-performance computing with the use of third-party libraries?
High-performance computing (HPC) often requires the use of multiple software
packages and that they are optimised on the target machine on which these
computations are running. The optimisation constraints are such that it is
widely accepted that the deployment of this software can only be done manually
or by relying on the work of the administrators of the target machine (typically
via a load module). However, the complexity of the dependencies often results in
strong constraints on the exact functionality, version and build processes of
the requested software. As a result, many HPC codes choose to provide some
functionality themselves, which in principle could be provided by third party
libraries, contrary to the canons of software engineering.
In this talk, we will first review our quest (CMake, Spack, and now Guix) for an
environment to reliably deploy HPC software in a portable, high-performance, and
reproducible way, so that the use of third-party libraries is no longer a
concern. We will then present our experience of deploying such a complex
software stack on several machines in the French and European ecosystem. We show
that we have been able to ensure a robust deployment while achieving top
performance, not only on machines with guix available but also on supercomputers
where guix is not (yet!) available.
------------------------------------------------------------------------------------------------------------------------------
