Our colleague Thomas went to PackagingCon in Berlin in October 2023, below is his summary of the conference.
I arrived at PackagingCon on a rainy day near the end of October in Berlin. The first thing I had to do is find the sticker with my name on.
In the large meetup room, attendees could grab some various drinks and snacks. Additionally, during lunch, a food truck was available.
The conference included several talks covering topics such as securing packages, resolving dependencies through machine learning, the SBOM principle, and many other interesting subjects.
I’ll provide some summaries of a few presentations.
The first talk I attended was about software security and functional safety. The thing is that dependencies could be open source and can be insecure. A solution that they mentioned is SBOM: Software Bill of materials. They could use SBOM standards to install packages. The purpose of an SBOM is to provide transparency and visibility into the software supply chain, helping organizations understand the composition of their software and identify potential security vulnerabilities.
Another talk was focused on RSTUF, a tool designed to enhance the security of downloads from repositories to the client.
Another talk I saw was about you can bootstrap trust for the open-source ecosystem by Trishank Karthik Kuppusamy. How can we know to trust a software package from open source. There are three components to know this: In-toto, The Update Framework (TUF), and Sigstore. That play crucial roles in enhancing the security of the software development and distribution process.
In-Toto – Supply chain:
A software supply chain is a series of steps involved in the creation, testing, and deployment of software. It involves various stages from writing the source code to packaging, publishing, and distribution.
- Who wrote the source code?
- What built the package? Continuous Integration (CI) tools, such as GitHub Actions, contribute to building reliable software packages.
- What package got published? Identifying the platform where the package is published, like PyPI for Python packages.
Those policies will prevent malicious software.
TUF acts as a crucial layer in the supply chain, providing a framework for approving and trusting software packages.
Sigstore:
Sigstore focuses on storing the comprehensive history of a software package, using Merkle tree hash structures for secure and efficient data storage.
Of course, there were also talks not focusing on security as main target. One of the talks I saw: Ensuring Runtime Reproducibility in the Python Ecosystem by Jaime Rodriguez Guerra. They discussed best practices for Python-based data science workflows. For instance, when an end-user shares a Jupyter notebook or script with another user in a different environment, it may not work smoothly due to variations in pip dependencies, operating systems, and libraries.
To enhance this workflow, they proposed a hierarchy of trust levels-six in total, ranging from the highest to the lowest:
- Explicitly state the source of packages and the Python interpreter.
- Provide a lock-file containing all packages. E.g: Use pip freeze to specify package versions.
- Share a tarball with all conda packages.
- Share a Docker image.
- Provide a virtual machine (VM).
- Ship a machine with everything pre-installed.
One solution they suggested is conda-store, which could be explored for implementation in RDI environment. Data scientists can access the necessary tools in a reproducible environment through a web interface.
BuildXYZ: Automatic on-demand dependency dispenser by Ryan Lahfa
Building software is becoming increasingly complex, and minimizing the time spent on software installation is crucial. One approach is to develop a tool that automates the detection of dependencies, so it is no longer needed to manually read the README.md file and search for dependencies.
A potential solution could be creating a FUSE (Filesystem in Userspace) that automatically identifies missing dependencies. This system includes a database designed for tracking missing dependencies.
The proposed tool, named BuildXYZ is a tool with two main parts: the FUSE filesystem and the runner that takes care of your build system. The runner injects environment variables to prompt your program to explore the FUSE filesystem whenever standard search paths fail.
BuildXYZ relies on FUSE and nixpkgs, with all nixpkgs paths stored in a locate database. If a binary is missing from the PATH environment variable, BuildXYZ resolves the issue automatically. This approach streamlines the software installation process by handling missing dependencies without requiring manual intervention.
These are one of the few talks I wanted to highlight.
To finish, I want to mention that it was a really interesting conference related with package managing. I really enjoyed the opportunity to talk with some contributors of package managing systems I use in my daily work environment: e.g. anaconda, Jupyterlab, Spack, EESSI, …
While the conference was the highlight of my visit, I have to admit I also really enjoyed being in the great city of Berlin.