Happy New Year 2022 with two new releases

Bandsaw 0.3 and Multimeter 0.1

Happy new year, everyone! We celebrate occasion by announcing 2 new releases. In the past month we worked on functionality that improves the insight into workflows by collecting metrics, logging and tracking the individual runs. All this work culminated in to two new releases:

Measure resource usages of your python code using “Multimeter”

In december we published a new library “Multimeter” which makes it easy to collect run-time metrics like cpu usage, memory consumption or io operations of python code. The collected data can automatically be pushed to time series databases like InfluxDB and visualized with Grafana.

Visualization of metrics

A more thorough description of the library can be found at our documentation hub.

Track your workflows and their resources

Building upon on “Multimeter”, version 0.3 of our library “Bandsaw” brings new features that allow to track runs of your workflows and the consumption of computation resources of their individual tasks. This gives the possibility to compare individual runs and identify tasks that are bottlenecks in the workflow. The collected metrics and the logging messages for specific executions are stored for later inspection. Here a list with the major new additions:

  • New TrackerExtension to keep track of workflow runs, executions of tasks, their results and attached files.
  • Add new MetricsAdvice that gathers metrics when executing tasks
  • LoggingAdvice adds per-session log file as attachment
  • JsonFormatter for storing log items in a structured format

As always the new version has been published on PyPI and running pip install bandsaw will give you the latest version. The documentation has been updated as well.

Our next plans

Now in January we will focus on the last big missing piece of our fundamental libraries, the data management library.

  • 0.4: Support for tracking data.
  • 0.5: Run tasks asynchronously using a scheduler.

If you like what we are doing or if you have question, please join us on discord.