Kantai Homepage
blog

First release of new library "Bandsaw"

Finally, we got our first version of one of our products out of the door. Yesterday we successfully built and published version 0.1 of our new library “Bandsaw”.

Bandsaw will be the foundation of our products. It is a python library for splitting up a python workflow into individual tasks, that can be run separately and on different platforms. The idea behind it is that breaking a complete workflow into smaller pieces allows to optimize each individual task independently of each other.

Our initial release is not feature complete yet, but it should allow to give some idea about the direction we are heading. With 0.1 the library supports already workflows that run on a single machine. Here is a short list of features that are already available:

  • Splitting workflow into smaller tasks
  • Adding additional logging to each task
  • Cache individual task results in file system to speed up multiple runs.
  • Running tasks in sub processes for better isolation
  • Use different python interpreters per task which allows combining different frameworks with conflicting dependencies.

The package has been published on PyPI, so installing it is as easy as running pip install bandsaw. Additionally, we published documentation including a detailed user guide that explains the inner workings of bandsaw. For all of you people who want to “use the source”, the code can be found in our GitLab repository, enjoy. The code is released under the MIT license, so there shouldn’t be any problems with trying it out.

Now, what’s next for bandsaw? There is still some way to go until we are feature complete and can release our version 1.0, but we think that it won’t take us too long with the foundations being in place. Our preliminary roadmap looks like this:

  • 0.2: Support for remote execution of tasks using SSH.
  • 0.3: Track computation resource metrics.
  • 0.5: Run tasks asynchronously using a scheduler.

So much for now and our plans. Have fun and don’t hesitate to reach out for questions, ideas or all kind of feedback.

Cheers,

Christoph

References

2021-10-29