Can anyone recommend me a good workflow management system (WMS), preferably in Python? So far I have been using GNU Make, but it introduces a layer of complexity that I want to avoid. A good WMS should have the following features:integrate easily with command line tools and Python scripts,simple to use and lightweight,handle dependencies,provide command line interface,provide logging mechanism,(optional) provide data provenance.I know that WMS are very popular in bioinformatics (for example Galaxy), but I am looking for something more general.
Scientific workflow management system
software;python
For logging that allows full reproducibility, I highly recommend the Sumatra python package. It nicely links the version control commit number, machine state, and output files to each program run and has a django web interface to interact with the database of run info. The python API makes it very easy to include logging in my scripts.