IRC channel logs

<wdkrnls>I have some questions about the Guix Workflow Language and supporting tools.

<wdkrnls>Are there plans to extend past DRMAA version 1?

<wdkrnls>I am very weary of running out of disk space and wasting time computing things I don't need yet.

<wdkrnls>I would like to be able to periodically rebalance the priorities of active jobs based on some function of the already completed jobs.

<wdkrnls>From perusing the GWL documentation and reading the Wikipedia page, it seems like DRMAA version 1 might not be rich enough semantically to support that.

<wdkrnls>I would also love to be able to garbage collect store items in a way which is inversely proportional to the computational effort.

<wdkrnls>Of course this would have to be done conditionally with regards to a dependency analysis. From that perspective, the relevant computational effort might have to itself be a function of multiple store items.

<wdkrnls>I'm particularly interested in these features for Monte Carlo simulations.

<wdkrnls>It looks to me like SLURM supports a lot of the prioritization features necessary to implement this out of box.

<rekado>wdkrnls: I have no plans to write DRMAA bindings for version 2, but I would certaily accept patches to guile-drmaa

<wdkrnls>Gotcha.

<wdkrnls>My first step is just to send you any patch :p. I tried to send a new version of my r-gpg patch today.

<rekado>excellent!

<rekado>I wonder how much of the DRMAA version 2 API would need to be exposed to the GWL

<rekado>currently with version 1 we just submit and wait

<rekado>not much more to it.

<wdkrnls>Before I saw this DRMAA thing, I was thinking of just using SLURM directly.

<wdkrnls>I would like to have a local system service for it for my own use.

<wdkrnls>It's priority scheduling stuff would be really useful even without a cluster.

<wdkrnls>That also brings to my mind that I was going to write a patch for r-targets as well.

<wdkrnls>That's the workflow system I have been using.

<wdkrnls>However, that one also doesn't provide as much control over prioritization as I would like either.

<rekado>wdkrnls: I’m running slurm as part of the test suite for gwl (or for guile-drmaa? or both?)

<rekado>it’s pretty easy to run locally for testing

<wdkrnls>That's promising. Maybe it would be cool someday to just have a service for it that just works out of box.

<rekado>IIRC efraim may have something for that

<efraim> https://gitlab.com/genenetwork/guix-bioinformatics/-/blob/master/gn/services/science.scm untested

<efraim> https://gitlab.com/genenetwork/guix-bioinformatics/-/blob/master/gn/deploy/octopus.scm and an untested system using that config

<efraim>Compute node, not head node, that one was running debian

<efraim>sometimes I think I need a 'hold my beer' emoji

IRC channel logs

2023-02-17.log