IRC channel logs


back to list of logs

<wdkrnls>Hi all
<wdkrnls>I have some questions about the Guix Workflow Language and supporting tools.
<wdkrnls>Are there plans to extend past DRMAA version 1?
<wdkrnls>I am very weary of running out of disk space and wasting time computing things I don't need yet.
<wdkrnls>I would like to be able to periodically rebalance the priorities of active jobs based on some function of the already completed jobs.
<wdkrnls>From perusing the GWL documentation and reading the Wikipedia page, it seems like DRMAA version 1 might not be rich enough semantically to support that.
<wdkrnls>I would also love to be able to garbage collect store items in a way which is inversely proportional to the computational effort.
<wdkrnls>Of course this would have to be done conditionally with regards to a dependency analysis. From that perspective, the relevant computational effort might have to itself be a function of multiple store items.
<wdkrnls>I'm particularly interested in these features for Monte Carlo simulations.
<wdkrnls>It looks to me like SLURM supports a lot of the prioritization features necessary to implement this out of box.
<rekado>wdkrnls: I have no plans to write DRMAA bindings for version 2, but I would certaily accept patches to guile-drmaa
<wdkrnls>My first step is just to send you any patch :p. I tried to send a new version of my r-gpg patch today.
<rekado>I wonder how much of the DRMAA version 2 API would need to be exposed to the GWL
<rekado>currently with version 1 we just submit and wait
<rekado>not much more to it.
<wdkrnls>Before I saw this DRMAA thing, I was thinking of just using SLURM directly.
<wdkrnls>I would like to have a local system service for it for my own use.
<wdkrnls>It's priority scheduling stuff would be really useful even without a cluster.
<wdkrnls>That also brings to my mind that I was going to write a patch for r-targets as well.
<wdkrnls>That's the workflow system I have been using.
<wdkrnls>However, that one also doesn't provide as much control over prioritization as I would like either.
<rekado>wdkrnls: I’m running slurm as part of the test suite for gwl (or for guile-drmaa? or both?)
<rekado>it’s pretty easy to run locally for testing
<wdkrnls>That's promising. Maybe it would be cool someday to just have a service for it that just works out of box.
<rekado>IIRC efraim may have something for that
<efraim> untested
<efraim> and an untested system using that config
<efraim>Compute node, not head node, that one was running debian
<efraim>sometimes I think I need a 'hold my beer' emoji