IRC channel logs


back to list of logs

<rekado>hmm, snakemake now also generates Docker things:
<zimoun>ouch! Using a generate Dockerfile? Using Conda as package manager?
<zimoun>snake folks should have missed the Zen of Python: “python -c 'import this'” ;-)
<rekado>looks like it does a few different things here, including the generation of an archive
<rekado>bah, I’m too slow
<rekado>no time to hack :-/
<rekado>I wonder what it would take to package this for Guix:
<rekado>the mention of Tensorflow bindings makes me quiver in fear
<civodul>rekado: re Snakemake, Docker *and* "integrated package management"
<civodul>you GWL folks didn't invent anything!
<civodul>i remember reading about aiscm in the past, looks interesting
<civodul>it could be that the TF bindings are optional
<civodul>apparently it's a separate module, anyawy
<zimoun>well, it could cool to have something like Zygote for Guile
<zimoun>source-to-source automatic differentiation (AD), where "Source-to-source" means that Zygote hooks into Julia's compiler, and generates the backwards pass for you.
<civodul>crazy that differentiation can have a meaning on functions on general
<zimoun>you need at leat an arithmetic though; a ring I guess, maybe less.
<civodul>colleagues would like to have metis with REALTYPEWIDTH = 64
<civodul>perhaps we could do like scotch/scotch32
<civodul>bavier[m]: ↑
<bavier[m]>I think for metis, many use REALTYPEWIDTH=64 as the default. I have not heard anyone complain about a meaningful difference in performance.
<rekado>civodul: reading the snakemake documentation always makes me sigh and grunt
<rekado>that’s probably why I avoid it and only hear of their features too late
<bavier[m]>civodul: and the scotch/scotch32 is to differentiate the integer type size, which is a bit different. I'm fine either way though.
<rekado>I really should work on this “guix workflow pack” thing
<rekado>I thought about it when I had just finished translating the pigx-rnaseq workflow and then noticed that the rnaseq process scripts were not statically determined (they depend on the pigx configuration file / sample sheet), so that made me defer work on this feature.
<rekado>for really simple, static workflows this is trivial to implement
<rekado>I’m guessing that the PiGx case is more common, and perhaps this means that some changes to GWL are necessary.
<civodul>i suppose "guix workflow pack" could boil down to creating a pack around the entry-point script, no?
<civodul>sounds like worth giving it a try
<civodul>bavier[m]: perhaps we should change the default to REALTYPEWIDTH=64 then, and add a 32-bit variant?
<bavier[m]>civodul: yes. We should consider the naming carefully though, to not confuse users, since metis also has a 32/64-bit choice in index sizes.
<bavier[m]>changing the real width doesn't fit nicely into the "familiar" ilp64-type naming.
<rekado>civodul: my expectation for “guix workflow pack” is that Guix would not be required at runtime.
<rekado>but if the job scripts depend on user inputs, then we need Guix to compute them at runtime.
<rekado>that’s what I remember from my experiment with pigx-rnaseq
<rekado>I think it’s just a matter of finding the right “level” to shift the code to, but as it stands the pigx-rnaseq workflow creates *different* job scripts dependent on the contents of the configuration file
<rekado>just as I moved the values of inputs and outputs outside of the job scripts (they are now passed as arguments to the script instead of being embedded), I hope I can do the same for whatever causes these job scripts to be variable.
<rekado>(oof, I hate being so vague)
<rekado>in other news: Guile AWS really works and looks much cleaner now.
<rekado>I’m still tweaking the conversion of API return values; takes me longer because I don’t know what exactly I want.
<rekado>I found that the Scheme values that I generate are a little awkward to work with, compared to the plain JSON that I could use instead.
<rekado>we now have Guile DRMAA and Guile AWS in a state that is probably sufficient for the needs of the GWL
<zimoun>rekado: awesome! Really cool. And you said earlier «no time to hack». Wow! what it should be with time to hack ;-)
<rekado>heh :)
<rekado>I miss being able to waste a lot of time losing myself in hacking
<rekado>I try to squeeze in a hack session here or there, but I need to be much more efficient than I like to be
<zimoun>I know what you mean. Sometimes life happens. :-)
<rekado>it certainly isn’t bad, that whole life part. But it becomes very noticeable to me how limited my remaining time is, and it forces me to consider what paths I will have to leave unexplored.
<rekado>a bit like the game “passage”.
<civodul>bavier[m]: right; a colleague of mine mentioned this nomenclature: (fr)
<civodul>"i4r8" for 32-bit indexes and 64-bit reals
<civodul>maybe a little bit heavyweight tho
<civodul>package parameters would come in handy
<civodul>rekado: re moving inputs outside of the scripts, i see what you mean (i think we discussed it earlier)
<civodul>that makes sense
<civodul>kudos on the progress on DRMAA and AWS!
<civodul>that could make for lovely demos