IRC channel logs

<zimoun>hi!

<rekado>hmm, snakemake now also generates Docker things: https://snakemake.readthedocs.io/en/stable/snakefiles/deployment.html#containerization-of-conda-based-workflows

<zimoun>ouch! Using a generate Dockerfile? Using Conda as package manager?

<zimoun>snake folks should have missed the Zen of Python: “python -c 'import this'” ;-)

<rekado>looks like it does a few different things here, including the generation of an archive

<rekado>bah, I’m too slow

<rekado>no time to hack :-/

<rekado>I wonder what it would take to package this for Guix: https://wedesoft.github.io/aiscm/

<rekado>the mention of Tensorflow bindings makes me quiver in fear

<civodul>rekado: re Snakemake, Docker *and* "integrated package management"

<civodul>you GWL folks didn't invent anything!

<civodul>i remember reading about aiscm in the past, looks interesting

<civodul>it could be that the TF bindings are optional

<civodul>apparently it's a separate module, anyawy

<zimoun>well, it could cool to have something like Zygote https://github.com/FluxML/Zygote.jl for Guile

<zimoun>source-to-source automatic differentiation (AD), where "Source-to-source" means that Zygote hooks into Julia's compiler, and generates the backwards pass for you.

<civodul>fun

<civodul>crazy that differentiation can have a meaning on functions on general

<zimoun>you need at leat an arithmetic though; a ring I guess, maybe less.

<civodul>colleagues would like to have metis with REALTYPEWIDTH = 64

<civodul>perhaps we could do like scotch/scotch32

<civodul>thoughts?

<civodul>bavier[m]: ↑

<bavier[m]>I think for metis, many use REALTYPEWIDTH=64 as the default. I have not heard anyone complain about a meaningful difference in performance.

<rekado>civodul: reading the snakemake documentation always makes me sigh and grunt

<rekado>that’s probably why I avoid it and only hear of their features too late

<bavier[m]>civodul: and the scotch/scotch32 is to differentiate the integer type size, which is a bit different. I'm fine either way though.

<rekado>I really should work on this “guix workflow pack” thing

<rekado>I thought about it when I had just finished translating the pigx-rnaseq workflow and then noticed that the rnaseq process scripts were not statically determined (they depend on the pigx configuration file / sample sheet), so that made me defer work on this feature.

<rekado>for really simple, static workflows this is trivial to implement

<rekado>I’m guessing that the PiGx case is more common, and perhaps this means that some changes to GWL are necessary.

<civodul>i suppose "guix workflow pack" could boil down to creating a pack around the entry-point script, no?

<civodul>sounds like worth giving it a try

<civodul>bavier[m]: perhaps we should change the default to REALTYPEWIDTH=64 then, and add a 32-bit variant?

<bavier[m]>civodul: yes. We should consider the naming carefully though, to not confuse users, since metis also has a 32/64-bit choice in index sizes.

<bavier[m]>changing the real width doesn't fit nicely into the "familiar" ilp64-type naming.

<rekado>civodul: my expectation for “guix workflow pack” is that Guix would not be required at runtime.

<rekado>but if the job scripts depend on user inputs, then we need Guix to compute them at runtime.

<rekado>that’s what I remember from my experiment with pigx-rnaseq

<rekado>I think it’s just a matter of finding the right “level” to shift the code to, but as it stands the pigx-rnaseq workflow creates *different* job scripts dependent on the contents of the configuration file

<rekado>just as I moved the values of inputs and outputs outside of the job scripts (they are now passed as arguments to the script instead of being embedded), I hope I can do the same for whatever causes these job scripts to be variable.

<rekado>(oof, I hate being so vague)

<rekado>in other news: Guile AWS really works and looks much cleaner now.

<rekado>I’m still tweaking the conversion of API return values; takes me longer because I don’t know what exactly I want.

<rekado>I found that the Scheme values that I generate are a little awkward to work with, compared to the plain JSON that I could use instead.

<rekado>we now have Guile DRMAA and Guile AWS in a state that is probably sufficient for the needs of the GWL

<zimoun>rekado: awesome! Really cool. And you said earlier «no time to hack». Wow! what it should be with time to hack ;-)

<rekado>heh :)

<rekado>I miss being able to waste a lot of time losing myself in hacking

<rekado>I try to squeeze in a hack session here or there, but I need to be much more efficient than I like to be

<zimoun>I know what you mean. Sometimes life happens. :-)

<rekado>yeah.

<rekado>it certainly isn’t bad, that whole life part. But it becomes very noticeable to me how limited my remaining time is, and it forces me to consider what paths I will have to leave unexplored.

<rekado>a bit like the game “passage”.

<civodul>bavier[m]: right; a colleague of mine mentioned this nomenclature: http://calculs.unice.fr/fr/assistance/environnement-de-travail/logiciels/metis (fr)

<civodul>"i4r8" for 32-bit indexes and 64-bit reals

<civodul>maybe a little bit heavyweight tho

<civodul>package parameters would come in handy

<civodul>rekado: re moving inputs outside of the scripts, i see what you mean (i think we discussed it earlier)

<civodul>that makes sense

<civodul>kudos on the progress on DRMAA and AWS!

<civodul>that could make for lovely demos

IRC channel logs

2021-03-12.log