IRC channel logs

2020-09-25.log

back to list of logs

***rekado_ is now known as rekado
<civodul>the "Tier 1" and "Tier 2" supercomputer folks in .fr: first one says they use Spack to generate modules, second one says they do things by hand but plan to use Spack and also generate containers with it (?)
<civodul>it's a bit depressing
<civodul> https://spack.readthedocs.io/en/latest/containers.html
<civodul>we need to work on it, comrades
<zimoun>civodul: right! On the other hand, they is Modulefiles main developper. :-)
<civodul>right, but speaking on behalf of the Tier 1 supercomp, so...
<civodul>but we need to think about what we're doing wrong compared to Spack (& CONDA)
<civodul>i think communication and training is one thing
<civodul>then there's lack of political support (not necessarily "our" fault, but...)
<zimoun>Political support? I remember the Cyclades projects. ;-)
<civodul>heh
<rekado>AFAIK Spack has just dedicated advertising people everywhere
<rekado>they are at all conferences that matter
<rekado>even our HPC admins who hadn’t even heard of modules (!) heard of Spack at one of those hot air conferences
<rekado>another thing they do that we don’t: they don’t mention drawbacks
<rekado>ever
<rekado>I haven’t seen any Spack presentation that talked about alternatives, limitations, future work, etc
<rekado>maybe it’s because I haven’t seen much, but their advertising material comes across as just a tad arrogant
<rekado>I think we’ve been doing very well in absorbing Docker and Singularity with “guix pack”
<rekado>our messaging here is effective: these container images are just lower level representations of our much more maintainable and transparent graph
<rekado>this is easy to understand
<rekado>and just yesterday even one of our least technical users at the MDC used “guix pack” for a publication to let the overwhelmed reviewers use their Docker thing
<rekado>(the reviewers said: could you, like, maybe use Conda instead of this SCM stuff? Because you can use Conda in Docker, you know.)
<rekado>(“SCM” capitalized, because it’s the new “GUIX”, I’m sure)
<rekado>however: our messaging wrt Spack and Conda is much less clear
<rekado>is Guix an alternative to these that gives you just a bit more “reproducibility” in edge case but is also harder to install because of that daemon and root and all that?
<rekado>that’s what I heard often
<rekado>maybe that’s *their* message
<rekado>you ain’t gonna need it
<rekado>and it’s really effective
<rekado>I heard claims from that one prolific Galaxy dude that Conda gets you 99% of the reproducibility you need
<rekado>and like 74.3% of all statistics that one was also made up on the spot
<rekado>that was at a bioinfo conference, right before my talk
<rekado>remember the idea of The Irreproducible Paper? Maybe it’s time for that.
<rekado>looking at this: https://spack.readthedocs.io/en/latest/containers.html — it’s absolutely terrible!
<rekado>so it generates an ugly Dockerfile from your current environment?
<rekado>we could do that – but does anyone really want that?
<rekado>perhaps they do; people like to have a magic snapshot/capture button that does what they mean
<civodul>hey rekado!
<civodul>isn't our "advertising material" (perceived as) arrogant too? :-)
<civodul>i don't know
<rekado>only by Holger ;)
<civodul>oh right :-)
<rekado>your presentations always include limitations, open issues, etc
<rekado>not saying that’s bad
<rekado>at all
<civodul>but yeah, i think you're write: Guix vs. containers is becoming quite "well" understood
<civodul>Guix vs. Spack maybe less so
<civodul>yeah
<civodul>but in HPC/"science", i think network effect is super important
<civodul>bioinfo people use CONDA because bioinfo people use it
<rekado>yes
<civodul>HPC (of the linear algebra/numerical simulation kind) people use Spack likewise
<rekado>can Guix swallow these tools?
<rekado>if Guix could output Conda recipes – would that help or be self-defeating?
<civodul>i don't see how that would help
<civodul>producing container images has a direct benefit
<civodul>put Conda recipes?
<civodul>*but
<rekado>yes, it’s true
<rekado>hmm
<rekado>Conda is kinda useless on its own :)
<civodul>heh :-)
<civodul>but people already have conda packages, no?
<civodul>what might help for the HPC folks is module files
<civodul>for instance, in that big thread, there were admins explaining they use Spack to generate module files, period
<civodul>users don't even know it's Spack
<zimoun>I was using Conda in PDE/Linear Algebra stuff before because it was super easy to get a Python environment (Scipy & co.) and easy to share. But it was before I see the light. :-)
<zimoun>One really cool feature Guix provide is: “guix pack -f docker” and then the Docker image contains the profile/manifest so even this opaque blob is not and one can reproduce it without the initial Guix material
<zimoun>“docker export” to tarball and then it is easy to extract the profile/manifest.
<zimoun>The missing piece is a converter from profile/manifest to manifest.scm+channels.scm
<civodul>yeah, people asked me about that again recently
<civodul>it's getting higher in my to-do list :-)
<zimoun>yesterday, I have started a tiny script to do so.
<civodul>nice!
<zimoun>But my TODO list is a bit to big. :-)
<zimoun>Well IMHO, (wider) communication on topic that Guix can do (easily) that other tools. And one thing is (re)generate Docker images without Dockerfile.