IRC channel logs

<rekado>I got ESS down to 3 test errors.

<rekado>three test failures in ESS

<rekado>especially weird: "Wrong method specification for ‘mock’"

<rekado>the three failures: ess-mock-remote-process, ess-test-r-startup-directory (off by one sub-directory), and test-ess-r-fontification

<rekado>here’s what I’ve got: https://elephly.net/paste/1669807759.diff.html

<zimoun>ah thanks. So you fixed the issue with Tramp, right?

<zimoun>well, I added bash as native-inputs for ess-r-load-ESSR-github-fetch-yes

<zimoun>Is it the test you removed with Fix r-help-mode?

<rekado>ess-r-load-ESSR-github-fetch-yes requires network access

<rekado>setting “TRAVIS” to “true” skips that test

<rekado>there’s still a tramp problem

<rekado>something about the “mock” specification. Not sure what’s up.

<rekado>my daughter’s got a fever again (just after two weeks of all of us having a bad cough), so I’ll be slow to make progress here.

<rekado>I guess setting tramp-encoding-shell breaks the “mock” specification and then we’ll just never get to the point where /bin/sh would be used.

<zimoun>rekado: oh, fever again. I hope all is fine.

<rekado>there’s a respiratory virus going around here. It’s annoying but doesn’t seem to be too dangerous.

<rekado>zimoun: I enabled more debugging output and see this: File error: Couldn't find a proper `ls' command

<rekado>weird!

<rekado>tried setting tramp-remote-path

<rekado>that fixed it

<rekado>got a different test failure… now it’s command-without-trailing-newline-test, ess-test-r-startup-directory, and test-ess-r-fontification

<rekado>I can’t get rid of the latter two test failures

<rekado>the fontification thing may be an actual bug when building with Emacs 28

<rekado>no idea what the ess-test-r-startup-directory test even wants

<rekado>got rid of the first one (it seems to be creeping in depening on how I patch the files…)

<rekado>so… just two test failures

<zimoun>hi!

<zimoun>rekado: oh cool! I have seen your patch fixing ESS. I will try later today. :-)

<rekado>zimoun: it’s not really fixing it, just ignoring problems :-/

<rekado>the TRAMP stuff: sure, that’s fixed. But the other test cases don’t make sense to me.

<rekado>so I tried to be as specific as possible in my attempts to ignore the problems.

<zimoun>civodul: pwoua! done with my presentation in seminar :-)

<zimoun>Guix should run soon in another parisian cluster ;-)

<civodul>zimoun: woow, well done!

<civodul>let's hope you don't receive all the support requests ;-)

<zimoun>yeah, I hope that «community is small» is understood as «long queue for support» ;-)

<zimoun>it is impressive how Conda is popular… and broken for reproducing in the same time.

<zimoun>Ok, now let focus on the next tutorial in Grenoble :-)

<rekado>I don’t mean to be mean by introducing this analogy, but Conda’s popularity is not too different from how pathogens colonize a sanitized surface.

<rekado>for many people Conda was a *huge* step up over the barren software toolshed they found themselves in

<rekado>for some it was “pip install” this and ask the sysadmin to install some other library globally

<rekado>Conda was one of the first tools to be vastly superior to the terribly inconvenient state of software installation tools for scientists

<rekado>ecological succession is a slow process, and sometimes hindered by the success of early colonizers

<rekado>ironically it is often the incredible success of early colonizers that becomes their liability

<rekado>(incidentally, *this* is how I like to use the term “ecosystem”, not as a poor synonym for “market economy”)

<zimoun>«Therefore, the worse-is-better software first will gain acceptance, second will condition its users to expect less, and third will be improved to a point that is almost the right thing.» https://dreamsongs.com/WIB.html

<zimoun>I agree about «*huge* step up over the barren». That’s why I initially used Conda. And it does well this job.

<zimoun>I often miss from questions in audience this point: Conda does X for reproducibility. When Conda is broken by design for reproducing. It is in their manual, section Solver. Ah, a classical RTFM. ;-)

<zimoun>rekado: this morning, I had questions about GWL. Yeah!

<rekado>oh

<rekado>surprising

<rekado>I’m again playing with multi-user rstudio in a Guix container talking to sssd on the host system

<rekado>using pam_sss.so inside the container and mapping the host system’s /var/lib/sss/pipes allows our sss client library to talk the wire protocol with the system’s socket

<rekado>I guess I just need to map all uids and gids into the container to make it work.

<rekado>it’s not a full replacement of nscd, though

<rekado>but it would allow processes in a Guix container to authenticate users from the host system

<zimoun>the questions about GWL were very interesting. Seeing the potential

<PurpleSym>My brain-dump for a blog-post about guix-cran: https://6xq.net/paste/guix-cran.md.html

<PurpleSym>Am I missing anything?

<rekado>looks good. What you could add is a reference to https://cran.r-project.org/web/packages/guix.install/index.html, which automates the first part.

<zimoun_>PurpleSym: ’s/into Guix package descriptions/into Guix package recipes’ otherwise it could be confused with package-description.

<zimoun_>Nice post!

<zimoun_>It could be worth to mention guix.install :-)

<zimoun_>What is the concrete issues about usability (except slowness) for the last paragraph?

<civodul>nice! +1 on the guix.install suggestion

<civodul>might be worth answering the question "why would i bother instead of using plain R + pakrat or similar?"

<civodul>as alluded to in https://hpc.guix.info/blog/2022/07/is-reproducibility-practical/

<civodul>"answering" is a strong word; alluding to it maybe

<civodul>rekado: re pathogen colonization: that's a great analogy :-)

<civodul>(you should write more, you're so at ease with words and ideas)

<zimoun_>civodul, PurpleSym: about title, “CRAN, practical example for being reproducible at large using Guix”

<civodul>or: "The 'R' in 'reproducible'"

<civodul>:-)

<zimoun_>neat!

<PurpleSym>I’ll have a look at guix.install – never used it.

<civodul>rekado: we should discuss your container use case; i tend to think system containers are too much and not in the spirit of "microservices"

<PurpleSym>zimoun_: Well, only that `guix pull` can take 5 minutes. That might be annoying.

<zimoun_>On my machine, it already takes 5 minutes ;-)

<PurpleSym>Okay, five additional minutes. That’s why I wouldn’t recommend using it globally until we fix Guile.

<zimoun_>I see

<PurpleSym>civodul: I’ve never used pakrat, because I’m not an R person. So it’s hard for me to answer that question.

<zimoun_>PurpleSym: I think civodul was doing a suggestion for the title.

<rekado>"The 'R' in 'reproducible'" is a great title!

<rekado>doesn’t say much about the content, so if you don’t pick it I’ll call dibs on using it for my own blog post in the future

<civodul>heh :-)

<civodul>i'm good at finding titles that don't say much about the contents

<PurpleSym>Yeah, I like the title, but I’d have to massage the contents to fit it. If we go for something with “large scale reproducible CRAN” I’d have to add some statistics, which would be nice.

<rekado>PurpleSym: I wonder how this big CRAN channel behaves when packages are added to Guix proper

<PurpleSym>In what sense, rekado ? Packages added to Guix proper are removed on the next update.

<rekado>there was a package with an interactive check phase, for example, so the build was never completed. I then added the package (with a fixed check phase) to Guix.

<rekado>I see

<civodul>(incidentally, i wasn't aware of the 8K module limit, that's... interesting)

<PurpleSym>I had a version, which kept references to the packages in Guix proper. But in an effort to reduce the number of exports I removed them. Would have been nice to not break existing manifests when moving, but it happens all the time anyway, so -.-

<PurpleSym>rekado: Unfortunately this split between guix-cran and Guix proper sometimes causes version conflicts, with packages in guix-cran requiring newer versions of packages in Guix proper.

<rekado>civodul: re microservices: yes, the system container is a bit heavy for what I want to accomplish

<rekado>but “guix shell -C” is too light

<rekado>I think it would be useful for the run-container script to gain the ability to set up a network device inside the container.

<rekado>the process to create a veth pair and move one end into the container (as I describe in the cookbook) is pretty awkward

<civodul>yes

<rekado>I punted on network interface handling; generated the container launcher with “--network” (this should really be an option for the run-container launcher, not for “guix system container” which generates the launcher), set LD_LIBRARY_PATH for sssd in multiple locations, shared the host system’s /var/lib/sss/pipes, fiddled with pam… and it works

<rekado>we now have a multi-user RStudio on our server, running in a reproducible container, talking to the host system’s sssd.

<civodul>neat

<civodul>we'll have to see what we can extract and generalize from this

<civodul>i'm particularly curious about sssd

<rekado>the LD_LIBRARY_PATH thing is a little sticky

<rekado>I would have wanted to just set it through extension of nscd-service-typ

<rekado>*type

<civodul>ok

<rekado>but that’s not enough for even something like “id”

<rekado>so for “id” to work I had to add LD_LIBRARY_PATH=/gnu/store/…sssd…/lib to /etc/environment

<civodul>uh, ok

<civodul>ACTION -> zZz

<rekado>good night!

<civodul>night!

<rekado>but that was still not enough for the rstudio server service type; had to change it to allow for setting LD_LIBRARY_PATH explictly

<rekado>here’s all I did: https://elephly.net/paste/1669937572.html

<civodul> https://fzakaria.com/2022/09/12/making-runpath-redundant-for-nix.html

<civodul>Shrinkwrap was inspired by https://guix.gnu.org/en/blog/2021/taming-the-stat-storm-with-a-loader-cache/

<civodul>meaning we indirectly contributed to SC22 :-)

<rekado>heh :)

<rekado>no mention of Guix’s solution there

<rekado>sad face

<civodul>there's a link to a previous blog post that mentions it in passing

<civodul>but yeah

<civodul>ironically https://fzakaria.com/2022/03/15/shrinkwrap-taming-dynamic-shared-objects.html says Shrinkwrap could be integrated in Guix

<civodul>which is true, but in practice Shrinkwrap came after the ld.so cache was in Guix

<rekado>not sure if you saw this, but here’s all I did to make the Guix container work with the host’s sssd: https://elephly.net/paste/1669937572.html

<civodul>i hadn't seen it, nice!

<civodul>i'm surprised sharing /var/lib/sss/pipes is enough

<civodul>it contains named sockets, right?

<civodul>AFAIK they cannot be shared over 9p

<civodul>(rather: you can share them but they're not connected)

<zimoun>hi!

<zimoun>is the logs still messed?

<zimoun>Yes, there are https://logs.guix.gnu.org/guix-hpc/2022-11-30.log

<zimoun>Since we are not 2022-11-30 but 2022-12-02. Or do I live in another time frame?

<rekado>bleh

<rekado>it’s supposed to close the old log file and start a new one as soon as it gets a message on a new day

<rekado>seems like it’s still writing to the old file handle

<rekado>I can’t investigate this today

<rekado>this is the relevant procedure: https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/hydra/goggles-bot.scm#n96

<zimoun>it is also the case for #guix

<rekado>yeah, same program after all

<zimoun>thanks. I do not know if I would have the time today.

<rekado>it keeps track of days and ports per channel

<rekado>I wonder what happens when ports and days are modified — how does this affect day-for-channel and port-for-channel, which capture the ports and days variables via closure.

<zimoun>civodul, rekado: re about “stat storm”, I would be interested to know the performances between caching in “binary header” vs caching “per-application“

<civodul>binary header?

<zimoun>that’s what Shrinkwrap does, IIUC.

<zimoun>«Shrinkwrap adopts this approach by freezing the required dependencies directly into the DT_NEEDED section of the binary by having it point to an absolute path.»

<zimoun> https://fzakaria.com/2022/03/15/shrinkwrap-taming-dynamic-shared-objects.html

<civodul>Shrinkwrap changes DT_NEEDED instead of DT_RUNPATH

<civodul>in terms of stat calls it doesn't make any difference

<civodul>it renders LD_LIBRARY_PATH unusable though

<zimoun>My understanding was that Guix uses DT_RUNPATH to point to ld.so.cache (I did not know about DT_RUNPATH :-)). But Shrinkwrap embeds all the paths in DT_NEEDED.

<PurpleSym>Hm, looks like Cuirass broke for guix-cran. It’s not showing a percentage of successful derivations, the last derivation did not yield any new builds (despite changes to packages) and the dashboard is not showing any builds (https://guix.bordeaux.inria.fr/eval/5368689/dashboard).

<rekado>ACTION plays with ChatGPT

<rekado>this thing is really impressive

<nckx>Release song?

<rekado>okay, these lyrics are beyond terrible

<civodul>one has to create an account no?

<civodul>i've seen super impressive things

<rekado>catchy: ‘Guix, Guix, the future is here\nGuix, Guix, the best package manager, my dear’

<rekado>roll eyes emoji

<civodul>uh :-)

<civodul>maybe it could write the release notes?

<rekado>it does agree, though, that ‘GNU/Linux’ would be more accurate than ‘Linux’.

<rekado>‘the name "GNU/Linux" more accurately reflects the fact that the operating system is a combination of the Linux kernel and the GNU userland, and recognizes the important role played by the GNU Project in its development and evolution.’

<rekado>it keeps regurgitating ‘Guix, Guix, <corny platitude>’ in poems and lyrics. It’s great.

<rekado>this one is not too bad: https://elephly.net/paste/1670104031.html

<civodul>woow!

<civodul>maybe machine learning will let us all become poets

<civodul>i like this one anyway

<civodul>can't wait to see the t-shirt :-)

<rekado>what a great toy this is! https://elephly.net/paste/1670104466.html

<civodul>this one's tragic :-)

<rekado>it builds up all that hope only to come crashing down rapidly in the last few lines

<rekado>so blunt, so raw

<nckx>I like it.