IRC channel logs

<rekado_>I have only ever used it to keep people from yanking on audio cables.

<rekado_>it’s a really poor choice for the concept it maps to in Emacs

<civodul>yeah, i don't get the metaphor

<rekado_>“pull something from the kill ring back into the buffer”…?

<rekado_>“paste” is a bit odd, too, actually

<rekado_>the German “einfuegen” is more direct (translates to “insert” or “inject”)

<zimoun`>Well, VSCode and co. are popular because people do not need to learn how to catch fishes and then prepare them but simply buy the already fried fish. Until there is no fisherman and cooker around and they starve… or pay more to have another dish. Repeat. :-)

<rekado>just shaved off 16 seconds from the startup of the GWL-ized pigx-rnaseq

<rekado>simply by avoiding repeated calls to process->script

<rekado>but it’s not good that process->script takes so long

<rekado>it takes a handful of seconds per script, likely in order to generate the profile manifest. (It’s not building anything at this point.)

<rekado>loading the wisp file is also rather slow (several seconds). I wonder if I can set some Guile compiler options to make it a little faster.

<rekado>yes, parameterizing default-optimization-level to 0 or 1 shaves off another 5 seconds.

<rekado>I’m still puzzled about the origin of /gnu/store/m80ya18bqdk34yj5ribd2kmjhmnirmyv-git-minimal-2.31.1.drv

<rekado>it’s due to grafts, but I find it odd that there is no substitute (the derivation doesn’t exist on ci.guix.gnu.org). So I still suspect I’m doing something wrong somewhere.

<rekado>by far the slowest part of preparing a workflow is to generate the script files in the store — even if nothing has to be generated.

<rekado>this takes about 14 seconds for the 30 scripts of pigx-rnaseq.

<rekado>I wonder if that means that there’s potential for more memoization and caching when using inferiors.

<civodul>uh, that's pretty bad indeed

<civodul>are you sure you're using a single store connection?

<civodul>like, not reopening new connections with open-connection or with-store

<rekado>yeah, pretty sure

<civodul>you can try running with GUIX_PROFILING="object-cache add-data-to-store-cache rpc" to get some stats

<rekado>I have a procedure inferior-store that connects only once and returns the connection when called more than once

<rekado>cool, will try

<rekado> https://elephly.net/paste/1619273156.html

<civodul>nothing looking obviously wrong here

<civodul>would be nice to compare that to just "guix build pigx-rnaseq -d"

<civodul>with --no-grafts i see lower figures

<rekado>it’s rather different because with GWL I’m building one Guix profile per GWL process.

<rekado>the number of packages is also different, because we don’t have snakemake and all its Python inputs, nor do we have pyyaml etc — instead we have guile-dsv and guile-libyaml.

<civodul>ok

<civodul>so you should compare with an equivalent "guix package -m ... -p ..." command

<civodul>to see if the problem is already there in Guix, or if there's something else in GWL

<rekado>here’s an equivalent manifest: https://elephly.net/paste/1619274929.scm.html

<rekado>and here’s the result: https://elephly.net/paste/1619274968.html

<rekado>21 seconds

<rekado>all store locations already exist, so nothing needs to be built.

<rekado>I’ll try again without the inferior.

<rekado>I think I found the source of that git-minimal graft…

<rekado>I ran all of this inside of a Guix environment, which includes the “guix” package that I use as the stable Guix library.

<rekado>since it was added to the PATH, GWL used that old Guix as the inferior

<rekado>no wonder I wasn’t able to reproduce it with my current Guix installation

<rekado>(of course, this has no impact on performance)

<rekado>comparing performance when using an inferior to the current Guix with using the current Guix directly I see a difference of about 10 seconds.

<rekado>here’s the complete test case with results: https://elephly.net/paste/1619276001.html

<rekado>there are many more add-data-to-store calls when inferiors are not used, but there are also way more cache hits; it’s considerably faster.

<rekado>with inferiors there are few cache hits, but much fewer RPCs, and considerably more time spent waiting.

<rekado>I’m not sure I understand what’s going on yet, but it’s good to see that it’s a problem independent of the GWL

<civodul>rekado: so the guix command is already very slow

<civodul>may be like https://issues.guix.gnu.org/41702

<civodul>could you submit the manifest and timings there?

<civodul>that gives a good test case

<civodul>or maybe in a different issue for now

<civodul>and could you submit the inferior test case separately?

<rekado>I submitted a short example manifest that takes 13 seconds (when nothing needs to be built) as bug #48005

<rekado>I’ll prepare an inferior test case in a moment

<civodul>awesome, thanks

<civodul>i probably won't look into it until after the release

<civodul>but having reproducible test cases will be super helpful

<rekado>obviously there’s no rush!

<rekado>I really appreciate you taking the time to comment and think of new things to try

IRC channel logs

2021-04-24.log