IRC channel logs


back to list of logs

<rekado>looks like I’ll have to figure out a way to speed up ‘guix upgrade’ when users have accumulated changes to their profile.
<rekado>I have another user here who kept doing ‘guix install this; guix pull; guix install that’, and now ‘guix upgrade’ takes >5 mins to do nothing.
<rekado>‘guix upgrade’ also doesn’t seem to upgrade all packages, so the user cannot run ‘guix install’ due to conflicts.
<rekado>~/.guix-profile/manifest weighs in at 24M
<rekado>I’ll use --export-manifest and then install packages from the generated manifest.
<rekado>the new manifest file after the upgrade is also 24M big
<rekado>with R packages there’s a *lot* of duplication in that file
<rekado>perhaps it would be good to support references
<rekado>or maybe it would be enough to cache more
<rekado>read-manifest needs about 27 seconds on that 24M file
<civodul>rekado: could you share one of these manifests so we can profile things?
<civodul>or we can have a manifest that just picks say 50 r-* packages to reproduce that?
<civodul>it's important to have that fixed
<rekado>I see that READ alone takes a little over 5 seconds. On this NFS thing it’s 27s.
<rekado>so there isn’t much we can do on the Guix side to improve it, other than to generate smaller manifest files.
<civodul>the Scheme 'read' or the syscall?
<rekado>(on my laptop with SSD a little caching by store item name reduced the time to process the manifest file from ~8 secs to ~4.5 secs)
<rekado>Scheme READ
<rekado>scheme@(guix-user)> ,time (define m (call-with-input-file "/tmp/bad-manifest" read))
<rekado>;; 27.116401s real time, 5.478444s run time. 2.620153s spent in GC.
<civodul>can you try with (read-disable 'positions)?
<rekado>;; 18.041167s real time, 3.645781s run time. 1.860371s spent in GC.
<civodul>less bad
<civodul>we could also compress things by supporting references to shared substructures
<rekado>right, that’s what I meant above.
<rekado>we could use the store name as a unique key
<civodul>i was thinking of SRFI-38
<civodul>if that works, it comes for free
<civodul>problem is that using write-with-shared-structure would introduce a syntactic incompatibility
<civodul>so instead we'd probably need to roll our own
<civodul>but first it'd be good to prototype with SRFI-38 to see if it helps at all
<rekado>is compatibility a concern at all? I thought we could just bump the manifest format version number and do whatever.
*rekado didn’t know about srfi-38
<civodul>rekado: bumping is fine (mostly), but here would fail to parse #1#=xyz and such
<civodul>so we'd get an ugly crash rather than a "format not supported" error
<civodul>s/here would fail to parse/'read' would fail to parse/
<civodul>rekado: the manifest at gives me a 632K manifest file
<civodul>i'd really like to know how to get a 24M file :-)
<civodul>i mean that's two orders of magnitude higher
<rekado>I opened a bug report with four packages that get us to 7.1M
<rekado>(I Cc’d you)
<rekado>I can prepare a manifest with some trial and error that pushes the file to 24M
<civodul>thanks rekado
<civodul>7.1M is already a good start :-)