IRC channel logs
2025-04-23.log
back to list of logs
<kkremitzki>I'm about to submit my first patch, how necessary is running `guix style`? <kkremitzki>Also, it would be neat if one could do `guix system build --tune` in the same way one can do `guix build --tune` <ArneBab>the_tubular: --tune optimizes the packages for your CPU. If clear linux benchmarks are correct, that can speed up many activities by 10-30%. <the_tubular>But that would make the package non-reproducible right ? <ArneBab>The package itself may be non-reproducible (or rather: may not have a substitute, so you have to build it yourself), but that does not affect the rest of the system, because tune "grafts" its result into the non-tuned package (only linking changes). <the_tubular>Can you explain that part : because tune "grafts" its result into the non-tuned package (only linking changes) <the_tubular>But I remember reading somewhere that Guix's python package was slower than like Debian's one <lfam>It would be great to some help testing and analyzing this problem <ArneBab>the_tubular: if I understand grafting correctly, grafting means that the dependent package changes its LD_LIBRARY_PATH to the new package. <ArneBab>the_tubular: only some packages actually react to the --tune option: they need to be marked as tunable explicitly. And that’s only done for packages where the benefit (faster) more than offsets the cost (additional build, often a local rebuild). <lfam>ArneBab: That's correct in general, but the replacement happens at a lower level than LD_LIBRARY_PATH. Guix actually scans the built packages for strings that look like store items, and rewrites these strings if it finds one that needs to be replaced (i.e. grafted) <lfam>But, it's a similar idea to changing LD_LIBRARY_PATH <the_tubular>Guix actually scans the built packages for strings that like store items, and rewrites these strings if it finds one that needs to be replaced <lfam>It's actually really simple. It basically reads the files byte by byte, looks for strings that start with /gnu/store, checks if the rest of the string matches an item in the Guix database '/var/guix/db', and then checks if the string is registered as requiring a graft <lfam>If a graft is required, a new store item is created with the altered strings <the_tubular>They can be encoded in the binary ? Or are they always in plain text ? <lfam>It depends on how the compiler is used. We take care to use the compiler such that these references are not compressed in the binaries <lfam>Many binary formats contain strings. The Unix `strings` command is a simple way to poke around <lfam>That particular aspect is beyond my knowledge <the_tubular>And let's say there's a name change for some reason, could that still work ? Could you replace a string with a bigger/smaller string with no impact ? <lfam>But, pick a program that you've installed with Guix, and run `strings $(which $yourprogram)`, and you'll see the references to other packages <lfam>No, the length of the string has to remain the same. <lfam>For this reason, sometimes we fudge the version number when grafting a new version of a package that has a different length of the version number <lfam>For example, if foo-1.2 needed to be replaced via grafting with foo-1.2.1, we might make the new package have the version "1.A" <lfam>It means "do something hacky" <lfam>The implementation is optimized to go faster than the method I described, but the principle is the same <the_tubular>And replacing strings that references specific packages is the only thing you have to do ? <lfam>So, it can break if somehow those references are obscured, like if they get compressed or broken up into chunks. It's happened before <the_tubular>Is there a specific language that is more prone to this ? <lfam>I don't have a good sense of that <lfam>We had to adjust how we used GCC <lfam>I like to think we've ironed it out at this point <lfam>You can see early in the thread, Mark expresses one of the core design assumptions of Nix and Guix: "[...] our simplistic assumption that references to the store in compiled code will be easy to find and replace." <lfam>If you pay attention to the timestamps on the emails, you can see how tenaciously Mark debugged the problem. I remember following along and it was kinda thrilling <lfam>These early efforts built a very solid foundation for the project. <lfam>Grafting is not a perfect solution. They have some negative points that can be optimized away, but others might not be able to be optimized away. <lfam>For example, if a bug fix is necessary in a core library, and we can't rebuild the world quickly, we will use a graft to deploy the update soon. However, applications that use the library may have compiled themselves differently based on the new library. There could be subtle incompatibilities or inefficencies that won't manifest if we actually compiled the application using the new library, rather than merely rewriting the store reference. <lfam>For this reason, we only use grafts to deploy changes that are binary-compatible with the old un-fixed library. <lfam>But it's still kinda hacky and we never know if some subtle problem is hidden <the_tubular>So by grafting, in some cases, you can't get a reproducible package ? <lfam>It depends what you mean by reproducible. For Guix, we talk about building packages like mathematical functions. If you input A and B into the function, you get X. If, with a graft, you input A and B`, you could get X`. <lfam>That's deterministic and reproducible, 100% <lfam>Grafting is actually more deterministic than almost any other "build" in Guix <lfam>Or, to be more precise, I could say that it's more guaranateed to be deterministic <lfam>Because we are just doing "Find and replace" with some strings. How could it not be deterministic? :) <lfam>Compared to optimizing compilers like GCC, which have taken years of effort to usually be reproducible <the_tubular>To do this, you still need to be able to get the old version, which just doubles the problem no ? <lfam>Which problem is doubled? <lfam>The old version can be substituted, but grafting always takes place on your computer <lfam>But yes, it takes extra space and time <lfam>A good reason to eventually "ungraft" and rebuild the world <the_tubular>ungrafting is just the same process in reverse I guess ? <lfam>Do you know that Guix substitutes built packages for you? Aka downloads them? <the_tubular>I know it can, but I can't never tell if it is actually doing it <lfam>It doesn't really matter. Guix is a build from source distro, like Gentoo, but we are able to download things instead if they are available. The best of both world <the_tubular>Yeah, I've been following guix for a while, but my questions are still basics lol <lfam>Ungrafting is not the reverse of grafting. If we deliver a bug fix with a graft, we will eventually do some "ungrafting", which means we deliver the bug fix without a graft, but just integrating it fully into the affected package <the_tubular>What is "integrating it fully into the affected package" <lfam>Yeah, like take the bug fix patch and put it directly into the package <lfam>Rather than a replacement package for a graft <the_tubular>You just build it with the targeted's version's commit right ? <lfam>I don't understand the question the_tubular <the_tubular>I mean, the opposite of having a grafted package is building the package from source with the targeted commit ? <lfam>Sometimes we use grafts to deliver a patch file, sometimes a new Git commit, sometimes a new version <apteryx>ACTION wonders if we could fix substitute* to be less surprising w.r.t. its treatment of $ <apteryx>I'd expect e.g. "^ some-word$" to match, but instead I had to use "^ some-word.*$", although the only thing after 'd' is a newline. <apteryx>is something new in shepherd causing my services CWD to be poining to the store location where their executable lives instead of in my HOME (I think this used to be the case). <apteryx>For example, I have an emacs user service, and the CWD is set to /gnu/store/yhcns4bjmgvfd79ad9s9hx6shpn54q77-profile/bin/, which is annoying. <apteryx>hm, './pre-inst-env guix refresh -u webkitgtk' appears to hang; anyone else? <apteryx>it just finished, it took a suprisingly long time <weary-traveler>Gooberpatrol66: it's been off and on like that. Noticed something similar yesterday, but not earlier today <ieure>Gooberpatrol66, Yes, Savannah is really struggling, it's been awful lately and today is some of the worst I've seen. <podiki>apteryx: i've noticed guix refresh being very slow randomly these days <apteryx>podiki: I'm guessing one of the urls we poke at is slowing everything down, but we'd have to trace it <podiki>evaluation 2054781 of master on berlin failed (not sure why based on the short log), i restarted it <podiki>and rebased mesa-updates on latest commit as well <podiki>godot has failed after python merge, no idea why <cobra_>trying to boot latest i686 installation image on a pentium iii and it instantly kernel panics <cobra_>its too large for me to be able to see the shepherd error <apteryx>is there a relationship between postmarketos and gnome? <apteryx>probably our postmarketos applications should go to their own module and not gnome-xzy, no? <apteryx>e.g. (gnu packages postmarketos), maybe <ArneBab>ieure: do we have a fallback for savannah? <sham1>I like guix-mirror. Makes it very easy to search for things when I don't have the guix sources handy <tae>What are the prepended hashes in gnu/store? I tried to figure this out from docs and hashing the files, but I can't see what they are. <tae>It's not nix-base32 either. <tae>So if I do guix -hash <some_option> /gnu/store/xyz-name, how can I get xyz? <tae>In the files I do see sha256 hashes, but it does not explain the prefixes. <decfed>guix hash is to hash the inputs and ensure they are unchanged. The store hashes are a result of the build derivation I believe but could not tell you more <decfed>I really wish we had more of the dissecting guix posts. These really helped me grasp key concepts. Hope unbalanced parens is doing well. <ArneBab>csantosb: can that be configured as fallback-repo for guix pull? ⇒ (and=> (or (pull-from-savannah) (pull-from-codeberg)) report-changes) <sham1>You could always have an alternative channels file which points to the codeberg repo and then time-machine that when needed <sham1>One should just be careful to not have both fail at the same time <apteryx>lilyp: I just found an embarrassing thing with the GNOME calendar; adding reminders to events and saving does not actually save the reminders. Very easy to overlook things then. <Kabouik>What package should I add to a guix shell if I'm facing a "GLIBCXX_3.4.32 not found" when compiling? <apteryx>there was an error shown, caused by some exception <apteryx>I forgot what it was about already, sorry. Maybe it'll be visible in the logs? <cbaines>no worries, it seems to be working now at least <cbaines>unfortunately even I find the logs hard to read, I think QA could do with logging to multiple files <cbaines>anyway, at this point QA is waiting for substitute availability to recover after the python-team merge <ieure>I'm working on a patch to add an autofs service, but I have a very strange build failure. I think this might be a Guile bug, I filed #78004 about it. But maybe someone here can explain why changes in gnu/services/nfs.scm are causing gnu/build/linux-initrd.scm to fail to load with an error that looks entirely spurious to me? <ieure>If I `make clean && make' on my branch, the build fails. If I `git checkout master && make clean && make && git checkout autofs-service-type && make', the build succeeds. <ieure>The file the build fails to load is untouched by my changes. <ieure>I'm at a loss to explain why my changes break the build. <ieure>Does that work if you have multiple channels? <yelninei>is it possible that the configure-flags for libssh are wrong? I somehow get the literal list procedure when cross compiling as a configure-flag <ieure>civodul, How does it know which channel to replace the URL for? Assumes Guix is the first one and uses that? <civodul>--url applies to the ‘guix’ channel, always <podiki>“Repeated allocation of very large block” during ‘guix pull’ issue <civodul>someone should attach gdb to it when that hapens <podiki>first i've seen it since i last commented on that issue (i'll add this to the issue thread today) <ieure>I've seen this intermittently, but not much lately. <podiki>but i have no idea why or what is possibly the cause, some indirect dependency update? <podiki>(no direct python packages as input) <ruther>I think this should work for choosing the codeberg mirror https://paste.debian.net/1371125/ in case Savannah is down. Including multiple channels. Haven't actually tested this for scenario when savannah is not running though. <podiki>i guess best if we have a "mirrors.guix.gnu.org" or something that does that (and not hosted on same machines as savannah/codeberg/etc.) <ieure>civodul, I was reading over an old patch of yours to get substitutes from IPFS, have you thought about picking that up again? I think making Guix less reliant on SPOF infrastructure would be extremely valuable. <civodul>ieure: somebody posted patches on this topic recently <civodul>i haven’t taken the time to look into it though <ruther>Could someone remind me what the page on (I think) QA is that shows numbers of jobs running / failing / scheduled for individual commits? I know futurile was showing it in their beginners video to choose a commit that has substitutes <ruther>So I am thinking about making a utility that would choose this automatically <ruther>Recently I got to know find-latest-commit-with-substitutes, but that one queries just the guix evaluation, and I am thinking of alternative possibilities of implementation, imo reading html page with those numbers is going to be easier than going over all the jobs through the cuirass api (if that even is possible) <podiki>i'm stumped by this godot build failure, the symbols that are undefined come from bundled third party files, so i don't see what has changed on guix <podiki>eg undefined reference to buminiz::mz_uncompress <podiki>which should be from the included minizip library. what has changed with python-team merge to affect such a thing? <Kabouik>Is it possible to install a package with package transformation in a guix shell, e.g., to install an older commit temporarily? <podiki>yes, just pass the transformation options <podiki>(transformations work for guix shell) <Kabouik>Something like `guix shell remind --with-commit=xxxx? <podiki>possibly with --with-git-url=remind=<url> if needed too <podiki>afterall, guix shell is making a profile with the packages, so it is just like guix build/install <Kabouik>Yup I was checking it but didn't expect it would work with guix shell so easily, nice. I was thinking I may need to build in the shell. <Kabouik>Of course it's actually not as simple as it looks if the package definition has changed since the commit I'm aiming at <podiki>you can use time-machine too, if you need to travel back in the guix commits too <podiki>godot: all the undefined references are from and to included thirdparty libraries, i don't understand what guix change would call this to fail now <Kabouik>Yup time-machine will probably be easier in my case, package transformation may be more useful when trying to find a faulty package though <cbaines>although data.guix.gnu.org is really struggling to answer requests in a timely manor at the moment <cbaines>ideally this kind of tool to check for substitutes wouldn't be necessary though as we would always have them <ruther>cbaines: Yeah, but the way Guix works this is not possible at the moment, and maybe it's for the better, as security fixes can be delivered in a timely matter - not something Nixpkgs can do well. <cbaines>it should be, with QA we can build patches and branches prior to merging <cbaines>(and grafts enables not affecting massive numbers of packages to apply security updates) <ruther>I know, I was talking about what nixpkgs does that they first put everything to master and only then when CI builds it, they update nixpkgs-unstable to master's commit that has been built fully <ruther>Which I think is the only way to ensure something like that happens. If you do it manually, I don't think you can ensure it as there is multiple people involved. Ie. someone pushes changes A, someone else changes B, and A makes B changes rebuild. <cbaines>that situation is indeed possible, however currently some things are still being pushed to master before QA has had a chance to build them <ruther>cbaines: yeah, it's a bit unfortunate. <ruther>cbaines: though even if that wasn't the case and if what I outlined is not so common, we still have Guix itself to be built <ruther>cbaines: or could that be worked around as well? I think it would be very inconvenient in the only way I can imagine <cbaines>yeah, the question of Guix itself is even more complicated. I still want a solution in place for #74942 before I can even start to think about what's actually going on <ruther>cbaines: do you happen to know... if the substitutes stored grafts, would it be possible to actually use them - are the new hashes in paths known before the derivation is built? <ruther>cbaines: because as far as I can tell, the answer is probably not, or at least that it's not utilized? Because if it was, why do we need to get all the non-grafted variants first, even if the grafted ones are already in the store? <ayys>I made my first contribution to guix last week by packaging a guile library I wrote (although its not merged yet). I want to continue contributing, is there a list of beginner friendly issues I can start hacking on? <ayys>Maybe I can go through the guix-devel mailing list? Is that a good idea? <civodul>ayys: hi! often a good start is by using it yourself and finding bugs and missing features or packages that you’d like to address <ieure>ayys, If you want to look at open patches and review them, that's very helpful! <ieure>You don't have to be a committer to review, but it helps committers to see reviews.. <identity>ayys: apart from what was already said, you could look at packages that are outdated <ayys>from where I am standing, I can see that guix is short on reviewers, so I'd love to lend a hand where I can. <ieure>ayys, Yes, that's one way. If you're an Emacs user, you can also set up debbugs and email reviews that way. <ayys>oh I am an emacs user. How do I setup debbugs in emacs? are there any docs for that? <identity>ayys: you want the emacs-debbugs package, i think it's mentioned somewhere in the guix manual <cobra_>how do i find out what the shepherd error is after the kernel panic since the trace fills up the screen <cobra_>what do i add to the kernel arguments to get serial output again <ruther>cobra_: something like serial=ttyS0, the ttyXX depends on what type it is exactly <futurile>Hmm my day went: <goes to send email to mailing list> -> <realises MUA is messy> -> <starts updating packge in Guix!> -> <now playing with config> <futurile>still haven't sent the email I intended to! <isf>This usually indicates a bug in one of the channels you are pulling from, or some incompatibility among them. You can check the build log and report the issue to the channel developers. The channels you are pulling from are: <isf>but what mean pulling? What do you want to say with that concept of "pulling"? <lfam>It's talking about `guix pull`, isf. Do you know that command? <isf>so that is just other word to download? <isf>"guix pull: the command downloads the latest Guix source code and package descriptions, and deploys it" <isf>so would be similar speaking about download right? lfam <isf>Can you explain it with other words what pull mean? <lfam>It's about deciding which version of Guix you will use