IRC channel logs

2022-04-19.log

back to list of logs

<arjan>looks like it is getting flooded by the musb-hdrc module
***rekado_ is now known as rekado
<Ox151>hello, i am trying to break up my config.scm files into seperate files for different systems. i have a base-system.scm and then a config.scm for each system. if i have a services in the base-system.scm but want to add more services for a specific system can they be appended to the base-system.scm service list or does it overwrite them?
<apteryx>you'll have to extend them explicitly
<apteryx>e.g. (services (cons* some-service another-service ... (operating-system-user-services %base-os)))
<Ox151>apteryx: thank you, if you define say packages in one config and then define packages in another does it overwrite the one that was called before?
<Ox151>and is that the same for services?
<Ox151>w.e one is called most recent is used. even though you use append, it doesnt really append it to the 'method' defined previously ie packages/services
<apteryx>you can think of it as functional; there's no mutation involve to setting a field. The value specified is the value taken. If you want to extend the previous value, you need to explicitly define it that way like I did earlier
<apteryx>I should have written "there's no magic involved when setting a field"; the value of the field (e.g. 'services') of the record object (e.g. 'operating-system'); it's the same for any field
<apteryx>oof. too late to write :-) I'll get some zzz. Good luck!
<philip>It looks like `maybe-inputs` from `(guix import utils)` produces "old style" inputs. Is it still the preferred function to use?
<apteryx>it should be updated
<apteryx>(to produce new style inputs)
<ap5>I am pretty new to guix, but not at emacs and lisp. Is there any configs around that can be read to better get a grasp on a "godd" config?
<kitty1>Hey, I'm kinda struggling to understand guix home a bit; anyone know if there is any good home configurations floating around I could dissect?
<MysteriousSilver>there are some listed at https://git.sr.ht/~abcdw/rde#1.3.1.
<abrenon>hello guix
<kitty1>MysteriousSilver: thanks so much ; I feel like that needs to be somewhere easier to find :P ,, not sure if I have found the specific thing I'm looking for yet but this will be a huge help for when my brain refuses to understand documentation lmao
<kitty1>think I might be able to figure out the question that come to think of it I never directly asked from here lmao
<kitty1>woah yoctocell hahs a very pretty config, I, should really make mine like that lmao
<civodul>Hello Guix!
<civodul> https://lwn.net/ :-)
<abrenon>hi civodul !
<abrenon>can't believe I missed the exact day of the 10 years anniversary : /
<civodul>hey abrenon!
<civodul>many of us were on a long week-end yesterday :-)
<civodul>i enjoyed your contribution to that blog post, BTW, with its "intro-clusion" :-)
<abrenon>: )
<abrenon>thanks
<abrenon>I love reading the other posts too, it's great to discover a bit of all the personal stories behind guix
<civodul>yes, it's interesting to learn how people came to the project
<jpoiret>wow, there's quite a lot of posts :)
<jpoiret>that and it being cross-posted to lwn should hopefully bring in some more enthusiastic people
<civodul>yup i hope so!
<civodul>it also tells something about how free software works that not everybody realizes
*jonsger likes that 10 years GIF :)
<abrenon>that it's just made by the people who want to make it ? that all contributions matter ?
<civodul>yes; that it's just people that come together, who share values but also have different preferences and motivations
<civodul>you know, people tend to view software, even free software, as "products"
<civodul>it's an opportunity to tell a different story
<abrenon>hmmm that's so true, it's hard to start a conversation with someone without having them wondering what you are trying to sell
<civodul>yeah
*jonsger tries to figure out when I started to use Guix :)
<abrenon>and on the other people want no involvement, they want the ability to demand that things work
<civodul>right
<civodul>well not willing to be involved in every software project one uses is legitimate :-)
<civodul>jonsger: that was a long time ago, no? :-)
<abrenon>I think it depends on the level of involvement
<abrenon>not having to contribute packages or fix code is one thing
<abrenon>but using the complex tool that computers are demands at least some level of understanding in the end
<jonsger>summer 2017 was my first commit :)
<abrenon>jonsger: wow that was ages ago !
<civodul>:-)
<abrenon>you're almost (* 0.5 (project-age)) old !
<jonsger>but I really wonder how I first heard of Guix back then. And what motivated my to try it out...
<civodul>heheh, a grownup :-)
<abrenon>I think the continuing success of commercial solutions despite their many privacy issues is not a matter of personal taste and the "market" rewarding what's best so much as an "offering"
<abrenon>an offering to the Computer gods in hope that what they need will continue working, because they paid for it and have the right to expect it to work
<jonsger>running Guix system on 3 of my 5 private "computers" is nice :)
<jonsger>sneek: mothacehe
<civodul>sneek: seen mothacehe
<sneek>mothacehe was last seen in #guix 27 days ago, saying: hey civodul!.
<civodul>jonsger: ↑
***dgcampea-2 is now known as dgcampea
<zamfofex>Hello, Guix! I noticed the replies I sent to Maxime didn’t show up in <https://issues.guix.gnu.org/54832> Now, I think I figured out that I should have replied to <54832@debbugs.gnu.org> in addition to Maxime’s email address. Is there any way I can rectify that? Is the appropriate solution to send another set of emails? Also: I really wish someone could investigate if the patch is appropriate to be accepted. (cc civodul)
<paladhammika>hello guix. how would i go about changing the defaut gid of the "users" group?
<civodul>zamfofex: hi! i think you can resend your message, Cc'ing debbugs this time
<civodul>paladhammika: you can use 'map' to iterate over %base-groups and add a 'gid' field for that one group
<civodul>does that make sense?
<zamfofex>civodul: Thank you, will do! I wanted to say that I didn’t want to sound obnoxious or picky when I asked for someone to look into the patch again, but I’m just a bit afraid it might end up being forgotten, is all, I think.
<zamfofex>Also: Is it appropriate to send one email as a reply to each message, or should I send only one replying to all of them?
<paladhammika>civodul: somewhat, I see an example that uses map in the manual "(packages (append (map..."
<zamfofex>paladhammika: You should be able to do a similar thing for groups in the system configuration.
<civodul>zamfofex: no problem, pinging is a good idea; i hope more people can come and review and apply patches though, we're a little low on humanpower these days!
<paladhammika>so perhaps something like "(groups (append (map (user-group (name "users") (id 100)) (%base-groups))
<zamfofex>civodul: I understand. I wish I could help in some way, though I think my issue is that I don’t have enough understanding to be able to review patches.
<paladhammika>civodul: thats a real shame. i had to set up arch linux for my organization and it was really a pain. guix is truly superior.
<zamfofex>paladhammika: I don’t think you need ‘append’ in this case.
<zamfofex>‘append’ is used to concatenate lists. Whereas ‘map’ will create a new list by calling a function on each entry of an existing list.
<zamfofex>The idea is that you should try to create a new list with the group entry you want modified to something else, and leave the others intact. The modification you want to perform is adding the ‘id’ field.
<civodul>paladhammika: something along these lines: https://paste.debian.net/1238364/
<civodul>yeah like zamfofex explains
<reza[m]>are there any VPS hosters that support guix?
<zamfofex>reza[m]: I think most VPS hosts support uploading an ISO image. I was able to run Guix on a Vultr VPS once.
<jayspeer>hi #guix, does anyone have problem with "git submodule"? Both debian's and guix's git give me this - git: 'submodule' is not a git command. See 'git --help'.
<abrenon>`git submodule --help` works here
<WesterWest[m]>are packages from melpa allowed to come into guix repo?
<jayspeer>abrenon: which version of git do you have?
<abrenon>git version 2.34.0
<abrenon>guix@f1d18adbed39a3bacae93be29346fd4c86b480ef
<apteryx>jayspeer: even from a pure environment? e.g. 'guix shell --pure git'
<apteryx>if it's present (and broken) on your host, it may interfere
<arjan>WesterWest[m]: yes, because melpa requires GPL compatible licenses, all packages should be allowed, there is even an importer to make it easy: https://guix.gnu.org/manual/en/html_node/Invoking-guix-import.html
<jayspeer>apteryx: I'm getting weird errors testing it from 'guix shell' (never used it much before), like "basename: command not found" & "sed: command not found"
<sneek>Welcome back yewscion :D
<apteryx> ah, it seems to require coreutils impurely
<yewscion>sneek: botsnack.
<sneek>:)
<apteryx>and perhaps more; so 'guix shell --pure git coreutils'
<WesterWest[m]>arjan: oh! I didn’t realize that it was in the manual! great
<abrenon>jayspeer: isn't your PS1 using complex bash commands calling basename or sed at some point for instance ?
<jayspeer>'guix shell --pure git coreutils sed' did the trick, however it complains that 'man' is missing (which I guess makes sense for a pure env)
<jayspeer>abrenon: I've never customised my PS1
<abrenon>ok, sorry, that was a long shot but it happens to me on some hosts, so I thought that could be the cause of these basename and sed errors
<jayspeer>my '$GIT_EXEC_PATH' is set to '/home/jayspeer/.config/guix/current/libexec/git-core' for some reason (along other guix env vars) instead of pointing to '$HOME/.guix-profile/lib-exec/git-core'
<jayspeer>Any ideas how to clean them all up? "Them" includes vars like "SSL_CERT_DIR" and such, which for some reason all point to the `.config/guix/current/`
<attila_lendvai>interesting discussion/issue for experts in character encoding and bootstrapping: https://issues.guix.gnu.org/54893
<sneek>Welcome back attila_lendvai, you have 1 message!
<sneek>attila_lendvai, lilyp says: sorry, you're right, I forgot to read your paste
<attila_lendvai>lilyp, np, and thanks for the message! (i meant to read the logs afterwards, but i forgot)
<paladhammika>civodul, zamfofex: got it working. thanks for the feedback.
<jonsger[m]>reza: I run Guix system on a Hetzner VPS. You need to ask them to upload a Guix ISO and back then I had to install traditional BIOS, so no EFI. That has maybe changed in the mean time...
<apteryx>jackhill: you could add man-db to get the manpages in the pure env too
<apteryx>jackhill: if you install git in your user profile (guix install git) and refresh the env (or source its ~/.guix-profile/etc/profile file), it should work
<apteryx>err, this was meant for jayspeer^
<apteryx>rekado: it's not built on the CI yet, but if you are interested in trying a fixed texlive-polyglossia, it's on a wip-ipython+polyglossia branch
<tschilptschilp23>Hi guix!
<apteryx>hello!
<apteryx>civodul: hi! it looks like some texlive change caused the evaluation to fail on core-updates: https://ci.guix.gnu.org/jobset/core-updates
<apteryx>ttps://ci.guix.gnu.org/eval/188332
<apteryx>sneek: perhaps of interest; a branch name with '+' seems to cause issues for cuirass: https://ci.guix.gnu.org/jobset/wip-ipython%2Bpolyglossia (try the edit button)
<apteryx>sneek: later tell mothacehe perhaps of interest; a branch name with '+' seems to cause issues for cuirass: https://ci.guix.gnu.org/jobset/wip-ipython%2Bpolyglossia (try the edit button)
<sneek>Will do.
<tschilptschilp23>Is anyone with experience in go-packaging here? I'm trying to package the new docker-compose on go with this definition: https://paste.debian.net/1238375. I totally get, that this is far from being ready as it seems to need quite some packages not yet in guix. However, I think I'm doing something wrong fundamentally, as the (failing) build-phase seems to recognize the input ~go-github-com-containerd-console~ but not
<tschilptschilp23>~go-github-com-alecaivazis-survey~ which I 'provide' and actually builds successfully. I would be fine packaging some more of the obviously missing packages, but not with the preassumption, that they'd also not be recognized by the target... Any hints are welcome! This is the backtrace of the fail: https://paste.debian.net/1238376
<attila_lendvai>tschilptschilp23, FYI, i have some extensive work in the queue for merging that makes go importing much more resilient: https://github.com/attila-lendvai-patches/guix/tree/import
<attila_lendvai>it depends on this, though: https://issues.guix.gnu.org/54906
<attila_lendvai>well, doesn't really depend, but it'll generate packages with #:skip-build? set to #t.
<apteryx>for what it's worth, from what I recall, our most pressing need for go is to support go submodules, since many things are moving to this (ad GOPATH stops working in that scenario). To implement it we'd need to be able to expose our own packages via GOPROXY, which the go machinery could consume
<apteryx>an update to docker is blocked from (the lack of) go submodules for example
<attila_lendvai>apteryx, did i get you right? we need to run a http server to satisfy golang now?!
<apteryx>it understands file URIs, so we could do without the HTTP server, IIRC
<apteryx>we "just" need to expose the right file system hierarchy (and the archive format/metadata it expects)
<attila_lendvai>apteryx, FYI, my fixes to the go importer properly handles submodules. well, i think. it can import a transitive closure of hundreds of dependencies, but i haven't successfully built anything yet.
<apteryx>does it use GOPATH?
<apteryx>It's scheduled for deprecation, if it hasn't been already (in future Go releases)
<tschilptschilp23>thanks for the info! Not sure if this is worth adding, but I've tried with the .tar.gz archive from github as well, it then seems to 'depend' on less packages, but still tries to get a bunch from the path ~github.com/[...]~
<attila_lendvai>apteryx, it's only the importer. do you mean #:import-path and #:unpack-path? it generates packages with those when needed.
<apteryx>OK; so it uses GOPATH. I guess it's good for our current needs, but it won't help when/if Go pulls the plug on GOPATH
<attila_lendvai>apteryx, and even that doesn't work, i think. when there are two packages with the same #:unpack-path then the second one fails with a read-only directory error (or something along these lines)
*attila_lendvai is nowhere near a go expert
*attila_lendvai has force-pushed that branch
<char[m]>I found I should be able to use etc-service type to customize/etc/bashrc, but I get "duplicate bashrc entry for etc". I can't find the existing etc-service-type or how to modify it to replace the default bashrc with mine.
<luchadoritos>Hello Guix! How may I compile a C file using GCC if it includes "#include <linux/limits.h>"? I've tried using the gcc and gcc-toolchain packages alongside installing musl, glibc, and linux-libre-headers.
<raingloom>luchadoritos: don't install those, use a temporary development environment
<luchadoritos>raingloom: Ah, will do. This means I will use guix shell -D ${packages-to-debug}?
<raingloom>luchadoritos: in general, don't install dependencies of a project into a "global" environment, like your user profile. although if you meant you installed them into a separate dev profile, then it's fine.
<raingloom>luchadoritos: i think you might have to pass -D before each package, but yes
<raingloom>btw, this: guix shell gcc-toolchain linux-libre-headers -- gcc foo.c
<raingloom>worked for a simple C file that just inludes linux/limits.h, so the problem is definitely not which packges you installed
<luchadoritos>raingloom: Makes sense! Thank you!
<raingloom>no prob. good luck! uwu
<char[m]>luchadoritos: if I remember correctly, header files with linux/ prefix have to be compiled as kernel modules.
<retropikzel>If I take mingw-w64-x86_64 package into guix shell with me I dont see the compiler command, if I remember right its something like x86_64-w64... should it be there?
<jackhill>attila_lendvai: I keep re-reading the go modules documentation, but haven't quite wrapped my head on how we'd implement that in Guix. I'm also far from being anything but a beginner gopher, but it seems like a first step might be modifying the way we create the source-code union to match the new go modules structure. I haven't thought about the version requirements yet. It would, of course, be nice if we
<jackhill>didn't have to create a union, but could use a real path. Not sure if go supports that yet.
<sneek>Welcome back yewscion!!
<yewscion>sneek: botsnack.
<sneek>:)
<attila_lendvai>jackhill, then you're already ahead of me. i only read bits and pieces to understand enough to fix the importer on my usecases
<jackhill>attila_lendvai: it's been a slow process of comming back to it periodically over a few years. I'm happy to help brainstorm, compare knowledge, etc. I don't see me actaully producing patches for our go-build-system in the near future though (but one never knows!)
<attila_lendvai>jackhill, thanks, noted! i'll ping you when i need some help.
<jackhill>I wonder if go's -buildmode=shared adn -linkshared would ever be useful for us
<EMax`0Mancer[m]>what's the best way of handling PATHs in Guix? (Specifically here actually adding the ~/.nix-profile/bin to the PATH in a systematic way.)
***yewscion10 is now known as yewscion
<maximed>EMax`OMancer[m]: Guix sets PATH automatically, at least if ~/.bash_profile / ~/.profile are set up
<maximed>Guix does not use ~/.nix-profile
<maximed>instead, it has a ~/.guix-profile
<civodul>see also https://guix.gnu.org/manual/devel/en/html_node/Search-Paths.html :-)
<civodul>howdy maximed!
<maximed>civodul: hi
<maximed>maximed: If you want to use both nix and Guix, probably nix has something similar to ~/.guix-profile/etc/profile (maybe ~/.nix-profile/etc/profile? not sure))
<maximed>so you could copy the ~/.guix-profile/etc/profile in ~/.bash_profile / ~/.profile but with .guix-profile replaced with .nix-profile
<maximed>This assumes no Guix Home or Nix Home which I am not familiar with.
<EMax`0Mancer[m]>maximed: I am trying to use Guix Home (unexpertly).
<EMax`0Mancer[m]>Nix on Guix seems to have some setup via `$ source /run/current-system/profile/etc/profile.d/nix.sh` [cf. https://guix.gnu.org/manual/en/html_node/Miscellaneous-Services.html ], which works if I run it in a shell, but doesn't seem to get inherited by the larger environment.
<EMax`0Mancer[m]> Maybe some way of running `source /run/current-system/profile/etc/profile.d/nix.sh` on system start?
<maximed>antioxidate-build-system update from some time ago: it now separates dev-dependencies / build-dependencies / dependencies more (in the invocation of 'rustc' -- native-inputs/inputs is still used), avoiding some build failures caused by ambigious imports.
<maximed>Next step: support build targets in non-default locations (basically read [[bin]] sections from the TOML), to allow cbindgen to build
<maximed>EMax`OMancer[m]: I guess that just adding the 'source ...' to ~/.bash_profile would suffice, like Guix does for Guix things?
<EMax`0Mancer[m]>maximed: editing `~/.bash_profile` by hand, or adding some Guix-specific configuration to manipulate `~/.bash_profile` ?
<maximed>Idea for utilising reproducible builds: a multi-sig system for substitutes! Currently, the 'authorized-keys' list makes an ‘union’ of allowed signatures, WDYT of also allowing an ‘intersection’ of signatures that all are required?
<maximed>No changes required for "guix publish", only for "guix substitute"!
<maximed>EMax`OMancer[m]: Except for the initial installation and when adding new users, there exists no ‘tooling’ for automatically manipulating ~/.bash-profile.
<maximed>It's just ‘copy this template to there’
<maximed>Summarised: manual.
<maximed>civodul: The (guix tests http) patches will be updated ‘soonish’, I found some simplifications
<EMax`0Mancer[m]>maximed: ty!
<char[m]>mbakke: did you ever figure out how to override the default /etc/bashrc?
<char[m]>Or Gooberpatrol66??
<Gooberpatrol66>I just edited ~/.bashrc
<char[m]>Gooberpatrol66: I see, I'm want the change apply to all users.
<Gooberpatrol66>I symlinked the ~/.bashrc's to one file
<char[m]>But then individual users cannot make changes to their own .bashrc
<Gooberpatrol66>i guess you could symlink ~/.main_bashrc and then source ~/.main_bashrc inside each ~/.bashrc
<char[m]>That could work, but it is more of a workaround than a solution. I don't see why I shouldn't be able to have a custom /etc/bashrc.
<zamfofex>Hello once again, Guix! Can anyone who might be familiar with it explain to me how the Mono package in Guix doesn’t make use of the prebuilt binaries that come in the tarball? Is there a mistake? The package definition doesn’t seem to address it in any way.
<apteryx>zamfofex: hello! Perhaps it simply has been overlooked
<apteryx>the release tarball indeed seems to contain loads of prebuilt .dll
<apteryx>janneke: perhaps you remember if there was some discussion surrounding mono and the prebuilt dll the source appears to come with?
<zamfofex>apteryx: I found this: <https://github.com/mono/mono/issues/7445> It seems the issue was introduced in a later version of Mono for Debian, so maybe they didn’t require those in the version in Guix. The solution they took in Debian seems to have been to simply consider some of those DLLs as bootstrapping binaries and to rebuild the rest.
<zamfofex>I don’t fully understand all the details, though, so I might be wrong.
<apteryx>it was originally contributed here: https://lists.gnu.org/archive/html/guix-devel/2016-07/msg00562.html, and that particular issue didn't seem to exist or was overlooked
<apteryx>zamfofex: I see!
<apteryx>we should see what others are doing, but if debian does this, it's probably the same or worst
<zamfofex>I was investigating updating Mono so that I could try to use it to bootstrap an earlier version of .NET (and eventually the current versions). I suppose maybe I should try to either build Mono from source myself for the time being (without avoiding the binaries) or to use the current version in Guix and see how well it works.
<apteryx>please report the issue to bug-guix@gnu.org so we don't forget about it
<old>I have a cuirass daemon that build a project of mine when new commits are added to a specific branch.  Since I want to build the package from local source, I use `(source (local-file ".." "libfoo" #:recursive? #t))`  However, this also copies some files that are not staged by git .e.g local configuration.
<old>Is there a better way of doing that?
<lilyp>you can use git-predicate to filter based on files, but it will still copy your local changes over
<lilyp>have a clean git tree :)
<lilyp>local changes being everything that shows up in git diff
<reza[m]><zamfofex> "reza: I think most VPS hosts..." <- Thanks for the reply, I just checked out hetzner where this is not possible
<ss2>reza[m]: with Netcup it was quite easy to set up a Guix host.
<ggoes>can anyone lend some guidance on what i might be doing wrong here? the file populates in modprobe.d with the correct info, but isn't being read. i still have to echo stuff to /sys/module/hid_apple https://paste.debian.net/hidden/538cf6bb/
<reza[m]>Does anyone has experience with the ant build system? I try to build a package but it keeps failing with missing dependencies although I specified them in the inputs
<old>`#:select? (git-predicate ".")` works if building the project from a shell, but does not work for cuirass :/
<reza[m]>ss2: Thanks for the tip, can you upload custom images there?
<ggoes>oh, maybe that i'm putting kernel-module-loader-service-type first..
<ss2>I don't remember tbh. :) Either I had supply an image over ftp, or some other way. Not sure anymore, though it did work.
<maximed>old: if you only want commits, you can use the undocumented 'git-checkout' from (guix git)
<maximed>It supports ssh:// and "/home/foo/stuff/" URIs, so they can be used for non-public things
***AIM[m] is now known as AIMOld[m]
<maximed>old: maybe replace "." by "/home/old/the/location/of/the/project"?
<maximed>If you use "." then your idea and Cuirass' idea of current working directory might differ.
<maximed>(technically 'local-file' normally encoded the location of the scheme file but there are some limitations ...
***Guest8410 is now known as roptat
<roptat>hi guix!
<roptat>long time no see :)
<zamfofex>roptat: Hello! 🎉
<roptat>I moved and it took one whole month to get internet again :/
<zamfofex>Ah. I hope you enjoy your new place!
<roptat>I do :)
<abrenon>hey roptat !
<roptat>hi!
<roptat>what have I missed since last month?
<maximed>roptat: I have been looking into not using Cargo to build Rust crate, to avoid #:skip-build? and to actually re-use previously built crates
<roptat>oh, how would that work? doesn't rust need source code of all its dependencies?
<maximed>It actually works already (and it moves #:cargo-development-inputs / #:cargo-inputs to native-inputs/inputs), but it doesn't support a sufficient amount of Cargo.toml to build everything yet
<maximed>More concretely, the terminal software ‘hexyl’ is already usable (see thread on guix-devel)
<roptat>pretty cool :)
<maximed>roptat: The rust compiler (rustc) doesn't need the source code of the dependencies, only Cargo does
<maximed>If we call rustc directly, then the ‘regenerate checksums’, ‘copy sources’ ... phases disappear entirely
<roptat>sounds like what I was doing with my soong-build-system for android tools
<maximed>Actually running rustc is pretty easily (at least, as easy as calling 'gcc' directly -- gcc has some complicated CLI corners as well)
<apteryx>maximed: keep going, that sounds awesome :-)
<maximed>the difficulty is in supporting a sufficient amount of features used in Cargo.toml, and setting the right environment variables for build.rs (= Cargo's equivalent of ./configure)
<maximed>Also, it turns out that Rust actually supports shared libraries! It's not restricted to static libraries.
<roptat>oh, I had no idea
<maximed>In fact, all ‘proc-macro’ crates need to be compiled into .so libraries!
<roptat>so can we build everything as shared libraries?
<maximed>Not yet, for non-proc-macro crates I get ‘personality ...’, ‘don't know a panic function’ (forgot the exact error message) errors
<maximed>though if someone understands the exact problem, should be easy, yes.
<bjc>is there a compelling reason to build everything shared? because there are compelling reasons not to
<maximed>roptat, bjc: Though from what I've heard, Rust crates often depend on LTO for performance
<bjc>as an option on the build, i don't care, but by default, i think static makes more sense for guix in general, and rust in particular
<maximed>So maybe some kind of balance could be made, e.g. rust-rustls being made shared because of relatively low cost of security updates
<roptat>bjc, static build needs grafts aren't possible, that's a lot of rebuild when there are security issues
<bjc>security updates come with static libraries too, since you have to rebuild the depender when the library changes anyway
<maximed>bjc: Also the usual space savings argument.
<roptat>*static builds mean grafts aren't possible
<bjc>guix is space hungry, and space is cheap anyway
<maximed>bjc: Grafts aren't possible for static libraries, but they are possible for shared libraries.
<bjc>like, if you're using this system, you probably have a ton of hard drive space
<roptat>my server has 20GB of disk space ;)
<maximed>bjc: Space is cheap, but some time ago my disk utilisation was about 90%.
<bjc>hmm. why aren't grafts possible for static libs?
<maximed>And a 'guix gc' couldn't lower it much.
<roptat>you could graft the static library but it won't fix any of the dependents, since the vulnerable code is statically compiled into them
<bjc>anyway, for specific use cases i can see why you'd want shared libs, but i think those are the minority these days
<roptat>you have to rebuild the dependents
<maximed>I had to search for roots to delete and such ..;
<bjc>you should be rebuilding all dependents when a library changes. that's the same whether it be static or dynamic libraries
<maximed>bjc: Also, static libraries make the output of "guix graph --references" practically useless.
<bjc>not doing it for dynamic libraries leads to hidden failures
<maximed>bjc: No, not in case of grafts.
<maximed>bjc: Possibly.
<maximed>But for some packages, where grafts usually happen such as libxml, openssl, gnutls ... the ABI is relatively stable, with promises by upstream.
<maximed>Also for glibc.
<maximed>glibc appears to be very careful about backwards compatibility!
<maximed>But yes, for many dynamic libraries, ‘simply’ adding a graft can cause trouble.
<bjc>we're not talking about specific libraries, though, but a general system
<bjc>so any argument has to apply to the general case
<maximed>bjc: Do you know in advance which libraries will be vulnerable?
<bjc>no, so the conservative estimate, given two choices, is that they all are
<bjc>because the other choice, none, isn't reasonable
<maximed>A priori, all libraries potentially need to be grafted in the _future_, so they need to be shared libraries _now_.
<maximed>Of course, when we actually graft it, we have to be careful about determining if we're not breaking anything.
<bjc>i don't know what you mean by "grafted". you mean making them available in "well known" locations outside of "/gnu/store"?
<maximed>E.g. by backporting individual patches only targetting a function body.
<maximed>bjc: It's like modifying /gnu/store/.../bin/hello in-place to replace /gnu/store/...insecure-glibc.so by /gnu/store/...-secure-glibc.so, except it isn't in-place and respects the immutability of the store.
<roptat>graft is a mechanism in guix that rewrites the reference to a store item (creating a new store item in the process, we don't modify anything)
<maximed>bjc: Grafts are documented in ‘15 Security Updates
<maximed>
<maximed>There's also a blog post on it I think.
<bjc>seems like i have more reading to do, then
<roptat>it's very lightweight compared to rebuilding dependents, since it's essentially a sed
<maximed>For large store items like openjdk, or worse, that large texlive package, it's slow though.
<maximed>Because it has to sift though hunderds of MiB.
<bjc>i'm sure it's more lightweight, but i'm not sure how much weight (no pun intended) i give to that. shared libs have a bunch of downsides, and there's a build farm for making all the substitutions already, right?
<maximed>bjc: The build farm is not infinitely fast or infinitely efficient and energy is not infinitely cheap.
<bjc>if i were maintaining my own system by hand, i'd go shared libs because it's easier to update one library and deal with the fallout. as everything has gone automated, the drawbacks of shared libraries have come to outweigh their benefits in most cases in my eyes
<maximed>In fact, sometimes we delay package updates to bundle them together, in the core-updates branch.
<maximed>Because the build farm cannot handle it.
<maximed>(in a reasonable amount of time)
<bjc>i'm not trying to suggest it's free to build, but, again, i don't know how much weight i give to that
<maximed>This is sometimes called ‘rebuilding the world’ -- do you want to rebuild everything, including things like the rust toolchain that can take multiple weeks to build?
<maximed>(At least, multiple weeks on a laptop, or was it one week?)
<bjc>if your build farm is a laptop, you should get another build farm
<maximed>Maybe a bit exaggerated but it's something among these lines.
<bjc>i had a substitutes error a few weeks ago, and had to build like 6 versions of rust -- it took many hours, not a day, but a long time
<maximed>bjc: I don't envy your energy bill.
<bjc>and that's on a commodity ryzen 3950x
<maximed>Still, a day of building is long though. Then add the time to build gcc, to build the rust packages ...
<maximed>Also some packages (webkit? firefox?) take a huge amount of memory.
<bjc>yeah, it takes a long time, but with few exceptions most things don't hit every package
<maximed>Not sure what the amount was but it was more than the 8G in my laptop.
<maximed>bjc: exactly, with a few exceptions!
<bjc>yeah, i had to up the memory in my vm from 4 to 16gb to get it to build everything
<bjc>right, but those few exceptions are already batched builds at 6+ months apart, right?
<maximed>bjc: yes, exactly, we have to batch them!
<sneek>Yey! yewscion is back!
<yewscion>sneek: botsnack.
<sneek>:)
<bjc>sure. and i wonder how much extra batching would have to be done in a pure-static system
<maximed>Also, not everyone uses the build farm.
<bjc>again, if you want an option for dynamic libs, go for it
<bjc>and again, specifically with rust, dynamic crates are more downside than up
<bjc>i mean, i obviously think that in general, but it's even more the case with rust
<roptat>bjc, and while we batch them, we can still use grafts for security updates, we don't have to freeze everything and pray nobody attacks our systems ;)
<maximed>Even then (about the existence of a build farm and rebuilds), there's such a thing as ‘climate change’, which is caused (among other things, neglecting a lot of subtilities here) by too much energy consumption.
<maximed>I don't know the % on how much IT contributes but it's certainly non-zero.
<bjc>datacenters are a significant energy impact
<bjc>build farms are not the main cause by any stretch of the imagination, though. they just don't use enough power even at full throttle
<bjc>mainly because the competition is asics and gpus and high performance scientific and ml stuff, but still
<maximed>Best avoid rebuilds where feasible (--> batch updates, shared libraries to allow for grafting allowing for avoiding rebuilds)
<bjc>and i wonder, since we're talking about energy use, how much it costs to rebuild a bunch of stuff vs hav ld.so go nuts every time a binary is launched
<maximed>Is there anyone who knows how much the ci.guix.gnu.org / bordeaux farm actually uses (energy, energy cost, ...)?
<maximed>bjc: There's an optimisation and a blog post for that!
<bjc>even if we did, there's no way to measure the excess energy use by ld.so going crazy on every guix computer every time anything is run
<maximed>bjc: https://guix.gnu.org/en/blog/2021/taming-the-stat-storm-with-a-loader-cache/
<maximed>Also, someone recentish had the idea of actually directly encoding the full /gnu/store/.../libfoo.so paths into the binaries!
<maximed>Not implemented yet though.
<bjc>if you encode the rpath for them, you can still graft them?
<maximed>bjc: The rpath (actually, RUNPATH) is already encoded!
<roptat>yes
<maximed>The grafting will replace the RUNPATH.
<maximed>I don't think the grafting actually understands RUNPATH, IIUC it just replaces every reference it sees?
<bjc>if it goes in the place where the rpath points, it'll just work
<maximed>bjc: Yes, but a binary typically multiple runpaths.
<maximed>So the loader has to search them one by one (or read the ld.so.cache I guess).
<jts>is there a way to import Nix packages into Guix? or use Nix build instructions in Guix without Nix itself?
<bjc>why does that matter?
<maximed>bjc: Why does what matter?
<bjc>why does it matter that a binary has multiple rpaths?
<maximed>bjc: because then the loader has to search them one by one (or read the ld.so.cache I guess), which is inefficient.
<maximed>Directly opening the right shared library would avoid some indirection.
<bjc>oh, i didn't think direct opening would be an option
<bjc>is there enough information in the binary to even do that?
<maximed>bjc: if the binary does "NEEDED /gnu/store/.../libfoo.so" instead of "NEEDED libfoo.so", yes (IIUC)
<maximed>but currently they just do "NEEDED libfoo.so" IIUC, so not implemented yet.
<bjc>does gcc/ld support that?
<maximed>bjc: IIUC, glibc's linker does, but I don't know if gcc and ld do.
<maximed>(ld != ld)
<bjc>glibc has a linker?
<maximed>There are two linkers here, the ELF interpreter, and the compiler's linker
<maximed>bjc: technically not part of glibc itself IIUC (unlike, say, libmusl), but yes
<bjc>the compiler/linker is what's going to write the elf headers that let ld.so find stuff
<maximed>(or maybe it is part? I dunno)
<maximed>The compiler/linker is called ld I think.
<bjc>the linker is ld, yes
<bjc>gcc just calls into ld when it builds the final binary
*maximed has to leave
<bjc>ld.so is run by the elf machinery in the kernel for shared library handling
<jts>and another, unrelated question: is there any reason guix system profiles aren't softlinked into the standard filesystem hierarchy?
<rotty1>can I refer to 32-bit (i386) inputs when running x86-64? I'm pondering running the test suite of mold as a build step (https://github.com/rui314/mold, search for "make test")
***rotty1 is now known as rotty
<roptat>jts, prevents direct reference to /bin and such from working, so we can find that kind of bugs more easily
<roptat>we want package to reference other packages, not confuse whatever's in /lib or /bin for what they really want to use
<roptat>rotty, I think it's possible, I've seen it somewhere
<attila_lendvai>jackhill, i think i have a question: the transitive closure of golang dependencies often contains several versions of the same library. is that resolved to a single one at compilation? or will each project be (statically) compiled together with a different version of the library?
<attila_lendvai>jackhill, actually, what do you think, shall we move this discussion to the mailing list? it's late here, i'll head bedwards soon...
<jackhill>attila_lendvai: sure, moving to the mailing list sounds good. I think there's is another thread on the mailing list already about go and guix shell and the can probably be combined.
<jackhill>I don't know what happens with the multiple versions. That doesn't sound intended
<attila_lendvai>jackhill, i've ran the importer on go-ethereum, and some of the dependencies have 10+ versions in the result
<jackhill>attila_lendvai: probably too much to take in now, but a lead go dev's thoughts on version selection. Of course, we're not yet using the module-aware code paths https://research.swtch.com/vgo-mvs
<jackhill>attila_lendvai: I think it might be a good goal for us to produce a "what we're trying to do" document and then maybe bring it to the go mailing list and have folks there opine on strategies we might use to acheive that.
<attila_lendvai>jackhill, thanks! i think i skimmed that article once, but apparently it wasn't enough...
<attila_lendvai>jackhill, the go build system is deep into the sidequest of a sidequest rabbit hole for me... :)
<jackhill>ha, yeah, I know the feeling.
<attila_lendvai>my parent sidequest here is to set up some form of reproducible build for go-ethereum
<attila_lendvai>jackhill, the answer is: "If a module appears in the list multiple times, keep only the newest version." (from that article)
<jackhill>ah, cool
*attila_lendvai gives it up for today and waves goodbye... o/