<dftxbs3e>marusich, you can give up single core builds, they arent the problem. It doesnt reproduce still. I have a feeling the reproducibility problem doesnt come from building the gcc-static package itself
<dftxbs3e>on my server's VM, when I guix gc then rebuild gcc-static single-core it reproduces
<dftxbs3e>on my laptop's VM when I guix gc then rebuild gcc-static single-core it doesnt
<dftxbs3e>so I am guessing guix gc doesnt clean up something that's causing reproducibility issues on the server VM
<marusich>dftxbs3e, yes, that's the conclusion I've drawn from my attemtps, too. Something in the inputs is different which carries forward to the outputs... We could try to walk the graph of derivations backwards and find the paths of non-reproducibility to see where it first gets introduced, but it is a very tedious affair.
<dftxbs3e>lispmacs[work], I don't know much about that :-S - too young
<dftxbs3e>marusich, it bothers me so much we can't reproduce GCC !!
<dftxbs3e>marusich, can't wait for commit access so we have a branch where we can maintain work
<marusich>Yes... I have thought about it, but have not found the time to build anything
<marusich>There is some graph machinery in Guix already, for getting information about graphs and printing graphviz diagrams. If I did implement something, I would first try to re-use that somehow.
<marusich>Part of the issue is that it requires comparing the same derivations from two different machines. I suppose "guix inferiors" could help with that. However, for debugging I wonder if just manually walking the graph backwards for now would lead to an obvious conclusion...
<dftxbs3e>marusich, do you have the motivation and energy for that? I'm a noob guiler, I couldnt create such a program for now
<marusich>I mean, it could be fun. It might not help us in the end, but it could be fun at any rate. More fun than babysitting broken builds to experiment with new ways to find out that the problem is hard.
<marusich>I was hoping to move forward with the bootstrap binaries we had built already. But I do want to make GCC reproducible, too.
<dftxbs3e>marusich, janneke suggested we look at reproducibility of object files in case it's the linking machinery at fault, since it doesnt reproduce on section offsets etc being different AFAICT
<dftxbs3e>marusich, have you noticed as well that --cores=1 doesnt affect cross-compilation?
<dftxbs3e>I had to make my VM actually have only a single core for it to work
<dftxbs3e>marusich, well when powerpc64[le]-linux-gnu-gcc runs, it's make -j$(nproc), even with --cores=1
<marusich>For GCC there are potentially two issues I found. They are mentioned in https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41669 but it is long bug report, so a summary would be: One problem is libstdc++.a, which contains different (identical) members if you build it with guix on debian vs. if you build it with guix on fedora; the cause is not clear. Another issue, beyond that, is that other binaries in the gcc-static output do not build
<marusich>reproducibly if you build all the inputs from source, even though they do build build reproducibly on two different systems if you re-use the exact same inputs.
<marusich>I see. Well, some packages will do that. Maintainers may have chosen to parallelize their builds or not respect the common ways of making it parallel. It is possible but not great for that to happen.
<marusich>So, to find the cause of libstdc++.a being non-reproducible, somebody would have to investigate further than I could do. I tried to reproduce the libstdc++.a issue using a recent gcc, but I could not do it, so I think it was a bug that was fixed somewhere between gcc 5 and 10.
<marusich>To find the cause of the more insidious problem of the build being reproducible with content-identical (transitive) inputs, but not reproducible when all inputs (transitively) are built from source, I think the best path forward would be to analyze the graph of derivations to trace those "paths of nonreproducibility" from gcc-static going backwards.
<marusich>I suspect that if you can find the first derivation(s) that have differing output on two different systems, you might have a chance to figure out why they differ and work towards a reproducibile gcc-static.
<marusich>What's an easy way to convert a derivation file name (e.g., /gnu/store/...foo.drv) to a derivation record in scheme?
<marusich>Someone knows the answer, but I'll look at some of the code to figure it out if nobody knows...
<dftxbs3e>marusich, I'm still here was re-reading the thread
<marusich>dftxbs3e, I will try to experiment making a little tool that walks the graph and can help us understand what derivations / packages are introducing the non-reproducibility across systems.
<marusich>In the meantime, I still think we should plan on using the binaries we built.
<dftxbs3e>marusich, right, let's do that. We'll keep on trying to solve this issue in parallel.
<marusich>In the best case, we can replace gcc-static with something built reproducibly later.
<dftxbs3e>Since it's not very motivating to dig this problem, having other problems to solve that come after in the chain will help us diversify the work and stay happy with a good struggle/success balance.
<dftxbs3e>marusich, you sent an email with your binaries already, I'll send one with little endian ones
<dftxbs3e>marusich, or can you do it since you seem to have a web server and all to do it..?
<marusich>OK. I'd like to reproduce what you did, so if you know the precise commit, that would be helpful. Otherwise, I'll use commit 662e7e28d576ada91fc9dec7d27c100666114f03 (gnu: libffi: On powerpc, patch with --force.)
<dftxbs3e>everything was make -j1 independently of GNU Guix's --cores=1
<marusich>OK. It'll still be slow, but I'll let you know once it's done. In the meantime, if you could send me your ppc64le bootstrap binaries, that would be great. Please sign them if you have a PGP key.
<dftxbs3e>marusich, I didnt keep them, rebuilding now, in my experience building bootstrap binaries from scratch on that machine is quite fast (unless you choose --no-substitutes..)
<marusich>Yeah, I'm trying to avoid using substitutes.
<marusich>I am running "guix pull" and will try to build the ppc64le bootstrap binaries from commit 662e7e28d576ada91fc9dec7d27c100666114f03 on a spare laptop of my own, as well as on the two VMs you made available to me. The VMs will probably finish sooner, since I'll run them with 12 cores each (the VMs each have 30 GB of RAM, so to avoid the OOM killer 12 cores seems reasonable).
<marusich>I find that when I try to build in parallel a lot, the OOM killer kills builds sometimes when I don't have at least 2 GB of RAM per thread available
<dftxbs3e>marusich, you could try adding zram-device-service-type
<marusich>I think 12 cores should be fine, though; in practice a lot of builds don't use all the cores all the time
<dftxbs3e>it also wont work for patch updates, you can't handle patching differently based on GNU Guix's checkout version
<raghavgururajan>sneek, later tell dannym: What shall we do regarding the wip-desktop patches that was reverted in master? I think you were mentioning something about building on a ryzen system. Any luck?
<atw>hohdfenhfdjlke: ime, packages do not break often (that's just my qualitative experience). I have occasionally had a package break, eg http://issues.guix.gnu.org/43610, but I could work around it by rolling back my profile: undoing is a very first class feature
<atw>I wanna give a sense for how using guix actually looks, so here's the command I used: guix package --switch-generation=-1
<ryanprior>If you can run it normally under strace on your system and send that output to me, or read through it and figure out what all icons it loads, that might give us the clues we need to get it working in a pure environment
<ryanprior>Presumably because I lack some files that your system has
<wleslie>pure gets rid of environment variables required for connecting to X, no?
<ryanprior>On my system, the pure environment still has DISPLAY=:0 set
<alextee[m]>im not sure what exactly is the problem on the guix side but what it's doing there is trying to load an icon from the zrythm-dark icon theme, which inherits from the breeze-dark theme, which has that icon, if this is of any help
<alextee[m]>the zrythm-dark icon theme should be in `~/.guix-profile/share/zrythm/themes/icons/zrythm-dark`
<ryanprior>I tried to open the file you sent and gzip is telling me "not in gzip format"
<nefix>hello! does anybody know how can I build a Go package and all its subpackages? I'm trying to build 'google.golang.org/protobuf', but I'm having to define a Guix package for each subpackage that has, and there are internal packages so they can't be build. Can't I just build a "source-only" package or something like that? Thanks :)
<efraim>OriansJ: this is after you pull? so code for current master, not from the 1.2.0 release
<efraim>I suppose you could try pulling with one core or using a checkout of the repository to just skip the tests for now
<efraim>OriansJ: I think you can also try rebuilding just that package, guix build /gnu/store/nn0wfw<tab>
<efraim>you're also missing the part of the build log with Error 1, which shows the actual error
<zamfofex>Hello, Guix! I feel like this must be asked frequently, but how do I compile a C program in Guix that I can then share with others? When I shared a binary with a friend, they got a “No such file” error, and using `readelf`, it seems the culprit is the dynamic linker path, which the executable asks to be at `/gnu/store/...`, which is not
<nij>Hello :D -- 1. Coming from NIX, I wonder if guix has things like home manager and nix flakes? 2. If I really want non-free software (for temporary use), is it possible to get it using guix as a package manager (not OS)? 3. How to write a declarative file specifying which package I want, rather than =guix install= every time (NIX equivalent: the file =configuration.nix=)? 4. Any tutorial that teaches packaging from scratch
<nij>request? (I am comfortable with Lisp and can understand how hello-2.10 is packaged.)
<zamfofex>nij: I’m a Guix newbie myself. But regarding flakes, I have heard they are not available like in Nix, but the alternative approach in Guix is to use channels (which solve the same kinds of problems, but are not as tidy). (Not that I know what either do, I’m just basing this on previous conversation in the channel.)
<zamfofex>nij: Also, I think there should be some non‐free repositories for Guix that are not maintained by GNU. I remember it being mentioned in a video about Guix once.
<bavier[m]>reversions like that should have a rationale, yes
<nij>Not sure if this is the right place to ask.. I'm using thinkpad x220 and am quite happy with it. But I am aware that probably some hardwares are "non-free". I wonder what's a good way to replace all of them to be free, or even how to build a free laptop by myself.
<bdju>with the X220 you can replace the internal WLAN card with one which needs no blobs, although there's a hardware whitelist you must first bypass either with a modded BIOS or coreboot install
<bdju>and the coreboot install itself is another step to make the X220 a bit more free, replacing the default BIOS
<bdju>the coreboot install requires a hardware flasher. you dump your flash chip's contents a few times as a backup, then flash the coreboot image to the chip
<bdju>I haven't succeeded in a coreboot install yet myself so I can't give much more help than that
<nij>bdju: thanks. Is there a definite way to check which part is non-free?
<nij>I have to at least know which parts are to be fixed..
<bdju>it's hard to define, as far as free hardware, probably none of it is, but as far as parts which need non-free software to work, that's much smaller. the WLAN card for wifi is the only thing I know of off the top of my head. if your model has a bluetooth daughterboard, maybe that needs blobs (I'm not sure)
<slimjim>hi guix! Question: is it possible to specify package (maybe via transformation? Or manually declaring one with inherit) that does not include any of the man pages, info pages, or other documentation?
<slimjim>Use case is with 'guix pack' to distribute a package to a user without guix. The resulting pack tarball/docker image includes a bunch of man pages (e.g. for the python package) that I'd like to strip out
<slimjim>as I'm not even including man-db or an info browser in the pack
<apteryx>I think transformation could be useful if it's to apply to many packages
<apteryx>otherwise here's just a couple packages you could define your own specially crafted inherited versions
<slimjim>are there any hooks in the 'pack' process where I could programaticlly start generating the pack (e.g. for a docker image) and then add a step that would recursively delete /share before creating the final pack tarball?
<cbaines>slimjim, not that I know of, but that would be pretty useful
<cbaines>you can perform that kind of transformation with a Dockerfile
<slimjim>cbaines: yeah but the reason I want to use guix is to avoid dealing with dockerfiles
<slimjim>I guess that could be a small post-processing step to import from the guix-created image, remove a directory, and output a new stripped image
<slimjim>similar question: is there a way for pack to not include native-inputs of any package? My understanding is that those are only necessary for building the package, not for running it, so removing those could substantially cut down on pack size
<rekado>if the package retains references to the native inputs they must be included
<rekado>but if the package outputs don’t hold any reference to these inputs they will not be included in the pack
<lfam>Additionally, if a package keeps references to a native-input, that's a bug in either the package definition, or the way the package is built. The dependency should be made a "regular" input, or the reference should be removed, respectively
<slimjim>rekado, lfam: ok, take the python package as an example: ```guix pack -RR -S/bin=bin -S/etc=etc python``` produces a tarball with a /gnu/store that contains gcc, which alone accounts for over 30 MB uncompressed
<lfam>The linux-libre package will always be better supported upstream
<slimjim>rekado: regarding "if the package outputs don’t hold any reference to these inputs" -- how do I check for this on the commandline? something like `guix graph --path` ?
<lfam>The newest longterm support kernel will sometimes be the latest kernel, and sometimes not. When it is not, it's not going to be supported as well as the latest kernel
<raghavgururajan>Hmm. I thought their support for lts versions are longer than regular versions.
<lfam>Yes, that's the point of the longterm support kernels. But it doesn't mean they are supported more fully
<cbaines>dongcarl, I think that could just be the client hung up or the connection dropped
<olivuser>I am on guix system and use exwm as my window manager. While logged into exwm, I cant use stuff like "dired" or properly open a file, because there always is a "directory or program not found: $PROGNAME"
<cbaines>unless it's happening repeatedly, it could just be the network
<lfam>The longterm support kernels are always going to be more buggy, and the longer they are supported, the worse they will get
<dongcarl>cbaines: Hmmm, I'm seeing it like 50 times in a row. I'm talking to my `guix-publish` at work
<dongcarl>client-side I see: "substitute: Throw to key `bad-response' with args `("Bad Response-Line: ~s" (""))'."
<lfam>It's not possible for the longterm support maintainers to backport all the bug fixes that newer kernels receive
<olivuser>Now it had been suggested in #guix that I might have to change something before "exec exwm", which i tried to figure out. I have not found anything related apart from the "xinitrc" file in my .emacs.d's exwm elpa folder. I added the "source /etc/profile" command (that has been suggested by other users).
<olivuser>Unfortunately, that has not helped, the problems persist. Is anyone else here using exwm and has (faced) similar issues and/or an idea on how to resolve that?
<raghavgururajan>I see. I was under the assumption that lts versions will be less buggy.
<cbaines>dongcarl, hmm, it's when it's writing a narinfo...
<olivuser>I already tried the source command in .bashrc or .profile, but that has not helped, and both git (opening files error) and ls (dired error) work fine in the shell
<cbaines>dongcarl, it could be there's something up with the HTTP connection handling
<raghavgururajan>Since their updates to an lts version will not add/modify any features and only fixes bugs or security issue.
<lfam>There are reasons to use the older longterm kernels, but I think that the Guix default should stay with the latest stable releases
<lfam>The "mainline" kernels are like a "latest" or "nightly" release
<raghavgururajan>> lfam: There are reasons to use the older longterm kernels, but I think that the Guix default should stay with the latest stable releases
<raghavgururajan>I agree now. I was looking into making linux-libre-lts as default, because I thought they the stable versions of kernel. But if all latest releases by upstream are stable, then this makes sense to be default.
<lfam>Does it make sense what I said, "The newest longterm support kernel will sometimes be the latest kernel, and sometimes not."?
<lfam>For example, currently, the stable kernel (5.10) will also be a kernel with longterm support. But in few months, the stable kernel will be 5.11, and that kernel will not have longterm support
<lfam>But, 5.11 will be supported as "stable" for a few months, until 5.12 is released
<dongcarl>cbaines: Hmmm, looks like those should still work tho... Strange... Might put up an issue
<lfam>During that time, 5.11 will receive the most attention from upstream maintainers for bug fixes. 5.10 will receive a little less attention, and 5.4 even less. 4.19, 4.14, 4.9, and 4.4 will get even less attention, as the bug fixes become harder to port to the older codebases
<raghavgururajan>> lfam: Does it make sense what I said, "The newest longterm support kernel will sometimes be the latest kernel, and sometimes not."?
<raghavgururajan>Yeah, currently it is the case. 5.10 is the latest but also the lts version
<lfam>The backporting work is supported commercially, but my understanding is that only a few people are actually doing the work
<raghavgururajan>I was just reading lfam: Does it make sense what I said, "The newest longterm support kernel will sometimes be the latest kernel, and sometimes not."?
<ride>gnutec: I used jami years ago and the battery drain because of DHT on mobile was too severe to be usable. I saw that this has been improved with a recent major update so I am interested in trying it again.
<ride>Do you have experience using it on mobile as of late?
<ryanprior>I mean, you can't prove a negative, no. But python:alpine runs Python apps and installs Python packages, as does Guix python.
<cbaines>ryanprior, I'm pretty sure you're using the compressed size for the Docker image, ncdu in the image says it's ~50MiB
<lfam>Smaller is definitely better, but maybe not if it takes a huge amount of CPU to shrink it (e.g. xz vs lz4) or a lot of human effort. Of course it depends on how many times the thing will be downloaded. For the home page of gmail.com, spending an entire year shaving a kilobyte off would be worth it
<ryanprior>justinmoon: it uses bootstrap binaries to build the whole toolchain from a small seed, which takes a long time and burns lots of CPU cycles, but if you're a bitcoiner that's already your domain ;D
<lfam>And of course, motivation is not fungible. If one wants to help reduce disk usage, they should do it :)
<OriansJ>justinmoon: the best to learn about bootstrap binaries is #bootstrappable where we are always working to reduce them down to "nothing" (357byte hex0 and 737byte kaem shell being effectively not in an audit sense)
<OriansJ>but be sure to ask questions as universal understanding is our goal
<OriansJ>efraim: to answer your question that is the error occurring when attempting the first guix pull on a new system without enabling substitutes.
<slimjim>lfam: I do think smaller is better, especially when my goal is to produce a pack I can give to someone not running guix. Even from an (admittedly personal) aesthetic standpoint, it seems "wrong" to include man pages in a pack that doesn't even include a utility for viewing them (and likewise for infodocs). That's why I'm trying to figure out a way