<Apteryx>sneek: later tell slim404: that guild error is normal, or at least innoffensive. It is because guild compiler front script is trying to auto-compile the Guile module it interprets. It usually saves this to $HOME but here fails to do so because in the build environment there is no $HOME.
<IpswichTriptych>So, I'm unsure of how to proceed. I cannot complete a guix pull because it invariably times out.
<IpswichTriptych>My thought is to guix build all of the largest packages... but for example "guix build java" is not possible...
<IpswichTriptych>Any advice? For clarification, I'm trying to install a simple GuixSD configuration on a Lenovo X60 with onl y 1GB of RAM and 4 GB swap. The installation was a success, but I have so far been unable to complete guix pull because once the swap is utilized, the compiling step takes so long that the process times out after 3600 seconds
<IpswichTriptych>I have more RAM on the way, but the X60 motgerboard only allows a maximum of 3GB, so this hardware configuration may simply not be the right one
<civodul>i had been meaning to write it down for some time already :-)
<efraim>hmm, with binutils-2.29.1, gcc-boot0's libcc1.so can't find libstdc++.so.6 in the runpath
<htgoebel1>roptat; Now that maven is finished we need something like a maven-central importer :-)
<Apteryx>Was there a change recently where compilation messages are now hidden by default?
<Apteryx>I have some guix commands takeing forever and I have no idea what is going on.
<Apteryx>I think it's because I changed git branch and lots of Guile modules need to be interpreted (I should run make).
<Apteryx>efraim: this happens when I try to test the addition of "perl" as a native-input to gcc-4.7.4... Apparently it tries to rebuild the world but I have no idea what is going on, there's no output on the CLI and top says guix is using all of my CPU (apparently no subprocess are doing compilation, so it's purely a Guile/Guix thing).
<Apteryx>Eventually it dies with a stack overflow exception...
<efraim>I think GCC5 is the next time native inputs get explicitly declared
<efraim>We use GCC 4.9 and 5 for building up from the bootstrap binaries
<Apteryx>yes, that's why I had to declare perl as a native-input in gcc@5 as well.
<Apteryx>(I'm really trying to fix all of our gcc-*). Except the gcc-boot0 apparently which I shouldn't touch?
<Apteryx>Circular dependency of perl... this happens when A <--requires-- B anb B --requires--> A ?
<efraim>And GCC-final, but with all the inherits the safest option is to work with 4.7 4.8 6 and 7, and to 'undo' the changes that would happen to 4.9 and 5
<efraim>Yeah, although I'm trying to think of the circle that adding Perl would make
<Apteryx>hmm, is it really a concern that perl is part of our bootstrap closure (is it not already?)
<Apteryx>I would think that perl is needed to build the kernel Linux, no?
<efraim>I'm pretty sure we're using Perl-final for GCC-final
<bavier>civodul: another issue with spack is having a *compatible* python installed
<Apteryx>efraim: for gcc-final we are recycling perl-boot0.
<Apteryx>it's used indirectly through the closure of texinfo-boot0 that is used for gcc-final.
<Apteryx>yep, and the texinfo is already used for gcc@5, which has "perl" has an input. So my patch should be OK as long as I remove perl for the gcc-boot0 package (with alist-delete as is done for texinfo) and add perl-boot0 for gcc-final... and fix the circular dependency. WDYT?
<efraim>Is Perl-boot0 the only Perl in commencement.scm?
<efraim>If there's still a curxlular dependency guix should let you know somehow
<Apteryx>efraim: yes, perl-boot0 appears to be the only perl dependency in the commencement.scm module.
<Apteryx>Is guix really able to warn for circular dependencies?
<efraim>Its normally stack overflows or unending 'thinking' before building
<Apteryx>random question: are info manuals meant to be read in a mono-spaced font? It seems some formatting would break otherwise, but at the same time, a roman font would be more readable for long texts.
<janneke>Apteryx: info manuals, well texinfo can be printed through latex
<janneke>the parts that need to be mono-spaced should be inside some @verb[atim] environment
<sneek>slim404, Apteryx says: that guild error is normal, or at least innoffensive. It is because guild compiler front script is trying to auto-compile the Guile module it interprets. It usually saves this to $HOME but here fails to do so because in the build environment there is no $HOME.
<sneek>slim404, Apteryx says: the real problems seems to do with a segfault in the tests: line 5: 1116 Segmentation fault ...
<lfam>This issue about location-addressed resources (tarballs) changing their contents is frustrating!
<janneke>lfam: thanks for your insight, not just a github thing
<lfam>But, ultimately, a pervasive issue that I bet will affect every single package source in time
<lfam>I noticed today that the upstream source for our net-tools package is 404 :/ I guess that SourceForge's Git snapshots are cached for a short period and only generated when they can tell a human is requesting it through the web browser, rather than through the URL we are using
<lfam>I should email SourceForge and ask if they can change this. I really wanted to avoid fetching net-tools with Git
<janneke>thinking about the content addressed-url, laertus (or someone using --no-substitutes) could use guix download on that url and thehn continue?
<lfam>Or maybe ask net-tools to make a new release
<lfam>Yeah, in this case I think the user should handle it ad-hoc, janneke
<janneke>lfam: for a moment i thought it was impossible for a user to get around and we needed to consider making a release -- if there's a workaround then we're good, sorta ;-)
<lfam>Well, in my opinion, if they are committed to using --no-substitutes, and the old source is not available anywhere, they could always just use the new source. This will invalidate substitution of all referring packages, but it shouldn't matter if they are using --no-substitutes
<lfam>But, we should try to find a better solution for core dependencies like libgit2
<lfam>Also, I hope the libgit2 team will start releasing their own tarballs
<janneke>lfam: how would they use the new source? they would need to copy the .scm and change the hash...that's much to ask as part of a `guix pull' install?
<lfam>Hm, true, I didn't consider `guix pull`. I'm not sure how it would work for that case
<janneke>lfam: yeah, that was the case at hand. you could argue the need to support --no-substitutes but otoh having that is kinda essential.
<janneke>yeah, libgit2 is somewhat special for us...i hope that they take some responsibility somehow
<lfam>Well, immediately, I guess the best option is to recommend they take our substitute / content-addressed copy of the relevant tarball
<laertus>i see the need to get a tarball of libgit2 if you don't already have git.. but... i already have git
<lfam>One of the suggestions in the upstream libgit2 discussion was to use the hash of the uncompressed (or maybe unpacked) tarball. This is interesting, and some projects sign that instead of the tarball.
<happy_gnu[m]>I was told you are working in packaging Go and Syncthing
<laertus>another thing i was thinking is if they are having problems with tars, maybe they could use another format, like zips or something
<lfam>laertus: I've seen HTTP interception garbage come in response to `guix pull` or other Guix downloads. We have to take a defensive stance. And since most of the compression and tar implementations are written in C, I'd probably argue against exposing them in this way
<laertus>i'm not sure if there is some better bit-reproduceable archive format than tar.gz
<laertus>but it's still a larger attack surface, no matter how you slice it
<lfam>laertus: Are you familiar with the concept of fixed-output derivations? In Guix, they are derivations that we know the result of in advance, like package sources.
<happy_gnu[m]>lfam: why the last ones? Are you not packaging anymore :( ?
<laertus>lfam: not really.. i've only heard of derivations (in the guix sense) for the first time yesterday.. but i think i understand what you mean
<lfam>happy_gnu[m]: No! :) I mean, I hope they are the latest Go build system and Syncthing patches before I push them :)
<lfam>laertus: There is a chapter on derivations in the manual. Basically, they are an intermediate level of build scripts, between the packages you see in 'gnu/packages' and what the guix-dameon reads and executes. When building a package derivation, we don't know what the hash of the result will be until we build it. Fixed-output derivations are simply derivations where we *do* know the hash in advance
<lfam>laertus: I'm interested in removing fixed-output derivations from the category of "substitutes", so that people who have security concerns with substitutes can get package sources in a more reliable way
<lfam>It should be something you can turn off, however, in case you have privacy concerns with fetching sources from sites beside the upstream source
<laertus>lfam: i think i'm going to have to read up on how that works exactly before i feel i can trust it
<lfam>The general problem of old versions of Guix being unbuildable without substitutes is more general, however. A while ago, we had a similar issue when a GnuTLS test accidentally failed to "wrap" the expiration date of a test TLS certificate. You can't build that version of our GnuTLS package anymore unless you change the date of your system (dangerous!)
<lfam>I recommend reading the chaper on derivations and reading a few derivations (they are the .drv files in /gnu/store). You will see that a fixed-output derivation is called "fixed-output" because it includes the hash of the result of running it.
<lfam>If Guix runs the derivation and the result does not match the hash, the result is rejected.
<civodul>lfam: you can't exactly guix download the /file URL because it'd have the wrong name
<laertus>lfam: i'll read the guix manual on derivations, but from your description it sounds like there's only a hash of the output of running it, which means a hash of the files generated, but i'm concerened that there could be a kernel exploit in the code which does not generate any file fingerprints to hash in the first place
<laertus>could a fixed-derivation be crafted to generate the same exact files (and thus the same hash) as another fixed-derivation, but also contain an exploit that the latter did not have?
<lfam>Yeah, that sort of issue is always a concern
<lfam>Wait, I said "Yeah" in reply to your earlier message.
<lfam>I don't have an answer for your second message.
<laertus>a hash of the original source tarball seems to be a solution for this problem that a fixed derivation does not address
<lfam>A derivation is a build script that is used to create a thing in /gnu/store (apologies to civodul for surely butchering the terminology and finer points)
<lfam>Any time Guix downloads a package source, it runs a derivation that downloads the source
<lfam>Any time Guix builds a package, it runs a derivation that builds the package
<lfam>Some derivations are "fixed-output", because the include the expected hash of the result of running the derivation
<lfam>About your previous question, "could a fixed-derivation be crafted to generate the same exact files (and thus the same hash) as another fixed-derivation, but also contain an exploit that the latter did not have?"
<lfam>It's an interesting and real problem, but not limited to the case of downloading things.
<lfam>The software we rely on is super-buggy and that will be the case for a long, long time
<lfam>A sufficiently motivated person could probably make your computer do anything they wanted if they are able to intercept your internet connection and mutate HTTP traffic, or convince you to download a file somehow
<laertus>lfam: you said earlier "I'm interested in removing fixed-output derivations from the category of "substitutes", so that people who have security concerns with substitutes can get package sources in a more reliable way"
<laertus>i guess i don't fully understand why they're in substitutes in the first place
<laertus>i only have a very tenuous grasp of how guix works, and definitely need to read more before i can comment intelligently on this
<lfam>I don't know the original motivation. I suspect a combination of privacy concerns and it being an implementation detail that is tricky to change
<lfam>Like, perhaps somebody doesn't want the Guix or GNU projects to know what software they use. They could achieve that with --no-substitutes.
<lfam>But other people with privacy concerns may be comfortable with GNU knowing what software they use, but don't want all the upstream projects to know. They have a different response to the question of "privacy"
<laertus>i meant more of why derivations were in substitutes
<lfam>Derivations are not the things that are substituted. They are instructions for building a package, downloading a source tarball, and other things like that. The results of the derivations are what can be substituted.
<lfam>I was thinking we could split the idea of "substitutes" into pre-built binaries and source tarballs, basically. It's just an idea, and mabye not the right approach, IDK.
<lfam>As for "so would using the guix project's own substitutes be any less secure than building from source?"
<lfam>If you build from source, you are relying on the security of your own computer to build the package properly. If you use our substitutes, you are also relying on the security of our servers to build and serve the right binary substitute. So, the number of things you trust has increased.
<laertus>i guess i'll stick with source, then, for now
<laertus>though my computer is so damn slow at doing that from guix for some reason
<laertus>my last "guix pull" took over 9 hours, with about 4 hours of that being just automake testing itself
<laertus>and that guix pull didn't even complete (because of that libgit2 issue)
<laertus>i'll definitely try to see if i can modify the guix program to not do tests
<lfam>Currently, there is a bug in Guile that causes the Guile compiler to be slower and use more memory than it should. We aim to fix this but it will take time.