IRC channel logs
2015-03-23.log
back to list of logs
<abnegate>even a rough estimate would be better than nothing <abnegate>so far this "guix pull" has cost me nearly 700 MB <abnegate>will it stop before it uses 100 TB? who knows? <mark_weaver>guix builds everything using its own software exclusively <mark_weaver>so if you start with an empty store and substitutes disabled, you need to bootstrap the entire system using a method roughly analogous to that described in Cross [GNU/]Linux from Scratch. <mark_weaver>how much disk it will require depends on what you are installing, but I guess for the prerequisites to run 'guix pull', probably less than 10 GB, or not much more. <Sleep_Walker>abnegate: and by the way - we have returning the money policy :b <Sleep_Walker>have you any experience with any source based distribution? <abnegate>i'm running guix on debian at the moment <mark_weaver>actually, even stage 1 is a lot more than the ultra-minimalist bootstrap tarballs that Guix starts from when not using substitutes <abnegate>i wasn't expecting to be installing a whole operating system <abnegate>just a package manager and maybe a package or two <Sleep_Walker>abnegate: it lives on the same FS, but is (should be) independent on host OS <abnegate>maybe i'd have felt different if i had some warning <Sleep_Walker>abnegate: could you help us with proper words selection for such warning? <abnegate>"A typical first install of Guix should be expected to take around 10 gigs of space" <mark_weaver>abnegate: that's only if you take that rather unusual choice of avoiding substitutes. <mark_weaver>even with gentoo stage 1, you're starting from a lot of precompiled binaries <abnegate>which said nothing about keys or subsitutes or anything <Sleep_Walker>I think that you can fit 10 gigs with all packages and some of them even build locally <Sleep_Walker>Right now I have ~13 Gb used and I have a lot of mess here from my packaging attempts <abnegate>on a somewhat related tangent: i wonder how many guix users actually verify gpg key fingerprints and use the web of trust <Sleep_Walker>having Enlightemnent, Xfce, and some other WMs, something even multiple times <mark_weaver>abnegate: you mean checking checking the sig on the guix tarball or installation image? <mark_weaver>I always check gpg signatures for the source tarballs I download before putting the sha256 into the guix package recipe. <mark_weaver>however, I must admit that it's very rare for me to have a path in the web of trust from a key I trust to the upstream key. <mark_weaver>and several crucial core upstream packages don't even sign their packages <abnegate>yeah, unless the web is verified it's not very trustworthy <abnegate>also, i'm not sure what the point is of allowing the user to proceed to download source and build when he hasn't authorized the public key from hydra <abnegate>if guix will stop the user from using substitutes in that case, why does it allow the user to build from source <abnegate>it's not like building from source using an untrusted guix is any better <mark_weaver>hopefully the user checked the signature on the guix tarball, which contains sha256 hashes of all the sources used in guix packages <mark_weaver>using binary substitutes implies that you trust the integrity of our build farm. building from source that does require that trust. <abnegate>when i did a "guix pull" i got this warning: "warning: ACL for archive imports seems to be uninitialized, substitutes may be unavailable" <abnegate>and guix proceeded to start building stuff from source <abnegate>as you're still getting info on what to build and how from guix <abnegate>so if using substitutes in this scenario is risky, so's building from source <mark_weaver>the flaw in your logic is that you are treating everything in guix as equally untrustworthy. <abnegate>the source you'd be being fed could be full of exploits, for all you know <mark_weaver>but if you open that black box, you'll know that binary substitutes depends on trust in a larger set of machines. <mark_weaver>and there's also the fact that when building from source, you have an opportunity to actually audit our recipes, most of which are quite concise. <mark_weaver>things like the sha256 hashes of upstream tarballs can be independently verified <mark_weaver>if you'd like to do some of that independent verification, please do! it would be helpful. <abnegate>but in practice, the whole point of using a package manager is that it does all the drudgery for you <mark_weaver>to me, it sounds like your argument could be used to say that downloading *any* source package and building it is no more trustworthy than downloading a precompiled binary from upstream. <abnegate>i really doubt many guix users actually audit many recipes <mark_weaver>look, computer security of mainstream operating systems is in *terrible* shape. you won't get any argument from me about that. <mark_weaver>when you really get down to it, there's good reason to doubt to integrity of *any* mainstream system. <abnegate>when i did the "guix pull" without authorizing the guix public key, guix did not prompt me to verify every piece of source that it downloaded, or its download location <abnegate>(btw, good luck manually verifying the source of gcc) <abnegate>it just merrily started downloading, building, and installing TONS of stuff <abnegate>don't really see that as any more secure than grabbing binary blobs <abnegate>yeah, i do doubt the integrity of every mainstream system.. but just not seeing much logic behind guix preventing binary downloads while allowing source downloads when the user hasn't authorized its key <abnegate>i'd just prompt the user to allow use of substitutes while making it clear that he hasn't yet authorized guix's key, and the possible consequences of that <abnegate>or offer to download the key and instruct the user how to authorize it <abnegate>in any case, i'd give the user a choice of how to proceed, rather than assuming that building from source must be what he wants in this case <mark_weaver>our build farm is doing exactly that operation. so it is trusting our package recipes. <abnegate>only if you verify both guix and the source of the source and the source itself <mark_weaver>abnegate: I'm sorry, what is "the source of the source" ? <mark_weaver>the sha256 hashes of all of those sources are included in the guix source code. <abnegate>then you have to verify those hashes are right <abnegate>perhaps they're hashes of compromised sources <abnegate>you also need to verify that guix is itself checking the hashes properly and not lying to you <mark_weaver>I'm sorry, I don't have time to continue this discussion. if you want to make a proposal to guix-devel, be my guest. <abnegate>after doing a "guix pull" and while guix tried to unpack guix-latest.tar.gz, i got an error: "guix pull: error: tarball did not produce a single source directory" <mark_weaver>abnegate: does /tmp/guix-file.TEuaQG still exist? if so, what is in it? <mark_weaver>are there any directories in /tmp whose names start with "file" ? <abnegate>that might have been from my previous, aborted attempt to do a guix pull <mark_weaver>we've never received a report of this problem before, and I'm not really sure how it could have happened. <mark_weaver>is there another directory of that form in /tmp with a recent timestamp? <mark_weaver>it's probably related to the old version of guile. we aim to support 2.0.5 but it's not often tested lately. <mark_weaver>the version of Boehm GC in wheezy is also quite ancient <mark_weaver>I would recommend building both Boehm GC and Guile from source code. <abnegate>will i need to recompile guix when i get a new version of guile and gc? <abnegate>looks like upgrading guile from 2.0.5-deb+1-3 to 2.0.11 fixed that "tarball did not produce a single source directory" error <abnegate>what directory do i need to add to my $PATH in order to get all of the binaries that guix installs? <abnegate>"For each user, a symlink to the user’s default profile is automatically created in $HOME/.guix-profile. This symlink always points to the current generation of the user’s default profile. Thus, users can add $HOME <abnegate>/.guix-profile/bin to their PATH environment variable, and so on. <abnegate>actually.. nevermind.. looks like it did happen after my first install <abnegate> warning: collision encountered: /gnu/store/p3ak0a9q67l0cxdgiynvaj78rm3k940z-info-dir/share/info/dir /gnu/store/sjhf79ml0ysdlqxy2z0c8vg8q32jl2kd-emacs-24.4/share/info/dir <abnegate> warning: arbitrarily choosing /gnu/store/p3ak0a9q67l0cxdgiynvaj78rm3k940z-info-dir/share/info/dir <abnegate>is there any way to prevent this or fix it? <Sleep_Walker>I made patch for that dbus-system service I'd like to test - what I need to do to use this code? <rekado_>hmm, openblas fails to build on a different machine. Will investigate. <rekado_>what module would be most appropriate for lshkit, a "Locality Sensitive Hashing Library"? <rekado_>I see that hashing libraries are in gperf and mcrypt already. <rekado_>or is it best to put it in maths.scm? <taylanub>if (append (package-transitive-inputs mpv) (package-transitive-native-inputs mpv) (package-transitive-propagated-inputs mpv)) contains only lua-5.2, how come 'ldd /gnu/store/...-mpv-.../bin/mpv | grep lua' returns a path to lua-5.1 ? <taylanub>I made sure the store directory corresponds the current state of the package object <Sleep_Walker>I made patch for that dbus-system service I'd like to test - what I need to do to use this code? <Sleep_Walker>(lua-5.1 could be some dependency of dependency, couldn't it?) <taylanub>Sleep_Walker: I think transitive-inputs probably cover all, but maybe it's just almost-all <taylanub>the hash I see in ldd output is not to be seen in the 'inputs' alist passed to a phase procedure. <taylanub>hm, I guess ldd can reach more packages than what are inputs to the package (or bag). I'm confusing inputs with dependencies. <Sleep_Walker>I made patch for that dbus-system service I'd like to test - what I need to do to use this code? <taylanub>ugh, finally figured it out. ffmpeg links libquvi, which links lua-5.1. <taylanub>and I think lua-5.2 just doesn't appear in ldd because, lo and behold, it provides no .so file in first place <iyzsong>Sleep_Walker: to test the dbus-system, I think you want '/path/to/guix/pre-inst-env guix system reconfigure ..' (run by root). <bavier>hmm, patch-and-repack doesn't support zip archives... <rekado_>icedtea7 builds when I resume the failed build with guix environment, but fails with guix build. <rekado_>this probably means that it links against something in the standard system paths. <rekado_>the build output of icedtea7 looks ... different from what icedtea6 produces. <civodul>bavier: yes there's an open bug, we should fix it in core-updates <civodul>rekado_: if you're on an FHS distro then yes, it might be using /usr/lib stuff <bavier>it looks like the "locales" input from %standard-patch-inputs doesn't ever even make it to the builder <civodul>right this is fixed in core-updates too :-) <rekado_>BTW: on Fedora with an SSSD account I still cannot use Emacs; I remember that this was something related to the bootstrap tarballs. Will these be updated in the next core-updates or have they already been updated? <bavier>oh, this might be non-trivial. I get a vm stack overflow error if I try to add unzip to %standard-patch-inputs and a conditional in decompression-type for the "zip" suffix. <rekado_>also: I still haven't been able to fetch texlive-texmf-2014 from hydra. It's already been a couple of days of trying repeatedly. Will try to build it without substitutes instead. <rekado_>Downloading big files from hydra often fails with corrupt, incomplete files. <civodul>bavier: there are bootstrapping issues <civodul>unzip should be added only when actually needed, as is done for bzip2/gzip <taylanub>rekado_: I just fetched it from upstream <rekado_>taylanub: that's what I'm trying now. (I wonder why guix seems to freeze after stating what derivations will be built --- what is it doing?) <civodul>rekado_: updating the list of available substitutes? <freaj>Hello everybody! I come from #Parabola (Freed Archlinux) and I'd like few informations about gnudmd because I plan to replace systemd by gnudmd (actually I can run openrc+eudev but gnudmd+eudev is the goal), anybody? :-) <civodul>freaj: be aware that dmd is simple, and most of the heavy lifting takes place in Guix itself <civodul>also, ATM dmd does not react to uevents <freaj>Okay, so it's a bad idea to replace systemd with gnudmd? We will have a lot of issues? <civodul>well, depending on what your expectations are <civodul>but typically if you want the full desktop experience, you'd run into troubles <freaj>Okay okay then... what kind of troubles? <civodul>as i said, lack of support to handle device events, things like that <civodul>now, i don't know if it's worse than what OpenRC provides, maybe not <civodul>the good thing is that before using it as PID 1, you can give it a try as an unprivileged user <freaj>Okay and last thing I wanted to ask: can I package it like.. a standard package everyone would install, with a separated configuration like a /etc/gnudmd.conf or is it incorporated inside the make? <civodul>it has a standard ./configure && make && make install procedure <civodul>so yes, you can make it a package in your distro <freaj>Okay but what about the config? <civodul>the package doesn't have to install one <effa``>Hi, each time I install or remove a package from my profile I get the following (and similar) warning: warning: collision encountered: /gnu/store/ql3w1ry6ch3hgswxxmsxn6jmb7b9vzd4-gmp-6.0.0a/lib/libgmp.a /gnu/store/1sjpwj0hlazs8nhx50dhihx6nr9mskjq-gmp-6.0.0a/lib/libgmp.a <effa``>However, I don't have gmp in my profile. Any idea? <bavier>effa``: gmp may be a propagaged input for one of the other packages in your profile <effa``>I mean, 'guix package -I' doesn't show it, but in ~/.guix-profile/lib/ I can see a link. <effa``>If it's propagated shouldn't I see it in 'guix package -I'? <mark_weaver>effa```: ~/.guix-profile/manifest will have the details <mark_weaver>this is part of the reason why I try to upgrade everything together in a given profile. <mark_weaver>otherwise you end up with multiple versions of the same package installed via propagated-inputs. <mark_weaver>you can also end up with libA that's linked with libC-1.0 and libB that's linked with libC-2.0, and then if you try to link libA and libB together, it will try to link in both libC-1.0 and libC-2.0. <davexunit>propagated inputs certainly are problematic. <mark_weaver>in theory, there are things we could do to reduce the number of cases where propagated-inputs are needed. <mark_weaver>e.g. for header files from libA that include headers from libB, those #include directives could be rewritten to refer to absolute paths. <mark_weaver>likewise, in theory we could perhaps enhance pkg-config, libtool, etc, to cope with library dependencies in a better way. <effa```>mark_weaver: thanks! I never realized that looking into manifest can be quite useful :-) <effa```>Indeed I see that guile and libgcrypt use different versions of gmp. <mark_weaver>in my own profile, I see that guile, guile:debug, and gnutls all propagate gmp <mark_weaver>interesting. I have libgcrypt in my profile as well, but it doesn't propagate gmp. <effa```>sorry, I wrote libgcrypt, but I meant gnutls. <effa```>I start to see why people here do not like propagated-inputs :-) <effa```>When you package your first program it looks very attractive... <Sleep_Walker>maybe reduce function of propagated-inputs to native-inputs of next package? <mark_weaver>well, they would go in 'inputs', not 'native-inputs', but I don't think that's a good approach <mark_weaver>and we sometimes add optional dependencies to a library <mark_weaver>we don't want to have to fix all of the packages built on those libraries every time we add an input. <effa```>what about explicitly warning the user that he should update some packages together when a situation like this happens? <mark_weaver>how about modifying pkg-config to do the right thing? <mark_weaver>and putting the absolute paths of the dependent libraries into the *.pc files <mark_weaver>and ideally coming up with a proposal that has some chance of being accepted upstream. <effa```>That would be good, but not enough. I'm thinking, e.g. about python libraries. <mark_weaver>I really think it's bad if we have to modify 100 packages when we add an input to some library that those packages are based on. <mark_weaver>the other piece would be to make sure that when a package is built, those indirect dependencies get added to the chroot. <mark_weaver>if the *.pc files include absolute paths to the dependent libraries, that would be enough. *Sleep_Walker is mentally exhausted for today so don't take me seriously <mark_weaver>effa```: regarding python libraries, I don't know enough about python to evaluate that. but reducing uses of propagated inputs would certainly require many different mechanisms to be improved. <mark_weaver>two examples that come to mind are: C #include files and pkg-config *.pc files. <mark_weaver>and of course programs that run other programs and assume they will be in $PATH <mark_weaver>and then there are uses of 'dlopen' in various places <bavier>and most all perl package needed their inputs propagated <Sleep_Walker>(IOW which package caused installation of some certain package) <davexunit>you could use the manifest and traverse the dependencies for each derivation? <davexunit>you of course will get a derivation as a result, not a scheme package object. <taylanub>Sleep_Walker: 'guix gc --referrers' might do <taylanub>hehe. you can install it with --no-substitutes for now. it will just take a whole to download 3 GB <Sleep_Walker>I tried guix as an experiment ~2 weeks ago and didn't switch back to gentoo since then :) <taylanub>so you'd like to see what triggers the installation of something, as you're trying to install something else <taylanub>have you tried --dry-run --verbosity=999 ? <mark_weaver>civodul: in order for GuixSD users to have http-pipelining in the guix-daemon started by dmd, we need to update the guix snapshot, right? <civodul>currently i'm just running it manually from the build dir <civodul>one thing i haven't tested against is 'guix publish', but i'm confident <mark_weaver>I tried it at one point and found it notably slower, but we can work on that later. it will be good to avoid the threading problems. <civodul>it's clear that it will be more sensitive to server slowness <taylanub>just noticed guix adds $foo/include to CPATH and $foo/lib to LIBRARY_PATH for any input $foo. in that case why is it sometimes necessary to pass these manually to ./configure under foo_CFLAGS and foo_LIBS? <civodul>mark_weaver: good news; i hadn't realized patchelf was this broken even on mips <civodul>taylanub: what makes you say it is sometimes necessary? <taylanub>:-) actually, one time so far in my experience: had to do in in libquvi because lua provides no .pc files. in mpv, it came out I can just use check_cc(lib='lua') in the wscript (waf) file, and I didn't need to add -I or -L paths manually <taylanub>surprised me at first, then seemed natural given CPATH and LIBRARY_PATH are set, then I wondered why I needed to do it in the past... <civodul>does waf clear the environment before invoking the compiler? <civodul>so the SConstruct file has to explicitly let LIBRARY_PATH and CPATH through, or you're doomed <taylanub>I had to pass foo_CFLAGS/LIBS in libquvi, which uses autotools. I didn't need to pass them in mpv which uses waf. <civodul>it could be that foo's headers are not in $foo/include but rather in $foo/include/foo-1.3.4 <taylanub>hm, not in this case. maybe passing foo_CFLAGS/LIBS to ./configure just serves to prevent pkg-config from being called, which will report failure due to the missing .pc file... <civodul>pkg.m4 generates code to handle these variables <taylanub>mark_weaver: qt-5.4.1 before 3e71b9ffd6 (on b021a2a) seems to be present on Hydra, having the hash ww33y9z5ng4a6k2vkyk9gn8lpj8jxvhh. unless I did something wrong: checked out b021a2a and did './pre-inst-env guix build -K qt' which said it will download qt-5.4.1 (along with everything else) <taylanub>er, with --system=i686-linux in that guix build -K line <taylanub>on c90a50e it says it will build qt. that will probably take all night, so I'll see tomorrow :-)