IRC channel logs
2017-07-11.log
back to list of logs
<lfam>Does anyone where the manual database (mandb) is on GuixSD? <reepca>being a bit unfamiliar with automake, before I dive further down the rabbit hole, could anyone tell me what the default for libexecdir is? <civodul>hmm i'm rebuilding ncurses several times <civodul>regarding RPCs that we'd like to group together, grafts that we'd like to group as well <rekado>python-numpy-documentation no longer depends on the big texlive package <rekado>moving on to python-matplotlib-documentation. <rekado>actually, moving on to python-ipython first <rekado>python-ipython also builds without texlive now <rekado>civodul: what do you think about building the debian package for Guix on hydra? <rekado>I’d like to keep this from bitrotting <jsierles>what features does the guix daemon need that require the 'privileged' flag on docker? <jonsger>rekado: could you install .deb packages with dpkg on guix? <efraim>rekado: I might have a wip for dpkg <efraim>it was incredibly easy, I never got around to splitting out the 5 packages into separate outputs <jsierles>rekado: do you have ideas for how i could import all of CRAN? at least to get an idea how many packages need special attention <roelj>So, after doing a `guix environment --pure guix' and then compiling guix from source from that environment, when I try to run ./pre-inst-env guix environment ... I get the infamous "substitute: ;;; Failed to autoload make-session in (gnutls):" <roelj>What have I done wrong here? Should I do guix environment --pure guix --ad-hoc gnutls specifically? <rekado_>roelj: what does GUILE_LOAD_PATH say? What version of Guile is avaliable in the environment? Are you using an old version of Guix to set up the environment? <roelj>rekado_: Guix from a week ago or so, I'm already out of the environment.. sorry. <roelj>Do I normally have to add gnutls to the environment? <rekado_>roelj: I think it’s part of the guix environment. <rekado_>ACTION has no idea how to build debian packages <efraim>rekado roelj: I had the same issue, I think just './pre-inst-env doesn't set env variables for guile-SSH guile2.2-gnutls or guile-json <efraim>I also have no idea, that's part of how I ended up with guix <efraim>As a one-off you can do 'debuild' and that'll build a deb <efraim>Mostly I think Debian developers use sbuild or cowbuild or something and not by hand <rekado_>I already told it not to check for dependencies (because I’m not running debian, so I don’t have them) <rekado_>but it also seems to require “dh”, which I don’t have yet <civodul>rekado_: re building the debian package, why not, but i'm not sure what it'd take to do it <civodul>probably that needs to be done in a Debian Vm <rekado_>civodul: yeah, looks like it isn’t easy to create the deb without Debian. <rekado_>If it’s not going to be added to the official Debian repositories anyway we might as well cheat. <rekado_>another backend to “guix pack” or “guix archive”? <civodul>rekado_: i guess 'guix pack' could create zero-dependency Debian packages <civodul>the resulting .deb would actually include everything <civodul>not very elegant, but that could work <davexunit>"f'ing package management" - a ruby tool to quickly pack files into a variety of package formats <davexunit>I use it at work to generate deb packages for rails applications <rekado_>I used it when I started here to quickly package RPMs <civodul>there's also "checkinstall", which can generate a variety of binary packages <efraim>Checkinstall is more useful iirc for compiling locally and registering the package with apt/aptitude <jsierles>can someone help me with the above compilation error? it's making it impossible to update guix. <civodul>jsierles: is it reproducible with current master? <civodul>(also, leave more time for answers, because we're all busy with other things!) <jsierles>just keep running into roadblocks and got frustrating :) <jsierles>doesn't make sense that it would fail given i just ran it successfully with the same git hash. i'll report back when master is done <jsierles>thanks for adding the wget and libsoup fixes <jsierles>once I get infrastructure setup, I hope to be able to contribute more. <rekado_>hmm, not sure what to do when a profile or multiple packages are given as an argument to “guix pack -f deb”. For now I’ll just create a minimal control file. <civodul>rekado_: "guix pack" always creates a profile <civodul>so it shouldn't make any difference if it has more than one package no? <rekado_>I just thought I could create a proper “control” file, reusing e.g. the package description. <janneke>does this ring a bell: guix-daemon: nix/libutil/serialise.cc:15: virtual nix::BufferedSink::~BufferedSink(): Assertion `!bufPos' failed. <civodul>janneke: vaguely, how did you achieve this? <janneke>./pre-inst-env guix build hello --no-substitutes ==> offloading to a fresh ubuntu+guix machine <efraim>signing-party 2.5 can't find sha1.h <civodul>janneke: is it the latest guix-daemon? <civodul>argh yes, the version string is wrong <janneke>civodul: it's v0.13 -- but we have similar* machines that work <janneke>ACTION * similar should have been `identical' -- now helping our sysadmin to script installing ubuntu+guix <rekado_>eacces: “system” should be replaced with “x86_64-linux” or whatever your system is. <rekado_>eacces: see the sentence right after the URL <eacces>oh that bits clear, its just the version string that's wrong <jsierles>civodul: guix pull failed for master too <civodul>jsierles: it worked for me on master a couple of minutes ago <jsierles>copying and compiling to '/gnu/store/mjpv734c8xlzrzr0ffaazrnksxv5mwjc-guix-latest' with Guile 2.2.2... <jsierles>i'm running guix pull from a fresh install of 0.13.0 <civodul>no idea what could be wrong, unless there's some non-deterministic problem <jsierles>sorry, I was using the tip of core-updates. the above is what I just fired off to test as well <jsierles>but master was merged into core-updates recently. <jsierles>civodul: OK - what's the correct way to get master? <jsierles>OK. why would that get something different from the tip of master? <civodul>"guix pull" gives the tip of master so it's equivalent to the command you gave above <civodul>but you said you were fetching core-updates? <jsierles>OK. yes, sorry I made a mistake - I had fetched the tip of core-updates and not master. now I'm trying master <jsierles>civodul: just checking one thing, I may have accidentally used a cached copy of a previous checkout because of docker. last try now. <civodul>good that you insisted on ME/AMT too <jsierles>civodul: no luck. definitely am using master. <paroneayea>that interview kind of ties in at the end with that whole worse is better thread davexunit and I had on Mastodon/the fediverse recently <quigonjinn>if we need to update two packages, and one will break without the updated other, it's best to update both in the same commit, right? <civodul>jsierles: how many cores/threads on this machine? <civodul>i wonder if this could be an issue with parallel computation <civodul>ok, so it's more likely to trigger thread-safety problems that the machines we have <civodul>could you try restricting it to fewer cores to check that hypothesis? <jsierles>ok, so I tried now with just 'guix pull' and it worked. <civodul>i thought you had already tried that? <jsierles>which you said was the same thing. but i see the output is not the same. <reepca>Hm, trying to test executing builders in the build environment stuff I'm testing and I'm getting "ERROR: In procedure execl: No such file or directory" despite being able to verify in the REPL that the file to be executed does indeed exist and has execute permissions set. And it isn't specifying *which* file or directory doesn't exist. <civodul>now, if it's a non-determinstic issue, it could be that you were lucky <civodul>could you try several times in a row? <civodul>reepca: it could be that that file doesn't exist *in the container* <civodul>strace would allow you to see you which file doesn't exist <reepca>the REPL is the same one I'm trying to execl from <jsierles>civodul: will do - i updated the gist to show the different outputs. since they are different, I think that may be the issue right? <reepca>GAH welp, never mind, turns out that I was thinking of a different file <reepca>Now I just need to figure out why 6l4q8s3g3b1hf0f6z0dhqn0fkgpq29vg-bash isn't getting included <davexunit>paroneayea: I love your description of guile: "Guile, which is a kind of Scheme, which is a kind of Lisp, which is a kind of programming language with lots of parentheses everywhere." <ng0>Does speech-dispatcher-0.8.5 fail for anyone else with a 404 on hydra even with --fallback? This is via offloading here <ng0>it can be grafted on the build machine, but if I offload the build from master this depends on speech-dispatcher fails <ng0>and local build just doesn't happen. <ng0>an endless stream of the 'process aquired at bla for build bla' <ng0>has something touched offloading? <ng0>a recent commit I mean <ng0>trying an full update of the buildmachine now, maybe it helps <jsierles>i'm now seeing this when trying 'guix pull' as well: fatal: could not read Username for 'https://gitlab.com': No such device or address <ng0>I had to delete the content of the mirror btw, because of disk space issues.. no idea how to handle this better. I <ng0>'m getting an upgrade soon <ng0>the gitlab.com issue.. hm <ng0>guile-ssh is on github.com .. why does gitlab.com come in? <civodul>jsierles: can you send that to bug-guix@gnu.org so we keep track of it and address it? <ng0>jsierles: oh, guile-git not guile-ssh <catonano>rekado: ludo mentioned that you are working on a B plan for the Guix build farm. Something about repurposing used hardware. Would you mind to say something about that ? How is it going ? What is likely to happen ? <ng0>should we wrap rtorrent? It only runs correctly with https trackers when you start it like this: rtorrent -o http_capath=/etc/ssl/certs <lfam>ng0: Hm, what do other distros do? <ng0>archlinux recommends that <ng0>what others do, no idea <ng0>but maybe this can be set in the rtorrent.rc <lfam>Is the Arch wiki really recommending that users add a root certificate without checking its authenticity?! <ng0>and by the way we do not include the sample config file in the output.. which isn't bad when you have a collection of old configs but it's not good if you have to consult their source for what we should have locally <lfam>That Arch wiki snippet makes no sense <lfam>I guess the right question is, does rtorrent have a default path where it looks for certificates? Does it check any environment variables for a certificate path? <lfam>If so, we can patch them <ng0>i think the -o can just be added to the config.. but its been years since I used rtorrent <ng0>I think offloading broke again :/ <ng0>I'll just build locally <lfam>Otherwise, the location of the certificate store varies between distros so it's hard for us to do the right thing in our packaging <lfam>Somebody should really report that part of the Arch wiki to their security team <lfam>It's bizarre and dangerous <ng0>no idea, I just noticed it <ng0>feel free to copy this conversation into a new bug <lfam>Wow, the Arch IRC channel is really not interested in fixing that problem <lfam>A chorus of "fix it yourself" and "then why do you cares" <ng0>I would have expected nothing else from them <lfam>I don't even understand why those instructions are relevant <rain1>i dont understand why .screenrc needs to be told about ssl certs <rain1>screen is launhing rtorrent, I get it <ng0>Gentoo forum says it can be added to the .rtorrent.rc <catonano>lfam: they are sitting where ? In the institution rekado works for ? <catonano>lfam: so that institution has to approve the move ? <ng0>to some degress wrong, for guixsd the folder worked for me <lfam>catonano: rekado can give more accurate information than me but I think he needs approval to make the servers publicly accessible <ng0>(not booted into Gentoo atm to check it) <catonano>yesterday I attempted to create a virtual machine after a long time and it started building a kernel. So I stopped it. Today I tried again and it started building some gobject reated stuff. I stopped it again <catonano>Today I read that Gnome is moving to a new build system (I can't remember its name now) and they are happy because it's faster than the autotools. I was mumbling that keeping Gnome up to date will require a functional build farm. <lfam>catonano: Let us know if you remember the name :) <lfam>We'll want to be able to support that build system whenever they make the switch <lfam>Looks like it uses ninja under the hood <lfam>mesonbuild.com's TLS is configured improperly :( <catonano>Polari, the Gnome chat client (that I love) already switched to Meson <catonano>lfam: I can see mesonbuild.com with no glitches <lfam>catonano: You should package polari using meson so that we are ready :) <rain1>A lot of thinsg are switching to meson very fast! <lfam>No for me in Firefox or Chromium <lfam>They are trying to use a certificate that is only valid for *.github.com <catonano>lfam: you're right. It doesn't work in Web but it wors in Icecat. Maybe it was in the cache ? Boh <lfam>This happens all the time for sites that are hosted by github <lfam>I guess they are making a build system, not a web site :) <rain1>rtorrent https #1 result is that wiki page <lfam>rain1: It has spread all over the net. I found it on some bug tracker discussions and forums too <rain1>i can't just delete the bad bit', I have to replace this with a high quality instruction set <rain1>this gentoo link seems to be the best <lfam>Is it even necessary? Do rtorrent users really need to fiddle with individual certificates? <catonano>I wanted to try again with hvirtual machhines to try to learn the services thing in GuixSD. Tryton needs a service <rain1>lfam: good question.. I should find a https tracker to test this on. I know ubuntu gives linux in torrets <rain1>it is only http hmm.. this is hard to test <lfam>I mean, those instructions are nonsensical. Obviously somebody had a tracker that used that certificate, and their CA store was too old to contain it. So, they fixed it for themselves without understanding what they were doing, and then they told the entire world to copy their example. <reepca>catonano: I'm surprised that enough of the build time consisted solely of configuration and other build-system-dependent stuff that they were able to reduce the build time by that much... <catonano>reepca: I know nothing about build systems. I understand that they're a necessary evil and I carry on <lfam>rain1: That tracker's TLS appears to be set up correctly for testing. At least, it works for me with latest Debian Stable CA store and `openssl s_client` <lfam>Awesome, thank you rain1! <lfam>That's a huge improvement <catonano>reepca: maybe I was a bit disenchanted. I know nothing about build systems though <reepca>catonano: I don't know much about them either. <jsierles>ok, so compiling guix on a 64-core machine fails a lot. dont try it at home! <reepca>ACTION mumbles something about not having a 64-core machine at home <catonano>I hope the reconditioned servers that rekado is woring on are not 64 cores machines ! <jsierles>package builds work fine. it's just compiling guix itself it seems <rekado_>catonano: the servers have to be accessible via SSH from the outside. <rekado_>catonano: because of that we don’t want them on the same network as all our other servers. <rekado_>some servers in the data centre hold confidential patient data etc. <rekado_>so the Guix servers must be on a separate vlan. <rekado_>furthermore we don’t want them to have network access to any other server, so it’s best to have one of them in the DMZ acting as a bastion server <rekado_>all other servers would be on a separate vlan, which can only be accessed via the bastion server <rekado_>and that’s what I want IT to approve. <rekado_>I wanted to have this ready two weeks ago, but it looks like this takes a little longe. <rekado_>I’m pinging them twice every day, though. <lfam>Do people eat doughnuts over there? I would be bringing them doughnuts every day ;) <rekado_>I don’t think that would speed things up. <rekado_>one reason seems to be that many people are out for summer holidays. <rekado_>but I can only guess. Other than receiving the paper work I didn’t hear from them yet :( <jsierles>how often does/would the build farm check out master and build? would it have to do the equivalent of 'guix pull' each time? <lfam>jsierles: We run it manually as often as possible, typically one or twice a day. It doesn't do `guix pull` something else that I don't understand fully :) <lfam>I mean, "but rather something else I don't understand fully" <rekado_>jsierles: cuirass on bayfront fetches from git every few minutes. <catonano>rekado_: thanks for this detailed information ! <rekado_>meanwhile I’m thinking about how to improve the installation procedure. <jlicht>I just used guix pack to share a couple of simple tools with some of my non-guix using coworkers. It works perfectly! <rekado_>all these servers use the same hardware configuration <jsierles>rekado_: ok, so cuirass is what distributes builds across the farm? <rekado_>unfortunately, there’s no way to install GuixSD over PXE <jsierles>would be great to see the entire config somewhere. we would also like to setup a build farm if we can get guixsd running on GCE <rekado_>jsierles: builds are distributed via offloading <rekado_>jsierles: did you take a look at the maintenance repository yet? <rekado_>davexunit: it looked like it would work over iPXE, but then it waited for the USB disk to appear… :( <rekado_>jsierles: it contains the system configuration for bayfront and the nginx config for the mirror. <jsierles>rekado_: thanks. i'll check that out. now trying to get a guixsd image running on GCE <rekado_>jsierles: since bayfront never actually took over full duties from hydra I’m not sure if it actually contains offloading configuration, though. <rekado_>I also tried to install GuixSD once and then create a disk image for installation to another machine. <rekado_>but I still had to download and build a lot of things. <catonano>jlicht: probably in autumn I will share a tiny pipeline that involves my guile-freexl thing, with guix pack too <rekado_>I’d like to create a fat installation image with a big store cache so that I don’t need to download anything while installing the machines. <rekado_>for the first two servers I actually installed them in the hallway because they were so noisy and the downloads / builds took much longer than I thought. <jsierles>rekado_: yes, that's exactly my goal now. what did you have to install? <rekado_>jsierles: it was little more than the barebones configuration. <jlicht>catonano: cool! The only thing that I am a bit worried about is distributing any security updates to my coworkers for the software in the pack <buenouanq>that's actually a great idea rekado_, they should offer it officially <jsierles>i'm confused about substitute server setup. i have mine setup without caching, so I expect anything that's in the store to be available. however, my guix install frequently compiles things anyway <jsierles>I see requests coming in, some for 'narinfo' files and others for /nar/gzip <jsierles>for example, I just completed a build for 'r-minimal' on my build server. yet my local install insists on compiling dependencies it should be downloading. <civodul>jsierles: while debugging, use --no-grafts on the client side, to avoid confusion <civodul>second, if you're using "guix publish" with --cache, the first request for a store item is always 404 <civodul>don't use --cache if this is not desirable <jsierles>civodul: not using cache here. that's why it's confusing <jsierles>with no-grafts, i just see a huge list of derivations that get queued for buiding <civodul>then make sure the .narinfo requested are really on disk on the server <civodul>pick one and check whether the server returns 404 or 200 for its .narinfo <civodul>do that for an xyz that's in /gnu/store on the server <jsierles>ok, one thing I noticed is the hash for a particular derivation is different than the one on the server. how can that be, if both use the same verison of guix? <civodul>there can be several .drv file names mapping to the same output(s) <civodul>don't confuse the .drv hash and the output hash, though <jsierles>so I get a 404 for the narinfos I see in the logs <civodul>pick the hash of a store item known to be in /gnu/store on the server <civodul>for instance, run "guix build coreutils" on the server <civodul>pick that hash, and run wget as shown above <civodul>(not sure what log you were talking about) <jsierles>ok, i meant the logs from the substitute server <civodul>there's no reason why something shown that appears there would be 200 <jsierles>so now i need to see why the local daemon is not finding that <civodul>anyway, long story short: if 'guix publish' returns 404 for a narinfo, that's because it doesn't have the thing in store <civodul>IOW, the client is asking for something the server doesn't have <jsierles>yes, so why would the client ask for something different? <jsierles>is there a way to see what the hash the client would try to fetch is? <jsierles>given the exact same version of guix in both places, and trying to fetch a core package (r-minimal), i'd expect it try to fetch the right hash. <jsierles>expat is a dependency of coreutils, and dry-run suggests it will be built. but, with --no-grafts i was able to download it. <jsierles>then, that dep is removed from the list of builds for coreutils. <jsierles>here the 'acl' dependency is first listed as a build, but then as available for download. <jsierles>a couple dry runs on 'guix build acl' also show some non-deterministic behavior. weird <jsierles>ok, I can confirm that for all deps that are listed to be built, the narinfo file is accessible over http <civodul>oh we actually have access to your server :-) <civodul>the "first shown as a build, then shown as available for download" may be misleading <jsierles>ok, now when i ran 'guix build --no-grafts r-minimal --dry-run' all deps show up for download <civodul>well dunno, but AFAICS it works as intended :-) <jsierles>also: ERROR: Bad Range header: bytes: 0-22 <civodul>"connection reset by peer" is fine, just means the client went away abruptly <jsierles>i tried running 'guix build r-minimal' and it started compiling PCRE <jsierles>now it looks to be starting a compile of gettext and gcc. <civodul>in general 'guix publish' is supposed to survive client misbehavior <civodul>well now you know how to debug anyway :-) <jsierles>not sure - seems like i'm geetting a different result every time. <civodul>another thing you need to know is that clients have a local cache, in /var/guix/substitute/cache <civodul>so if you want to be sure, "rm -rf /var/guix/substitute/cache" <jsierles>every time i run this command, it shows different results. that suggests something is very wrong, no? <jsierles>one says 'package will be downloaded'. run it again and it says 'package will be built'. or am i misunderstanding? <civodul>AFAICS you did not pass --ttl; i recommend using it <civodul>i've never seen it display different things if you run it twice in a row <civodul>now hydra.gnu.org advertizes short TTLs sometimes, when it's "baking" an archive <jsierles>OK. so i have to turn the cache on basically? <civodul>i don't see why you get "updating list of substitutes" the second time, because everything should be in /var/guix/substitute/cache already <jsierles>you mean the local cache is always on? i meant turning on the substitute server cache <jsierles>i see a cache directory locally, and it's being written to <jsierles>now python has added itself to the list of downloads :) <jsierles>is there any more client-side debugging i can do? <jsierles>since the server appears to be working fine. <jsierles>running against hydra's mirror seems to work fine though. <jsierles>for r-minimal, all packages just get downloaded *shrug* <jsierles>ok. i enabled it on my server, same results. <civodul>well don't enable it now since it complicates debugging <civodul>do you have only your server, or do you also have another one? <civodul>on the client side you could check what's in the files in /var/guix/substitute/cache/ <civodul>consistent with the server responses and with what you expect <civodul>you can open the files therein, they're just wrapped narinfos essentially <jsierles>it looks to me like the client is not actually checking my server for all of the substitutes <jsierles>i cleaned out the cache, and ran `guix build --no-grafts r-minimal --dry-run` <jsierles>but only see two requests on the server. shouldn't it look for every dependency? <civodul>no, it only checks for things it doesn't already have <civodul>so r-minimal, plus its build deps not already available locally <jsierles>ok - it reports all of the deps as 'would be built' except one <jsierles>why would it report a dep needs to be built if it has it already? <jsierles>the narinfo looks different in cache. it's basically empty <civodul>if it's a multiple-output derivation, it has only some of its outputs, and the missing outputs aren't available as substitutes <civodul>jsierles: what's the name of the file you pasted here? <civodul>(value #f) means that the server did not return 200 <jsierles>i just grabbed it manually and got a full narinfo. <civodul>can you remove this file and run "guix build r-minimal -n" again?