IRC channel logs
2017-07-19.log
back to list of logs
<roelj>sneek: later tell rekado: It seems that the import from a Python shell inside the eolie environment is just fine.. http://paste.lisp.org/+7IZ7 There seems to be a problem with finding dependencies, as LD_LIBRARY_PATH is needed to find libwebkit2gtk-4.0.so.37. <buenouanq>what's the guixsd way of getting a short shell thing to run on boot? <CharlieBrown>buenouanq: Unless you want to install it or do something ugly, you better write a POSIX-compliant shell script or prepare to have a headache. <buenouanq>The actual solution is for me to step up and write a legit service. <reepca>hmm. Top says "GiB Mem: 2260682544250088.0/7.782". Something seems a bit off. <dTal>reepca: can I hire some computer time from you <reepca>And restarting doesn't seem to change it. I wonder where top gets its information from... <Apteryx>How can I force a rebuild again? I have to use --no-grafts, even with --check? <Apteryx>Ah, probably --no-substitutes. But then I wonder what's the purpose of "guix build --check" if it can use substitutes... I thought it was expressly to "check" those substitutes. <Apteryx>Or maybe it's just libreoffice which is strange... I can't find the right combination to rebuild it, it just seems to graft a bunch of things together. <reepca>if all else fails, you could always try causing it to get garbage collected <Apteryx>so, "guix build --check --no-grafts libreoffice" in my case <Apteryx>Oh, right, you are working on the guix-daemon? <Apteryx>To port it from C++ to Guile, correct? <reepca>It's going... eh. I see the C++ scanner that's all "let's hash and compare 32 characters for every character we see!" and shudder, but I suppose it's probably I/O bound anyway. And of course it's not that bad, because it first checks that all 32 characters are base32-valid. But I still really feel like I should improve that somehow. <Apteryx>Sorry, I've been away for too long it seems :) <Apteryx>OK! That's the code that looks for references to graft? <reepca>Nah, it's the code that looks for references to detect in general after a derivation is built. <Apteryx>Interesting. I didn't know that such happened at that time. <Apteryx>Are these kept in the database or something? <Apteryx>By the way, which GPU did you end up choosing for your setup? <reepca>I have to wonder if it'd be helpful to let the client supply additional reference information for the cases where the output won't have the references in an unencoded form <lfam>reepca: I'm curious, like in the case of propagated references? <lfam>I mean, propagated inputs? <lfam>My naive understanding is those references are found in profiles rather than package outputs, so your idea is to also put them in package outputs? <reepca>I mean suppose the output of a derivation was compressed or encrypted. <Apteryx>Hmm. I'm not sure. What are those references in the DB used for? <reepca>Apteryx: I ended up using two 8400 GS'es. It's actually sort of lucky I ended up with two of them by accident (problem with the first one turned out to be an adapter issue), because each one only has 2 crtcs. So I need two of them to drive 3 monitors. <lfam>reepca: Yeah, that's a serious issue in practice. AFAIK, we've found and fixed two very serious bugs of that sort in Python and GCC <lfam>reepca: Those references are used for grafting, too, right? <reepca>I haven't looked at the grafting code, but I'd expect so. <Apteryx>So right now our garbage collector is "leaking" whatever reference doesn't appear in clear in the binaries/built product? <lfam>Python eggs used to be zip-compressed, obscuring references. Then, there was a string-chunking optimization in GCC that caused store references to be obscured. <lfam>Apteryx: I wouldn't call it a leak since it leads to store items being GC-ed too soon rather than accumulating ;) <Apteryx>I guess they are used for grafts; otherwise why would there even be the requirement that they *must* be unencoded to be saved in the DB? <lfam>How would you find them if they weren't in the clear? <reepca>Apteryx: the only reason they must be unencoded is because we don't see them otherwise. <lfam>When mark_weaver found the GCC issue, he suggested that it could be possible to adjust the reference scanner to be able to find the chunked references. I don't think we implemented this, however. <reepca>but conceivably the same issue could apply to grafts - patching a reference that is split up or unrecognizable doesn't really work. <lfam>It helps illustrate the severity and complexity of the issue of references in Nix / Guix systems <Apteryx>lfam: No it was deemed to complicated and prone to fail if the compiler changes; so GCC was patched to not do that instead. <Apteryx>*too, *changed. I need to reread myself before hitting that enter key... <Apteryx>reepca: Granted, we don't see them if they're not plain text, but it seems at that point in time in the daemon you have the knowledge of opaque references as well. If those references are added and also used by grafting I would think they would cause problems there. <lfam>Often I find unecessary or miscategorized inputs in package patch review. Sometimes dependencies are removed upstream when updating and we don't notice it right away. If we treated the list of inputs as the list of references, we would end up with spurious references. So, there are trade-offs either way <Apteryx>reepca: 8400 GS... With nouveau driver? I just recovered my 2006 desktop, it's equipped with a single 8800 GTS. From what I've read the power management is not working yet so it's going to be slow for 3D, but otherwise seems like it'll be fine with no binary blob. <lfam>Also, the current grafting implementation wouldn't work with encoded or obscured references, since the software wouldn't know how to execute the bug-fixed dependency. <lfam>I mean, it wouldn't work whether or not we have a way to register references when they would otherwise be obscured. <lfam>But, I wish we had a more powerful build farm so that we could rely on grafting less often and for shorter periods of time. <reepca>I'm a bit unfamiliar with most build systems - how feasible would it be to do a scan after configuring but before compiling? What would the risk of spurious references be like? <lfam>There are often references in the built objects themselves, so they need to be scanned, too. That GCC issue I linked above was a case like that. <lfam>As for the risk of spurious references, I don't know because I've never tried it :) <reepca>But in order to get into the built objects they must at some point be in the source, right? <lfam>I haven't read Eelco Dolstra's thesis on Nix, but it seems like the idea behind scanning references assumes they will never be obscured. Then, the reference scanner is a perfect idea, because it registers only things are actually referred to by the store item. <lfam>I'm not sure I understand. The source code won't always include store references after configuring. In the gnu-build-system, there will be some store references in the build directory after configuring, but I don't think it's the case for all the build systems. <lfam>And, the references that would be found at that point may only be needed for building, and so we don't need to register them. <reepca>Hmm. What if we only scanned *.c and *.h files? Could references be introduced through other means like compiler flags or environment variables? <lfam>I realize that my use of the verb 'register' may be different. We definitely want to protect build-time things from the garbage collector during building. But in this conversation I've mostly been thinking about what happens after the build is finished. <reepca>aye, that's what I think of too. <lfam>I'm not sure... I'm quite far from an expert on the Autotools / gnu-build-system. <lfam>I bet you could get some more useful feedback about this from some of the more experienced Guix hackers during the European day <lfam>civodul is way more familiar with the original Nix implementation and theory than me <Apteryx>lfam: Are you not in Europe? Or you just happen to live by night? :) <lfam>I do tend to stay up late, though ;) <Apteryx>Since our derivations are not referenced by contents, how does comparing the "hash" with say "guix build --check" of two builds useful? <Apteryx>That would also mean that unreproducible builds can share the same unique hash, correct? <Apteryx>It would only tell us that whoever built that binary used the same exact dependencies. It doesn't tell us that they haven't tampered with the final result? <lfam>`guix build --check` does compare the hash of the contents <Apteryx>I somehow thought it just validated the hash were the same. <reepca>Apteryx: there are two hashes for an output, the "store path" hash, which just depends on the inputs and derivation and which you see in the path, and an internal hash stored in the database of the actual contents of the output. <sneek>rekado, roelj says: It seems that the import from a Python shell inside the eolie environment is just fine.. http://paste.lisp.org/+7IZ7 There seems to be a problem with finding dependencies, as LD_LIBRARY_PATH is needed to find libwebkit2gtk-4.0.so.37. <gnewb>Although the proposed workaround works I wonder if it is the right thing to do, since there appears to be a vpnc-script installed in .guix-profile <rekado>There’s an archive for bioconductor source packages, but their hashes differ from the upstream files. <rekado>the hashes differ because of unnecessary changes to the DESCRIPTION file (e.g. line breaks) <rekado>there’s another archive here: bioc.ism.ac.jp/packages <efraim>Are the current versions also accessable from the archive? <rekado>would be nice if we could use that archive and stop worrying about switching to SVN or git. <rekado>sadly, the second archive does not keep older versions. <rekado>it’s just a mirror for every bioconductor release. <rekado>within releases of bioconductor it just keeps the latest version of packages <rekado>I get an error: ERROR: no code for module (guix build utils) <rekado>that’s when installing a package <rekado>caused by/gnu/store/jw6hlszg34zb2cjzg47cf4mdwhlj7ydq-info-dir-builder <rekado>I was doing something wrong. Updated without “make clean-go”. <fturco>hello. i'm trying to install guixsd on my laptop <sneek>Welcome back fturco, you have 2 messages. <sneek>fturco, ng0 says: with Gentoo experience it helps to compare the Gentoo Handbook and the GuixSD documentation. I found it relatively easy at first, compared to the first Gentoo install I did (Gentoo nowadays is very easy for me aswell, too many installs past) <sneek>fturco, ng0 says: I wanted to do an Gentoo / GuixSD comparison after a while… I might still do it in a comparison of systems and then ultimately from a Gentoo-person perspective which parts of Guix are the parts in Gentoo you are used to, etc.. <fturco>i created two subvolumes on my btrfs partition <fturco>now i'm trying to run: guix system init /mnt/etc/config.scm /mnt <fturco>i don't know the scheme language <fturco>now it says: guix system: error: service 'file-system-/home' requires 'device-mapping-root', which is not provided by any service <fturco>perhaps btrfs subvolumes are not supported by guixsd yet... <samplet>fturco: It looks like Btrfs is supported as of 0.13.0. <samplet>Just a guess, but there might be something wrong with the “mapped-devices” stuff. <samplet>Do you have a “mapped-devices” section in your config (as part of “operating-system”)? <fturco>samplet, yes i have a mapped-devices section <samplet>fturco: Following the example in the manual, you could try removing the “/dev/mapper/” prefix from the file system definitions, and just use “root” as the device. <fturco>maybe i have to drop the home subvolume for the time being <samplet>fturco: Could be. TBH, I’m no expert; I just didn’t want your question to go unanswered. :) What’s with the UUID, anyway? Why is it all x’s? <fturco>samplet, that's not for privacy reasons... it's due to the fact that i'm installing guixsd on another machine and i have to manually type everything <fturco>i dropped the home part and now it is installing guixsd... <fturco>it cannot find binutils-2.27 (error 404 not found) <samplet>fturco: Hm. Does it suggest adding “--fallback”? <lfam>fturco: You can either add --fallback to the command to build binutils or wait a little while for the mirror to retrieve it. <lfam>If you can tell us the full URL that returned 404 we can try to make sure it's on the mirror <rekado>fturco: this could be because of the unusual caching behaviour <fturco>ok, i added the --fallback option <rekado>fturco: when a substitute isn’t yet available in the cache (but available on hydra) it first returns 404 while preparing the download <rekado>a little while later you would get a binary <samplet>fturco: Also, looking at the Guix Btrfs test, it adds a “home” subvolume manually before running “guix system init”. <fturco>samplet, i created the @home subvolume manually and mounted it at /mnt/home <samplet>fturco: That’s exactly what the test does. Nice. <lfam>ecraim: That's very encouraging! <jlicht>just got my liberated t400 in the mail D: <rekado>I’m thinking about getting a t400 to replace my failing x200s <rekado>(but I can’t convince myself to do it while the x200s is still coughing) <jlicht>rekado: that makes sense. How did you get GuixSD on there? Currently trying to make a usb stick with dd, but I have literally no clue how to get the grub(?) menu to boot to the usb <rekado>jlicht: is there a Grub menu item to search grub configs from elsewhere? <rekado>otherwise you could enter the Grub console, type “ls” to see available disks, and then do “configfile (whatever)/boot/grub/grub.cfg” if that file exists on the disk <rekado>when I start my laptop with a bootable USB drive connected it shows up at the bottom of the internal Grub’s menu. <jlicht>rekado: hmm I´ll start playing with that. <rekado>when that doesn’t happen I “refresh” by selecting the last item, which searchs for boot targets. <jlicht>rekado: I think I will need to some time to process this :P. Most of these things do not seem to apply to the grub menu I can see. Thanks though <rekado>wigust: no, it’s just generally unreliable. <rekado>wigust: freezes up sometimes, corrupts the disk <rekado>wigust: every day the hotkeys fail at random points until a reboot <rekado>the shell is split open in a couple of places, but it still holds together <wigust>I bought one x200 dropped on left side. Broken plastic on corner. But it's a live :-) <rekado>wigust: well, to be fair, I did drop it more than once. <rekado>never had a problem with it until I dropped it in a hotel lobby <rekado>previous surfaces were a little more forgiving <rekado>ACTION goes to water the pumpkins <civodul>rekado: when you have a MIDI instrument plugged in, does "cat /dev/midi1" or so show something? <civodul>or is something higher-level needed for diagnostics? <jsierles>civodul: so since guix gc doesn't seem to show all my built packages, is there any other way to see them or query the database/daemon? <reepca>jsierles: is it important that you only see built *packages*, or will all stuff in the store do? Built packages are a subset of store items. "All store items" would just be the union of guix gc --list-dead and guix gc --list-live. At the daemon level it doesn't know about packages, only about derivations, but built packages are also built derivations. You can query derivation outputs from the daemon, but you'll also get stuff like the <reepca>output of a derivation that downloads source. But the derivations usually fall into patterns that you should be able to filter relatively easily. <jsierles>reepca: yeah, i can see they are filterable. just wondered if there was a distinction internally <jsierles>i only want to see packages, since the goal is to provide a simple package installation tool on top of guix, only using what's already in the store <jsierles>didn't want to do this since i have to parse out the version numbers, etc. and I'm sure that info is available somewhere I don't have to parse it out <jsierles>perhaps adding a --json option to 'guix gc --list-x' would be useful <jsierles>i suppose another concern is that since guix definitions are a moving target, some software could be in the store which no longer can be installed using 'guix package' <jsierles>for example, i have a profile with python 2.3.1, and guix updates it to 2.3.2. my old profile would still work, but i couldn't create a new profile with python 2.3.1, since there is no longer a definition for it. correct? <reepca>It would be relatively easy to check if any given package is already built, but getting a list of all packages that are built would be difficult. <reepca>jsierles: yes, it would fail at the step of trying to find a package definition to compile down to a derivation/derivations. <bavier`>jsierles: I believe the store can be queried directly with the (guix store) procedures. You could use fold-packages to check whether the store path exists or not. <rekado>civodul: oh, I’m sorry! I should have told you that I added these kernel modules: "snd-seq.ko" "snd-seq-midi-event.ko" "snd-seq-midi.ko" "snd-seq-device.ko" <bavier`>jsierles: you couldn't create a new profile with the new python, but this shouldn't be affected by something like 'guix package -i foo' <rekado>these are required for the MIDI device <jsierles>alright. so bottom line there is that I would also have to keep every version of guix package definitions around. <jsierles>bavier`: yeah. but my goal is to allow people to also downgrade, not just upgrade. <reepca>jsierles: a rollback would still work, but only if it was in a previous profile. <bavier`>jsierles: in that case, yeah, you'd need to the corresponding guix source that lead to the previous store path <rekado>jsierles: you can also install store paths into profiles <bavier`>jsierles: you'd need more than just the package definition, since other things contribute to the store hash <efraim>if I'm going to take an OS config, build it, copy it to an SD card and then use it, does it matter if the boot loader won't actually be /dev/sda1? <rekado>“guix package -i /gnu/store/…-foo-1.2.3” should work <jsierles>rekado: ok , good to know. that's pretty much what we want to do always. <bavier`>jsierles: input packages, build-system and utility modules. But maybe what rekado is suggesting could work too <reepca>jsierles: for info on what goes into a store path, see nix/libstore/store-api.cc line 97 and onward <bavier`>I guess depends on the interface you want to present <jsierles>bavier`: basically just an autocompleter with all available versions of packages. behind the scenes, we create guix profiles which get linked into your environment <jsierles>i think where this would get tricky is when the store has multiple copies of the same library, one grafted, one not, etc. <jsierles>is there a way to see enough info about them to pick one intelligently? <fturco>guixsd is still building gcc on my laptop... after 3 hours! <fturco>it needs about 1 hour with gentoo <reepca>fturco: it does a candian cross before building anything to remove dependencies on the bootstrap binaries, so that could explain it a bit <fturco>i'll leave my laptop compiling overnight... <rekado>fturco: are you purposefully compiling things from source? <fturco>rekado, no, but i had to add the --fallback option in order to go on <fturco>by the way it built gcc-5.4.x, but 7.1.0 is available too <reepca>fturco: aye, the version of gcc used in gnu-build-system doesn't get changed very much both for reasons of stability and because it requires rebuilding the world. Build glibc with a different gcc and now everything that depends on glibc has to be rebuilt to use that new version of glibc, and so on. One of the consequences of functional package management. <fturco>reepca, i like rebuilding the world, but not on my first attempt at installing guixsd <rekado>fturco: this doesn’t seem right. <rekado>fturco: I would have aborted long before it finished building stage 1 of GCC. <fturco>yes, i'm indeed considering that too <reepca>fturco: out of curiosity, which architecture are you on? x86-64? <reepca>ah, just wondering. Substitutes weren't available for i686 for awhile. <fturco>what do you think about parabola? is it a good alternative to guixsd? <fturco>it's either guixsd, parabola or gentoo <civodul>rekado: i could already see /dev/midi1 without explicitly loading these modules <civodul>loading them doesn't make any difference <civodul>i suppose my keyboard is kinda dead tho :-/ <civodul>jsierles: actually things that you built are not "live", unless they're reference from a profile or something <civodul>that may be the reason why you don't see everything you built <jsierles>but will need to do some more work. probably, track all operations we do with guix in a separate database <rekado>civodul: do you have /dev/snd/seq? <rekado>after loading the modules you should have /dev/snd/seq <civodul>so does "cat /dev/midi1" produce something normally? <rekado>I don’t have /dev/midi1, and never noticed it before. <rekado>when you start “jackd -dalsa -Xseq” it should start JACK for the audio interface and the MIDI sequencer <rekado>"cat /dev/snd/seq" doesn’t do anything for me <rekado>I do this: jackd -d alsa -s -X seq & <rekado>then: “patchage” to see all connections and devices <civodul>i tried "jack_midi_dump" after starting jackd, but that doesn't show anything <rekado>then: fluidsynth -a jack -m jack ~/soundfonts/FluidR3GM.sf2 <rekado>then I hook up the green outputs of the fluidsynth box with the green “system” playback ports <rekado>and connect the red “midi” port of fluidsynth with the midi interface in patchage. <rekado>jack_midi_dump will only print something when you connect it to you MIDI device (e.g. with patchage) <rekado>after having started “jack_midi_dump” I see a “midi-monitor” box appear in patchage. <rekado>I connect that to the red port representing my MIDI interface in the “alsa_midi” box <civodul>hmm i don't have "alsa_midi", only the "system" box <rekado>do you have anything that’s red? <rekado>this means that Jack hasn’t started the bridge between Jack MIDI and ALSA MIDI. <rekado>this doesn’t have to be a problem <rekado>let’s create a purple fluidsynth input: <rekado>fluidsynth -a jack -m alsa_seq ~/soundfonts/FluidR3GM.sf2 <rekado>this way you should be able to connect your purple MIDI interface with fluidsynth’s purple input <civodul>ok but i don't have this file so it ends up being green :-) <rekado>for fluidsynth to work you need to have a soundfont <civodul>i guess i should read a bit more before going further :-) <civodul>i was hoping there'd be some low-level diagnostic thingie <rekado>yeah, low-level diagnostic is hard <rekado>It was easier when computers had a dedicated Joystick/MIDI port <ng0>I think we should revert one piece of the tor hardened build.. 3 hours uptime: 684.3 MiB + 753.0 KiB = 685.1 MiB tor <ng0>it uses an ridoncoulus amount of memory compared to default. <ng0>half of what the bloated chromium with 55 tabs open is using <ng0>I can locate tor by looking for the only process in the list with a red "T" <ng0>I'll revert some switches and see when the RAM usage becomes acceptable <civodul>ACTION investigates this GCC nar 404 <civodul>i think nginx doesn't do what i think it does <civodul>looks like it caches things for longer than it should, which in turn leads to 'guix publish' evicting narinfos/nars from its own cache <civodul>ooooh, i think i got it: /var/cache/guix/publish is mounted... with "noatime" <ng0>first tor vm running, now I just need to wait :) <ng0>already less activity though <ng0>I think removing "--enable-expensive-hardening" is enough for most of the memory load, but I'll let this run some more time. <ng0>we could also have like "tor/hardened" with this still in place <ng0>at the moment it's at 1.6% Memory of 2048 MB in the VM <nckx>ng0: Wow. 685MiB would be *low* for my nodes after a few days of uptime. I don't recall seeing much of an increase after we enabled expensive hardening. <civodul>rekado: we see a zillion of "svn-checkout" store items, which i think come from the texlive components <rekado>I’ve started adding file-name fields <rekado>I’ll add them to the existing packages soon <rekado>I was hoping we could fix this in svn-fetch directly, but I guess it’s not too bad to add file-name fields <efraim>that was pretty mind-numbing, just put together a kernel config for aarch64 <rekado>catonano: I still need to figure out exactly what libraries must be on LD_LIBRARY_PATH, but it already works inside of “guix environment”. <ng0>nckx: with a desktop system that is almost immediately after boot <ng0>for the tor service alone <ng0>I've seen that even before we approved this config <nckx>ng0: whoops, s/days/hours/ but yes, they're not desktops. <ng0>*configuration of the tor package <nckx>What's interesting is that I just looked at my other one, and it's only ~220M on FreeBSD after 5 hours vs. ~950M after 4 hours on GuixSD, both with 2 GiB RAM. <ng0>anyway it really shouldn't happen and there should be a choice between the *marked as experimental by upstream* and the non-experimental standard <nckx>I know the two OSes account for RAM differently but that's a big difference. <ng0>it's one thing to have a hardened build, but another thing to enable experimental+hardened options which drive the memory consumption too high <ng0>I think after 2(?) hours the tor process in the VM is at the same resources wasted in terms of memory as the one on debian running for a very long time <ng0>I meant to say, it is low enough now <nckx>I'm reconfiguring without --expensive-hardening to see what difference it makes.