IRC channel logs
2015-05-02.log
back to list of logs
*paroneayea tries looking into compiling javascript stuff <paroneayea>is there a "needed for building inputs, but not for running" <paroneayea>because you only need npm if you're compiling jquery <paroneayea>davexunit: is that what native-inputs is, do you know? *paroneayea writing a jquery package <paroneayea>this checked out a ton of stuff in node_modules/ <paroneayea>davexunit: so I thought I'd look into how hard it was to compile jquery from scratch! <paroneayea>here is, recursively, how many *unique* dependencies it takes to compile jquery <davexunit>paroneayea: it's very bad that Nix just downloads the minified version! <davexunit>alright, so we're 265 packages away from having jquery in guix. <davexunit>paroneayea: also, re 'native-inputs': those are inputs that should use the native architecture version when cross compiling <davexunit>we don't explicitly distinguish between build dependencies and runtime dependencies, for a reason! <davexunit>we can inspect the store item to get the list of references <davexunit>so, when someone fetches the package from the build farm, it only pulls down the other store items that were referenced. <davexunit>paroneayea: I'm on the verge of having a postgresql-service written <paroneayea>davexunit: and yeah, 265 packages, that's unlikely we'll ever get that all installed... <paroneayea>mediagoblin has ~50 python packages total as dependencies, I think <paroneayea>that's my estimate based on looking at how many are installed in my virutalenv, plus guessing how many are from the site packages (shouldn't be many) <paroneayea>one must wonder, how deep in terms of dependencies are our javascript requirements, really? <davexunit>I don't know why, but python, ruby, and node projects have *massive* dependency trees. <paroneayea>probably because we're incentivized to use packages outside of the system package manager? :) <davexunit>and since folks essentially just upload pre-built binaries to their respective centralized package service, no one notices the dependency cycles, either. <paroneayea>I argue that efforts go get libre web applications and distros working together should have been happening more seriously a good decade ago. *paroneayea writing up a blogpost on the madness of modern web application development <davexunit>hmmm why doesn't my postgresql service work... <paroneayea>davexunit: think I should link it on the guix-devel list? <davexunit>it's crazy how nearly impossible it is to programatically capture the entire dependency tree for a web application. <davexunit>I have my work cut out for me with Ruby gems. have I told you about the cyclic dependencies in rspec? I complain about it from time to time. <davexunit>they aren't cyclic runtime dependencies, but you cannot run the test suites for rspec's depdencies without rspec! <davexunit>so building up the dependency tree is impossible without first making a bootstrap tree in which none of the builders run the test suite. <paroneayea>davexunit: someone's gonna thompson but yer rspec! <paroneayea>davexunit: have you read that (Ken, not Dave ;)) "Thompson Bug" essay? <davexunit>well in this case I'd be alright, because guix built everything, no special bootstrap binary required. <davexunit>paroneayea: not fully, but I'm familiar with the premise. <davexunit>which is why I'm curious to see the Guix fixed point experiment <paroneayea>davexunit: hey, if you don't mind posting things, might be good to post to r/linux on reddit *paroneayea learned that the hard way, mid-mediagoblin campaign! <davexunit>if either site picks up on this, I anticipate a fair number of npm apologists to chime in. *davexunit is on the verge of a working postgresql-service :) <Kolt>howdy, is this the irc for the GSD? <Kolt>or for the package manager only? <rekado->I almost got elogind installed. It fails during install because for some reason man/logind.8 is not found (even though it seems to be generated). <rekado->Unfortunately, I had to configure with "--disable-pam" <rekado->ah, got it. The XML files just need some more renaming. <civodul>rekado_: core-updates is ready to merge but gcj is still broken there (because of the dangling symlink) <civodul>so if you're around today, it would be nice to fix it, ideally before the merge <taylanub>civodul: I was hoping to get the Mesa update in as well, though maybe it's not so important. <civodul>taylanub: it can go afterwards, that's fine <civodul>if there's a risk of breakage, we can make a mesa-update branch and let hydra build it <taylanub>civodul: oh, is Hydra building core-updates already? the motivation was to avoid rebuilding stuff. <davexunit>some discussion about paroneayea's new blog post, with discouraging comments. <davexunit>I thought he did a good job describing a real issue in an accessible way, but many still fail to see an issue with it. <davexunit>ha and now I've been downvoted for disliking static linking! <civodul>taylanub: yes, it has been for a few days <rekado->civodul: gcj is still compiling... I'll submit a new patch as soon as I'm sure it does what it should. <civodul>davexunit: oooh, i just read paroneayea's blog post, it's insightful! <civodul>(and good advertisement as well ;-)) <civodul>the js/node situation is even worse than i thought <civodul>davexunit: and i agree with your comment about static linking <civodul>seems there a general trend toward static linking <davexunit>I will resist, but I don't know if we can win. <davexunit>to use paroneayea's term, it seems like the "fuck it, I'm out" approach <civodul>well, my guess is that web developers are not really concerned about software freedom or even the technical issues paroneayea raises <davexunit>I think that it's still a huge issue even, regardless of software freedom. <civodul>after all, npm gives them something that is arguably insane, but something that works <civodul>but regarding duplication, i guess there cannot be two same-named modules in a node program, right? <civodul>or maybe that's possible if one uses require('./path/to/module')? <civodul>i have quickly learnt and quickly forgotten these things when i was working with hop :-) <davexunit>funny how people don't think that 265 unique dependencies to build jQuery is an issue. <davexunit>if GCC's dependency tree was that deep, people would freak out. <civodul>well, i can also understand the fact that it's a different approach, with smaller dependencies <civodul>in the C world we tend to have relatively big dependencies <davexunit>yes, but the packaging burden is there regardless of size. <civodul>in Scheme we could have one module = one dependency <civodul>i mean Chicken's eggs for instance are probably not so different from typical Node packages <civodul>i didn't know npm could be used for client-side software like jquery <davexunit>but I guess the important thing is that to build it, you need node. <davexunit>and then from there, you could package it as a node package and as a bower package. <Sleep_Walker>were anyone succesful in running guix with systemd-nspawn? <Sleep_Walker>I can see there were discussion about that but I can't find any result of receipt <davexunit>Sleep_Walker: I haven't tried, but would be interested to see it. <davexunit>I'm writing my own container software with Guile, that will hopefully be integrated with Guix some day. <Sleep_Walker>I have it in chroot but it's without DMD so I'd like to see this running <davexunit>one question I have is: how will dmd react to being PID 1 inside a PID namespace? <davexunit>systemd detects that it's not the *true* PID 1. <cehteh>in linux-vservers each container can have a *true* pid 1 init <cehteh>why dont one want that? and wont it be possible to tag the container somehow not to use pid1 <cehteh>very old project, i use this since beginng of time :) <cehteh>except it doesnt manage kernel/hardware/network <cehteh>it uses pid namespaces since they are there <cehteh>initially it was a big stanalone patch to the kernel, but as soon things become available in the kernel the patch gets smaller <davexunit>I don't know what's different, it's just a question. <davexunit>I read that systemd changes what it does somewhat if it detects it's inside a container. <cehteh>systemd is imo a crude non unixy crap .. but i dont want to start flaming it here :D .. i admit it has some uses, but making everything depend on it (gnome) is the wrong way™ <davexunit>Sleep_Walker: does systemd-nspawm expect you to use systemd as the init system? <cehteh>why does a init detect if its a init within a namespace? <davexunit>cehteh: I don't know. needs further research. <cehteh>if it needs so i tihnk something is broken <civodul>davexunit: dmd doesn't really care what its PID is, i think <civodul>well there's just one case where it calls getpid, but it can be ignored <civodul>(and remember that dmd can be used as a normal user, as a process manager) <cehteh>one thing which annoys me in unix is that orphaned processes get adopted at pid1, would be nicer to walk the tree up and let anyone adopt a process with some signal handler or so <cehteh>that would fit the user-manages-his-own-processes better too <davexunit>containers would make testing out new services easier. <davexunit>currently I'm waiting for a VM to spawn before each test <davexunit>civodul: oh btw, I decided to give 'spawn' a shot as an action name in 'guix deploy' <civodul>davexunit: fwiw i use 'guix system vm' to test services <civodul>it takes a bit of time to boot but usually that's acceptable <davexunit>would be nice to tighten the loop in the future, though. <civodul>yeah, it would definitely be faster with containers anyway <remi`bd>I’ve read the last exchanges on the list, writing an answer right now :) *davexunit continues to debug postgresql-service <taylanub>civodul: my mesa update patch is actually ready now; shall I push it? by the way I pushed the libva one earlier today; I hope I didn't cause excess rebuilds on core-updates when we want them done ASAP? <Sleep_Walker>davexunit: I've got guix in namespace separated environment :) <Sleep_Walker>I need to alter filesystem configuration to pass the initrd but it looks quite cool <davexunit>Sleep_Walker: would you be interested in writing up the steps needed to create that? <davexunit>Sleep_Walker: also, why does it need the initrd? <davexunit>I didn't think a container had a use for such a thing <Sleep_Walker>davexunit: and I need init because I haven't finished my LVM journey <Sleep_Walker>I was also surprised that I can have another PID 1 in separated namespace :) <davexunit>I have a bit of guile code that can fork guile into a new process with all 6 new namespaces <davexunit>now it's a matter of mounting /proc /dev and stuff <Sleep_Walker>yeah, but thing is that I'm hacking that from openSUSE where I have systemd <davexunit>so now you're just using the guix package for it? <davexunit>it's been nice to start relying less and less on the host distro <Sleep_Walker>well, I came back only for the hacking and as I'm on openSUSE conference, it makes people around less angry :b <davexunit>paroneayea: your blog post has been on the front page for a long time now :) <_`_>but they'll blame it on cgroups and argue for a radical change of them that suites them. <davexunit>still trying to wrap my head around user namespaces <davexunit>paroneayea: y'know, the more I think about stateful upgrades for things like databases, the more I think that the answer is simply a dmd service. <davexunit>mediagoblin-service could run the db upgrade script at activation time. <_`_>I don't think dmd should be an issue. I could try guixsd in an unprivileged lxc next week perhaps which would be the same effect. Then let you know what I encounter. <davexunit>I'm really excited to get guixsd containerized purely with guile code, but I need to learn more first. <davexunit>I'm going to start by ignoring cgroups and just getting it running with (hopefully) all 6 new namespaces. <_`_>I'd like to help more, but I need more free time. :> <_`_>But hopefully the random bits of information and links might help in some capacity. <davexunit>_`_: you've been leading me in the right direction <davexunit>of course, there's still networking issues to figure out in order to be able to reconfigure machines after spawning them, but a local vm couldn't support these features anyway. <davexunit>or rather, a local vm with a temporary file system and a read only store. <_`_>networking wise maybe doing something like a bridge (public or private+natted) on the host and the container use veth/tap might be a solution, or maybe macvlan/macvtap. <davexunit>_`_: yeah, the current issue there is that I need to figure out how to add support for that stuff to guile's socket bindings. <davexunit>since it's linux only(?), maybe it's not acceptable for core Guile and I need to make my own wrappers. <_`_>is wrapping iproute2 a bad idea? or do you need to implement what it does in guile? <davexunit>_`_: need to implement some of what it does in guile. <davexunit>it seems that I'll need some sort of daemon to manage the containers <davexunit>cgroups and subsystems are confusing me to some degree. <taylanub>civodul: I went forth and pushed the mesa update <_`_>davexunit: if it's id 0 on the host running/starting unprivileged containers, some daemon probably isn't needed, since it can create the cgroups. If it's an unprivileged user, then something will be needed so that they can be created by root, yeah. Seems others are starting to rely on logind to help for this. <_`_>I currently use a crappy shell script to create the cgroups before starting unprivileged containers with an unprivileged user (for lxc). <davexunit>_`_: oh, so by creating the new namespaces, I can create a brand new cgroup hierarchy within? <davexunit>_`_: and regarding the daemon issue, we already have the guix daemon, so a long term goal would be to let that spawn the containers on behalf of unprivileged users. <paroneayea>davexunit: oh, you replied in here about the joeyh thing, heh! <_`_>Ahh I see. Well I'm also off for a while. (^_^)/ <davexunit>civodul: how would you feel about defining simple record types to be used as part of a service configuration? <davexunit>postgresql has a host-based authentication file, which is in a whitespace delimited values format. I was going to create a record type to encapsulate a row that could be written to the file. <davexunit>(postgresql-service #:authorized-hosts (list (postgresql-host ...))) <civodul>davexunit: you mean using a record instead of a set of keyword arguments? <cehteh>i've installed guix 0.8.1 .. but when i now install a new package it fetches the newest available and a whole lore of dependencies, isnt there a way in guix to snapshot on a guix release <civodul>davexunit: the guix deploy config looks neat, exactly what one would want to type i think <civodul>davexunit: re NETLINK, i think this requires C code, probably in libguile itself :-/ <cehteh>civodul: networking dying here in the vm every few minutes, i really suspect that a kernel bug or configuration issue, other kernels work in kvm <cehteh>might be some issue with the guix networking stuff as well, but i get no log messages what really happens except "Network unreachable" <cehteh>restarting the network then works, sometimes after a few tries <civodul>cehteh: that's weird, my (maybe limited) testing in QEMU+KVM worked well <cehteh>looks dhcp leases time out and it then pulls down the network, by that extending the lease will not work (at least thats something i guess, but i havent figured out how to debug the shit yet) <cehteh>i am pretty much unable to do anything which needs network (whats almost anything) <cehteh>.. again my earlier question, there is no facility in guix yet to tag some releases and install from that with package -i ? <rekado->cehteh: you would not have substitutes in those cases as hydra doesn't keep all older builds. <rekado->cehteh: you could simply use guix from git and move through history this way. <cehteh>yes but thats somewhat outside guix (and i am trying to use guixsd here) <cehteh>guix package always downloads the newest package list, wich is a bit pita <rekado->even on guixsd you can use guix from git. *cehteh likes rolling releases .. but some snapshoting would be more sane, if only hydra (or whatever server) would keep a latest release, daily or so <rekado->I think it would be nice to have a stable version eventually in addition to the bleeding edge, but for now you can use git to achieve this purpose. <cehteh>how can i override that guix downloads the substitutes list every time anew? <civodul>cehteh: so you have dhcp-client-service in the config, right? <cehteh>besides .. i would like if guix would just act as frontend to git in this case, fetching the list with git along with snapshots <cehteh>civodul: i really just kept the config template as is in the manual, only changed the username for my account, the grub install disk and the hostname <cehteh>ah and the partition label is GuixSD <davexunit>civodul: yes, libguile's socket.c is the place for the changes. <davexunit>thanks to guix, it will be easy to create an environment with the hacked guile in it. ;) <civodul>cehteh: not sure what's going on with QEMU's DHCP server, but maybe you could try static networking <civodul>'guix system vm' uses QEMU's statically-assigned IPs actually <cehteh>i tried 'bridged' networking with macvtap which connects to my own DHCP as well with the same results <cehteh>well i can restart trying that again as soon the stuff i am currently try to install finished <cehteh>(why does it run gtk testsuites now?) <civodul>cehteh: we'd need to investigate what's going on in the DHCP client/server interaction <civodul>it's running GTK's test suite maybe because substitutes are no longer available. <cehteh>network is dead again .. i restart it with bridged network <cehteh>so looking at my dhcp's log's now <rekado->wee, gcj finished building. The patch I sent to the ML appears to be fine. <davexunit>civodul: awesome! what remaining work is there? <civodul>davexunit: make-socket-address and such, aka. scm_{to,from}_sockaddr <cehteh>civodul: so network just died, no dhcp request from the client is made (ifconfig still shoes it configured, route is still up, everything times out) <civodul>there are portability issues here, because i think NETLINK is pretty non-portable <cehteh>dhcp server sees nothing of course <civodul>cehteh: could you check on tty1 of GuixSD whether dhclient says anything? <cehteh>cant guile load extra modules written in C for netlink and other nonportable stuff? <cehteh>civodul: did that, nothing at all <civodul>davexunit: we'll have to check, but at worst assume that its #ifdef __linux__ <cehteh>for the guixsd it looks like the network should be still up <cehteh>ah .. there are 2 dhclient processes running <cehteh>nah after a reboot only one left, but seems that deco stop networking didnt stopped the first one properly <cehteh>civodul: for once its clear, 'deco stop networking' leaves the dhclient running, i can reproduce that now <cehteh>.. and after killing it, it becomes a zombie <cehteh>starting dhclient manually same result, but now i can kill it (becomes zombie then), start a new and network is back for some time <cehteh>this zombie apocalypse is worrying me, seems dmd doesnt collect the pids properly <cehteh>yes sure, i am only reporting, not complaining <cehteh>(while i wished i had more success, because i think guix and dmd are extremely cool things) *paroneayea needs to learn dmd <paroneayea>I'd like to understand more about what you're talking about with dmd for upgrades, davexunit! <paroneayea>davexunit: civodul: btw, I think I can still finish my "guix environment as a virtualenv+bower replacement" example for mediagoblin, focusing just on the python side of packaging <paroneayea>for the package.scm in mediagoblin, it could have jquery-evil and viewjs-evil packages that just do the "fuck it, I'm out" method of just pulling down the precompiled files <paroneayea>which, for replacing virtualenv and bower, is sufficient... it's little different to what bower is doing anyway <paroneayea>and since it's just mediagoblin/package.scm, it should be fine <paroneayea>but obviously, in the long run we'd like real answers to this <civodul>paroneayea: yeah using guix this way would still be useful, hopefully on the mediagoblin side and certainly on the guix side :-) <civodul>cehteh: so there were several dhclient? the #<eof> error message suggests that dmd failed to read dhclient's pid from its stdout <civodul>zombies shouldn't happen, indeed, since dmd catches SIGCHLD <rekado->azr3 on mips has failed even though there's no error in the build. <rekado->--> "guix build: error: failed to create GC root `/home/hydra/offload-20121227-hydra.gnu.org-3192': File exists" <rekado->could a restart of the job fix this? <civodul>i've just restared it, that should do <civodul>but yeah, it's a long-standing bug in the offload thing <rekado->unfortunately, R still fails with a segfault in the test suite. I cannot reproduce this on my machines. <civodul>i'm not satisfied with this part of the code <civodul>could it have something to do with the ISA extensions being used? <rekado->the build of nss on mipsel times out. Can the timeout be raised? <rekado->the nss build failure is responsible for the build failure of icedtea6, icedtea7, and other java stuff on mipsel. <cehteh>civodul: .. i try to investigate and fix this next days, lets see <a_e>rekado-: I just restarted nss, maybe it is a transient problem. <a_e>rekado-: Do you have an idea what the problem with numpy is? It builds without problem on my local machine. <a_e>I think it was discussed on the mailing list a while ago. <rekado->a_e: it also builds fine on my machines, so I can't say. <a_e>rekado-: I just see that the problem is that numpy-bootstrap fails its check phase. <a_e>We could start by prohibiting parallel tests. <a_e>Or maybe disable tests altogether, just for the bootstrap package. <civodul>hopefully Federico will give some feedback <a_e>civodul: Nice you merged core-updates into master. <a_e>Just today I thought it should be done, since it would be nice to have qt again. <phant0mas>patch, sed, grep, diffutils, findutils, coreutils, make, guile, pkgconfig etc build <paroneayea>I wonder how something like wordpress could be used in guix... I think a service would have to be set up <paroneayea>since wordpress doesn't really "run" until it runs through apache <civodul>paroneayea: maybe the apache httpd service would be passed a list of "servlets", as people called it in prehistory ;-) <paroneayea>civodul: I'm very interested in combined "recipes" for deployment and solving the eternal challenge of "how to describe that apache/nginx config(s) based on your setup?" <civodul>yeah that's definitely the kind of problem we want to address <civodul>i think davexunit has given it some thought already :-) <paroneayea>well luckily, davexunit and I see eye to eye on most of these things :) 