IRC channel logs

2015-04-08.log

back to list of logs

*davexunit reads https://nixos.org/nixops/
<davexunit>cool stuff!
<davexunit>I don't know if it would an issue to support the "cloud", though
<davexunit>like providing a way to deploy to Amazon EC2
<mark_weaver>davexunit: is there a standardized format for OS images to run in any IaaS provider?
<davexunit>mark_weaver: I don't think so.
<mark_weaver>personally, I'd prefer not to steer people toward Amazon
<davexunit>there are things that are compatible with EC2
<davexunit>like Eucalyptus
<davexunit>I agree about not steering people towards Amazon
<davexunit>OpenStack would be a better target
*jxself prefers to steer people to their own computers
<joehillen>what if they're already on Amazon?
<mark_weaver>jxself +1
<davexunit>I think OpenStack would be a good idea, then. people can build their own OpenStack setup.
<davexunit>and we could support deploying VMs to it.
<davexunit>in addition to say, deploying to your own physical hosts
<davexunit>or a remote box with KVM on it, even.
<davexunit>the ability to control remote guix machines from the comfort of your workstation is very appealing to me.
<jxself>Too bad we don't have a free implementation of Intel's management engine stuff.
<mark_weaver>I confess I'm still using what might be considered stone tools by modern sysadmins
<davexunit>and that's fine :)
<mark_weaver>I just use things like TRAMP and emacs running in screen, stuff like that
<davexunit>that works up to a point.
<davexunit>I'm interested in fully automated, reproducible, distributed systems.
<mark_weaver>I'm open to being educated about the benefits of the new methods
<jxself>"Almost all AMT features are available even if PC power is off, the OS is crashed, the software agent is missing, or hardware (such as a hard drive or memory) has failed."
<jxself>Sounds pretty good. If only it were free...
<davexunit>mark_weaver: a lot of the new methods have good ideas, but bad implementations.
<davexunit>which is why Guix is a light in a dark tunnel for me.
<mark_weaver>so what else is needed, beyond what I describes plus Guix?
<mark_weaver>*described
<davexunit>a tool to create/destroy/update many machines at once, automatically.
<davexunit>in which there is an abstraction for the target infrastructure (VM, physical host, etc.) for each machine
<davexunit>rather than having to ssh into and maintain individual machines, you maintain the sum.
<mark_weaver>sure
<davexunit>I maintain my own servers like you do. it works fine for a small number of machines.
<jxself>Yes. Need to be able to combine and separate and reaggregate and do all kinds of neat tricks.
<mark_weaver>bah, baby is unhappy, gotta go
<davexunit>ttyl!
<jxself>This is when you say, "Baby, this is not a good time." :)
<davexunit>hehe
<mark_weaver>jxself: you don't have experience with young children, am I right? :)
<davexunit>I'm curious how nix ops stores the state of all the machines
<mark_weaver>*much experience
<jxself>mark_weaver: Sure I do but it's always fun to imagine.
<mark_weaver>heh :)
<davexunit>jeeze, nixops is written in python
<davexunit>how many languages are these folks going to use!
<jxself>How many languages exist? :)
<davexunit>Nix, C++, Perl, Python, Bash
<davexunit>did I get them all? :P
<jxself>In the world
<joehillen>Haskell
<jxself>Bit by bit, they'll all creep in.
<davexunit>joehillen: I don't think they have any source code written in haskell, do they?
<davexunit>I haven't seen it.
<joehillen> https://github.com/zalora/upcast
<jxself>Not YET being the key word. :)
<mark_weaver>presumably the language preferences of whoever volunteered to write each piece
<davexunit>joehillen: but this isn't an official nix project
<jxself>And so, over time and given enough volunteers that slowly expand to be: Every language on the planet. :)
<davexunit>joehillen: but I see your point.
<joehillen>well neither was nixops until it was added
<davexunit>good point
<davexunit>someone also wrote something called nixos-shell
<davexunit>and that uses Go
<davexunit>ha
<joehillen>now some part of nix needs to run on the jvm and they'll be set
<davexunit>I hope this doesn't come off as too mean. I really like the Nix project.
<davexunit>I just think that the proliferation of languages used is getting quite silly.
<davexunit>you have to know a different language to work on any given piece.
<joehillen>meh, it's a mess, and it all comes from them trying to work around the limited Nix language
<davexunit>they've certainly come farther in the realization of their system than we have, and they have way more users and hackers, so they're doing something right. :)
<joehillen>scheme isn't my first or second choice, but the fact that it is a proven general purpose langauge wins some major points
<joehillen>they got an 8 year head start
<davexunit>yes, that's true. :)
*davexunit tries to figure out how nixops manages state
<davexunit>it needs to store something to disk... does it use the store?
<joehillen>wouldn't is just be the nixops config?
<davexunit>there's that, yes, but there's gotta be some generated state files for 'nixops info' to work
<mark_weaver>so, now that I have both hands free...
<joehillen>nixops is just a means to deploy /etc/nixos/configuration.nix to other systems
<davexunit>it has to know where things actually got deployed to for future reference.
<davexunit>yes.
<mark_weaver>I confess that I consider the practice of running multiple VMs on a single physical machine to be a hack to paper over problems further down in the stack, with many disadvantages
<joehillen>it would know that from the config
<mark_weaver>anyone care to debate the point? :)
<joehillen>mark_weaver: I don't understand
<davexunit>joehillen: the config doesn't says what to build, but *not* precisely where the machine ends up. what's its IP? you won't know that until after deployment in some cases.
<mark_weaver>if you want to run multiple services on a single physical machine, why not just run a single kernel with multiple users and processes running on it?
<mark_weaver>don't get me wrong, I can see why it is appealing
<davexunit>containers make that way more appealing
<davexunit>for isolation
<mark_weaver>you get to give multiple people root on their own VMs, and play around with different kernels and system-wide daemons and so forth
<joehillen>I'm a big fan of LXC
<mark_weaver>it seems to me that the isolation is not so great
<davexunit>but it's something
<mark_weaver>even without hacking out of your VM and into the hypervisor
<davexunit>joehillen: want to work on adding container support to 'guix system'? ;)
<joehillen>that's my plan ;)
<davexunit>oh really!?
<davexunit>awesome!!!
<joehillen>if only I could get a development environment going
<mark_weaver>I've heard of it being possible for unprivileged user processes running in one VM to deduce private keys being used it other VMs on the same box
<davexunit>we're here to help!
<davexunit>mark_weaver: ouch!
<joehillen>yeah, I'm evaluating guix as a possible replacement for out Puppet infrastruture on EC2, we're definitely going to need deployment and container functionality once we do
<davexunit>the utility of 'guix ops' (or whatever it will be called) doesn't rest upon the utility of VMs or containers. it could just as well handle deployment to physical hosts.
<mark_weaver>one kernel with all the relevant information can allocate the physical resources more intelligently, it seems to me.
<davexunit>joehillen: I'm very interested in how this works out for you.
<mark_weaver>I dunno, maybe I'm just old fashioned. quite possible.
<davexunit>joehillen: I want to see Guix meaningfully replace Puppet and co.
<davexunit>joehillen: is this for work or for something else?
<davexunit>mark_weaver: I think you raise valid points.
<davexunit>one of the things I like about guix is can we support any infrastructure we want.
<davexunit>is that we can*
<mark_weaver>*nod*
<mark_weaver>fwiw, I agree that we should support VMs well
<joehillen>for work, we like the idea of stateless package management and holistic config mangagement. This had a big impact on us: https://www.domenkozar.com/2014/03/11/why-puppet-chef-ansible-arent-good-enough-and-we-can-do-better/
<davexunit>joehillen: that's an awesome article
<davexunit>really great reference for explaining why Puppet and co. aren't great in the long term
<davexunit>joehillen: do stay in touch with us about how your evaluation goes. I have my sights set on challenging Vagrant, Docker, and Puppet/Chef/Ansible/Salt/etc.
<mark_weaver>GNU Hurd is an example of a system architecture that should render VMs irrelevant.
<davexunit>yes, I wish for the hurd to be a success some day.
<davexunit>and not just the butt of many jokes.
<mark_weaver>(if it got even a fraction of the hacker focus that Linux gets)
<mark_weaver>heh :)
<joehillen>so since we're on the topic of VMs. This is the problem I am currently dealing with https://gist.github.com/joehillen/2d665451c5d42cdb2c98
<davexunit>joehillen: are you using debian?
<davexunit>looks like you don't have permission to use KVM
<joehillen>arch
<davexunit>okay
<davexunit>joehillen: as a hack, I do this: sudo chmod 666 /dev/kvm
<joehillen>I've got the kvm group, and I'm in it, I also added all the guix-builders
<davexunit>that will let non-root users write to /dev/kvm
<joehillen>davexunit: that did it, thanks
<davexunit>joehillen: great!
<joehillen>I wonder what guix related user isn't in the kvm group
<davexunit>I'm not sure...
<davexunit>all I know is that I don't have this problem on GuixSD machines :)
<davexunit>a-ha, found the nixops state file
<davexunit>~/.nixops/deployments.nixops
<joehillen>ah ha!
*davexunit creates a wip-ops branch
<kete>They're still making progress with Hurd and a complete re-write called X15 (sceen.net)
<kete>Minix is also a microkernel but with a permissive license.
<civodul>Hello Guix!
<freaj>Hi civodul
<iyzsong>civodul: hi, I have some patches for atk/at-spi2-core/gtk+-2. should I push them into master or glib-rebuild?
<civodul>iyzsong: i think glib-rebuild is gone, no?
<civodul>mark_weaver pushed a bunch of gtk-related patches by wingo to core-updates
<civodul>so maybe you could do that as well
<iyzsong>thanks, I'll look into core-updates.
<freaj>civodul: not everyday I see some FDN abonnés on freenode :P
<civodul>freaj: heh, you're from FDN too?
*civodul is really unhappy about Tor access to Freenode being discontinued
<freaj>civodul: FFDN somehow yes
<taylanub>one already had to register a user for Tor access, no? why are they discontinuing it?
<civodul>"because of abuse", they say
<civodul>which i find dubious
<freaj>It's kind of sad, OFTC manages it much better than here...
<civodul>yeah
<freaj>They force people to register with a non-tor ip address and they still have issues.. I don't know what's wrong.
<taylanub>they could just close/throttle new registrations for a while. that doesn't really leave room for abuse, and at least allows current users keep using it.
<taylanub>freaj: you mean Freenode? I thought one could at least register anonymously.
<civodul>evidently that's not high-priority to them
<freaj>taylanub: of course that's a way to deal with it
<freaj>taylanub: I mean, it's like wikipedia (if they didn't change their policy since?) you need to register here without using a tor ip
<civodul>BTW, what does it take to get a "cloak"?
<freaj>civodul: join #freenode and ask for an @unaffiliated/civodul :P
<civodul>ok, thanks :-)
<Sleep_Walker>or guix/civodul :)
<wingo>mark_weaver, git clone http://wingolog.org/git/elogind.git
<wingo>someone was asking about the synergy package in guix
<wingo>it works fine with debian's
<wingo>quite convenient
<civodul>wingo: you filter-branch'd the systemd repo?
<wingo>no, i removed things from the tip
<wingo>same history tho
<wingo>so patches could be merged in theory
<wingo>it builds but i haven't tried to run it seriously
<davexunit>systemd on guix?
*davexunit is pleasantly surprised by how little code there is in https://github.com/NixOS/nixops
<wingo>just logind
<davexunit>oh, neat.
<davexunit>writing 'guix ops' seems to be within reach, after reading some of the nixops source.
<civodul>wingo: nice!
<civodul>davexunit: yes it's relatively small
<civodul>and this one is in Python, not Perl nor C++ or one of the other languages in use ;-)
<davexunit>haha yes I was commenting on that last night
<davexunit>I'm currently working on the basic <machine> data type
<zacts>hello #guix
<civodul>neat!
<civodul>hey, zacts
<zacts>hello
*zacts searches for what nixops is
<civodul>davexunit: we should put guile-ssh to good use
<davexunit>which so far just has a name, <operating-system> object, and a "target" infrastructure.
<davexunit>civodul: yes, this will be an opportunity for that.
<zacts>I don't see a definition for nixops in my web search
<davexunit>zacts: https://github.com/NixOS/nixops
<civodul>Artyom has been very responsive and helpful, re guile-ssh
<zacts>oh neat
<davexunit>it's the way to manage multiple machines across many infrastructures with Nix
<zacts>is nixops kind of like chef or something?
<davexunit>yeah
<zacts>oh neat
<davexunit>nixops has a heavy "cloud" focus, but the core tool is really indifferent, it abstracts that stuff away
<davexunit>as a prototype, I'd like to use our existing KVM/QEMU support to get 'guix ops' to manage local VMs
<davexunit>I'm also interested in remote installations to physical machines.
<davexunit>provisioning of hard disks automatically and stuff.
<davexunit>but if I just get the core tool working and the infrastructure abstraction right, then anyone could add that stuff.
<civodul>yeah VMs are a good start, and then networking can be added on top, i suppose
<civodul>we should develop a "remote guile" protocol
<davexunit>oooh
<civodul>to evaluate Guile code remotely
<davexunit>REPL server?
<zacts>how would PXE boot fit into this picture?
<civodul>davexunit: yeah kindof
<davexunit>zacts: not sure, but it could.
<civodul>REPL server over SSH
<davexunit>civodul: that would be awesome
<davexunit>a killer feature, IMO.
<zacts>that does sound pretty cool
<civodul>i've wanted that for the offload hook
<civodul>wingo: the 9-patch series that you posted is for master, or does it require the updates in core-updates?
<wingo>civodul, some of it requires core-updates
<wingo>not really sure tbh
<wingo>core-updates is the conservative choice :)
<civodul>ok :-)
<civodul>i'll do that then
<davexunit>civodul: so this protocol, would it require any daemon besides the ssh daemon? would there have to be a guile process listening or would the protocol implementation launch a guile process on the client?
<civodul>davexunit: i guess you'd have to launch guile on the other side
<civodul>so connect, run guile, and from there talk directly to the REPL
<davexunit>would it try to serialize scheme objects and send them back to see the results of a remote evaluation?
<zacts>well wouldn't not having a client guile talking to a server guile defeat the purpose of having a protocol in the first place? I mean at that point you could just use plain ssh to control the server guile
<civodul>davexunit: only "plain" objects, like the regular REPL server does
<civodul>it just needs to be able to send sexps; when the result is a struct, it cannot do anything
<davexunit>civodul: okay
<davexunit>thanks
*davexunit imagines 'guix ops repl web-server'
<zacts>(sorry, I meant that more as a question than a criticism. I see how it could have come out differently than I meant)
<davexunit>zacts: well, we would be using plain ssh, but there would be some things we do automagically that would otherwise have to be done manually
<zacts>ah ok, cool
<davexunit>that's my impression, anyway
<davexunit>I see that the system architecture isn't stored in an <operating-system> object
<davexunit>how does guix DTRT?
<civodul>sneek: later tell Guix defaults to (%current-system) when building the OS
<sneek>Got it.
<mark_weaver>civodul: you forgot to tell sneek who to tell
<Sleep_Walker>sneek: later tell davexunit Guix defaults to (%current-system) when building the OS
<sneek>Got it.
<civodul>bah, thanks mark_weaver :-)
<guix>let's clear that entry :P
<sneek>guix, you have 1 message.
<sneek>guix, civodul says: defaults to (%current-system) when building the OS
<civodul>haha :-)
<mark_weaver>taylanub: thanks! :)
<Sleep_Walker>I have some bash scripts which can't be run with /bin/sh - what is the proper way to make working with GuixSD and other distributions without rewriting hashbang?
<civodul>Sleep_Walker: you mean they require another shell?
<Sleep_Walker>yes
<taylanub>it's strictly speaking a bug in the script if it has #!/bin/sh but uses bash-specific features. I'd patch the shebang.
<civodul>there's no good solution other than changing the shebang
<Sleep_Walker>that affects everything with shebang be it perl, python, bash...
<civodul>yeah this can be annoying
<civodul>someone in NixOS ended up adding /usr/bin/env to the default setup
<civodul>(a symlink, as for /bin/sh)
<Sleep_Walker>yes, that I was going to propose
<civodul>i don't really like it, but i sympathize with the frustration
<Sleep_Walker>:D
<zacts>how would I make a package for guix, that doesn't include a Makefile, but uses a Build.sh script
<civodul>so /usr/bin/env might be ok, but we won't add /bin/bash, /usr/bin/python, etc. etc.
<Sleep_Walker>/usr/bin/env should be sufficient
<civodul>zacts: see gnu-make-boot0 in (gnu packages commencement)
<civodul>it uses a build.sh script
<mark_weaver>on non-intel platforms, we also need /usr/bin/file, otherwise lots of configure scripts are broken. where does it end?
<mark_weaver>it is a slippery slope
<civodul>agreed
<civodul>maybe we could use binfmt_misc to redirect shebangs automatically?
<civodul>not sure if it's possible
<mark_weaver>I don't see how that would be preferable to adding a bunch of symlinks
<civodul>it could use stuff in the user's profile
<civodul>so it would be more "faithful"
<Sleep_Walker>or bind mount /run/current-system/profile/bin to /bin ;b
<Sleep_Walker>or create /bin symlink
<civodul>that would be Bad
<Sleep_Walker>>:D
<mark_weaver>one side effect of these compatibility measures, however they are implemented, would cause lots of bugs to slip through the cracks -- bugs where these compatibility symlinks were used in our software.
<mark_weaver>and that in turn would cause our rollback features to no longer be reliable
<mark_weaver>we already have such bugs
<mark_weaver>but this would create a lot more of them
<zacts>ok cool
<mark_weaver>to the extent that the proper operation of software depends on things that mutable, it may be broken by those mutable things pointing somewhere different
*zacts is looking at importing the mksh shell
<mark_weaver>we cannot avoid this problem completely in all cases, but we can minimize it
<rekado_>I have this patch on the ML that has been sitting there unreviewed for a while. I would appreciate a comment on whether this is okay or needs more work: http://lists.gnu.org/archive/html/guix-devel/2015-03/msg00537.html
<mark_weaver>rekado_: all of your added phases return unspecified values. they should return booleans indicating success. so they all need #t added
<mark_weaver>rekado_: the close parens after the configure-flags should not be on their own line
<rekado_>mark_weaver: got it.
<Sleep_Walker>so, proper solution is to create package from my bash scripts and let guix do the job - I started sharing my /home partition with other distributions so I went to this
<Sleep_Walker>the other way would be to create local wrapper which will detect running distribution
<mark_weaver>rekado_: the version number "4.0" in the source URI should be replaced with a call to 'version-major+minor'
<Sleep_Walker>or the ugliest and easiest - I'll add /bin/bash as symlink
<rekado_>mark_weaver: should I leave the commented configure flags for other available modules that currently cannot be built? Or should I just remove them and let future contributors add them as needed?
<Sleep_Walker>(I guess that will be my approach)
<mark_weaver>rekado_: commenting them out sounds good, but maybe add a comment explaining that they are desirable but they cannot yet be built, maybe with some brief explanation of what goes wrong
<mark_weaver>Sleep_Walker: how about putting /run/current-system/profile/bin/bash in the shebangs?
<Sleep_Walker>mark_weaver: it won't work on any other distribution but guix
<mark_weaver>but yeah, creating guix packages would be the best way
<rekado_>mark_weaver: thanks for taking a look. I've made all changes you suggested.
<mark_weaver>one could make such package creation a lot simpler by creating a procedure that creates such packages, given the script itself.
<Sleep_Walker>yes, definitely
<mark_weaver>rekado_: np! I think it's okay to push with those changes
<mark_weaver>sorry for the delay
<mark_weaver>civodul: regarding the armhf ld.so bug: how early can I interrupt the glibc build before running that command?
<mark_weaver>what are the constraints on when I run that command?
<mark_weaver>civodul: nevermind, I got it
<mark_weaver>civodul: sent a reply to your last request on the armhf ld.so bug.
<civodul>mark_weaver: sorry, i was sunbathing (first sunny days of spring here :-))
<civodul>mark_weaver: the "compatibility measures" i had in mind are only for the user environment, not for the build environment
<civodul>perhaps there was a misunderstanding here
<civodul>like GuixSD has /bin/sh, but the build environment doesn't
<mark_weaver>civodul: I understand, and I think my comments still stand
<civodul>ah ok
<civodul>at any rate, i must say that after 7 years of NixOS + GuixSD, i've never felt handicapped by having only /bin/sh
<civodul>occasionally it's a minor annoyance, but not more than that in my experience
<jxself>win 17
<jxself>Argh
<mark_weaver>civodul: on armhf, I didn't see any -rpath option when linking ld.so.new, and yet there's an RUNPATH in the resulting binary
<mark_weaver>I sent email with the details
<civodul> /me looks
<mark_weaver>maybe ld-wrapper is adding it?
*mark_weaver looks deeper in the strace
<mark_weaver>ah yes, there it is
<mark_weaver>sending another email...
<civodul>mark_weaver: right!
<civodul>ok
<civodul>which is weird still
<mark_weaver>sent
<mark_weaver>civodul: do you want me to restart the entire glibc build with GUIX_LD_WRAPPER_DEBUG=yes, or just that one manual ld.so.new link command?
<mark_weaver>when I run just that one command manually with GUIX_LD_WRAPPER_DEBUG=yes I don't see any output
<mark_weaver>maybe it's being redirected?
<mark_weaver>civodul: even when I run it in strace with that env var set, I still don't see the output anywhere. maybe it's not getting flushed before the exec? in any case, it seems that won't give us any additional information that's not in the strace.
<mark_weaver>I sent an email containing the "real" ld command from the strace.
<civodul>mark_weaver: GUIX_LD_WRAPPER_DEBUG=yes leads to a single line being printed on stderr
<civodul>starting with "ld-wrapper"
<civodul>i'm actually seeing it adding the -rpath
<civodul>now comparing with x86-64
<mark_weaver>alas, I have to go very soon
<mark_weaver>for several hours
<mark_weaver>it would be great to get this fix in core-updates though.
<mark_weaver>for now, I'm starting my novena building the current core-updates
<mark_weaver>(in a different git checkout directory, so we can continue debugging this ld-wrapper issue as well)
<civodul>mark_weaver: i'm getting there
<civodul>sorry for being unresponsive, i keep being interrupted
<civodul>at home, i mean
<paroneayea>btw, it's kinda guix related, at least in that it has a shout-out to Guix and hopefully-to-happen GuixOps http://lists.mediagoblin.org/pipermail/userops/2015-April/000042.html
<civodul>nice!
<civodul>i think going the imperative way is always going to be a pain anyway
<civodul>it's already so easy to deploy NixOS, say, on a whole network
<civodul>nothing can beat that approach, i think
<bavier>paroneayea: thanks for sharing.
<bavier>I've been meaning to subscribe to that list
<davexunit>civodul: have you deployed NixOS on a whole network before?
<civodul>davexunit: at work i used to run a Hydra build farm, and at the end i used "Charon" (former name of NixOps) to deploy to build slaves
<civodul>there were ~10 machines
<davexunit>civodul: cool
<civodul>then an Inria-wide CI infrastructure was deployed, based on Jenkins + CloudStack
<civodul>a different beast ;-)
<civodul>i realize my earlier comment sounded negative, but i think it's cool that initiatives like userops or Freedombox are striving
<civodul>it should be easier for "normal people" to deploy a box that runs network services
<civodul>sneek: botsnack
<sneek>:)
<civodul>who would be willing to co-mentor Rohan on the DHCP client project, should the proposal be accepted?
<civodul>it would be nice to have two mentors
***wenderen is now known as rohanp
***rohanp is now known as wenderen
<civodul>grrr
<civodul>i still have this Emacs bug where occasionally hippie-expand causes it to segfault
<freaj>civodul: pfff
<freaj>guix doesn't have systemd but it has emacs, same thing :P
<civodul>heh
<bavier>some progress on the perl Set-Object relicensing: https://github.com/rurban/Set-Object/pull/4
<civodul>sounds like it's pretty much solved, no?
<bavier>civodul: just waiting on one more copyright holder confirmation
<civodul>ah ok
<davexunit>civodul: yay stateless user accounts!
<davexunit>thanks!
<davexunit>fyi, mark_weaver and nully are building a GuixSD machine at the FSF right now
<davexunit>exciting times!
<jxself>Oh noes!