IRC channel logs
2015-04-28.log
back to list of logs
*davexunit sends braindump about 'guix ops' to ML *davexunit furthers his understanding of monads <davexunit>woo, 'guix ops build' works at a basic level <davexunit>I feel like I'm finally starting to understand the monad API <davexunit>you'll notice that the platform for each machine is #f, because there are no platforms implemented yet. <davexunit>but 'guix ops build' successfully builds each machine in the deployment now. <iyzsong>cool! what the state look like? does it decide where to deploy the <machine>? I never use nixops, have no idea about it :) <davexunit>iyzsong: I haven't written that part yet, but the state file will include details about what happened during deployment. like, what IP did the VM get dynamically assigned or something. <davexunit>iyzsong: now I'm back to learning about containers, I have a directory prepared with a bind mounted /gnu/store and system that I can chroot into successfully <davexunit>now to how to do the whole container song and dance... <iyzsong>oh! I think the next step is use 'guix ops' to deploy the container.. <davexunit>I need to get the container working manually first <davexunit>I think we'll need our own container creation code written in guile <iyzsong>yeah, will it support non-guixsd system? <davexunit>probably not since it would be integrated into guix <davexunit>I mean, someone could use it as a library to make containers for whatever <iyzsong>ah, that's really cool, packer (which can build/provision images for docker, vbox, etc.) written in scheme? <davexunit>it wouldn't build images, because 'guix system' (and the associated scheme modules) build that stuff <davexunit>it would just instantiate a container given a path to the root directory <iyzsong>Yes, we can just use what we alrealy have, and get pre-build containers (just same as packages) from our hydra. <davexunit>iyzsong: I'm unsure if containers need anything extra. we already build system derivations <davexunit>it's just a matter of bind mounting it + the store <davexunit>took a look at liblxc, seems to be too high level for our needs <davexunit>it's a small program, so it's source would be very useful for implementing a solution in guile <rekado_>bleh, I'm again getting only corrupted texlive packages from hydra... <rekado_>how do you set up geiser mode for guix? I have the git repo in ~/code/guix --- does this path need to be in GUILE_LOAD_PATH to be able to use geiser's "M-." feature? <rekado_>I already did this but M-. still won't jump to the definition of a value at point. <rekado_>the Ubuntu installer is vastly inferiour to the Fedora installer. The Fedora installer asks me to confirm any partition changes. The Ubuntu installer just goes ahead and does its thing. <rekado_>I just wiped a colleagues hard disk in my attempt to make Ubuntu do the right thing :-O <rekado_>if GuixSD ever gets an installer it must show the steps it's going to perform, and ask for confirmation. <rekado_>(of course the colleague doesn't have a recent backup, so now my mood depends on the degree of success of this recovery session.) *davexunit reads about cgroups <davexunit>it's nice to break down some of the mysticism behind containers <davexunit>paroneayea: thanks. it doesn't really do anything yet, but I have the UI boilerplate out of the way and some data types defined. <davexunit>we don't like you question askin' types 'round here. <paroneayea>I'm curious about how much of a "central" model it has. eg, puppet requires a central server, but propellor has each machine build independently and a central server just kicks off each machine <davexunit>paroneayea: well, you'd still a central machine that you managed the deployment from, though you could just ship the state files to another computer and control the deployment from there. <davexunit>it's not like how Puppet has a centralized puppet master daemon <paroneayea>I also saw something about building the derivations on your local machine for the remote machine, or something? or am I confused? <davexunit>when I have a real working deployment backend, 'guix ops' will copy the closure of the system to a remote machine <davexunit>one of the benefits of the functional approach. we can easily figure out all the store items that need to be copied to have a working OS. <paroneayea>davexunit: also, for the "state" stuff, one worry I have <paroneayea>is that it might require that deployments happen from only one machine <paroneayea>but say I'm doing mediagoblin stuff, I'd like simonft and I to both deploy mediagoblin.org <paroneayea>I think that's one advantage to the propellor approach of "build on server", multiple machines could do the deploy <paroneayea>also, could you in theory "reconstruct" the state by logging into each machine and verifying what the state is? <paroneayea>I think one thing I like about propellor is that there's no "watching" states of each machine <davexunit>the state is basically "where did this system *actually* get deployed" <paroneayea>my understanding is that propellor --spin actually just logs into each machine, compiles the deployment for that machine, and runs it <davexunit>propellor must do *something* to be able to reconfigure the sme machines <paroneayea>davexunit: the haskell source, with the specification of "this machine should be at this state" is what's pushed around <davexunit>you could check the guix state file into a git repo. <davexunit>if you don't have state, you can't reconfigure. only provision new resources. <paroneayea>reconfiguration in case of propellor is a matter of changing what properties that machine is supposed to have, it's fairly declarative, in my understanding <paroneayea>so the machine pulls down latest git (or gets that pushed to it), compiles and runs according to its own machine id <davexunit>but you have to know where those remote machines are to reconfigure <davexunit>if you deployed to EC2, you need to know the IPs <davexunit>paroneayea: those servers, sans one I think, have static IPs. <paroneayea>well, how do you know in guixops where a machine is? <davexunit>you ask some server to provision resources for you, and it hands you back the details. <davexunit>'guix ops' will store it in a local state file. <paroneayea>so I guess in propellor you would end up either having to have a special haskell routine that did and filled in those values or something <paroneayea>davexunit: okay, anyway, thanks for that clarification <davexunit>I'm amazed that he's been able to describe everything on his systems with propellor <davexunit>he builds on top of Debian and Docker, so we have to do more work <paroneayea>I contributed the docker ascii art at joeyh's request ;) <taylanub>do we have a way to amend e.g. #:configure-flags when inheriting from a package? <taylanub>i.e. leaving the rest of `arguments' alone as well as leaving existing entries in #:configure-flags alone <taylanub>maybe I should just move the inheriting package to right aside the one it inherits from, so one won't miss fixing the inheriting one when adding configure flags to the one inherited from... <davexunit>paroneayea: now that I know a bit about how cgroups work, I'm working wtf all the development time for Docker is spent on. <davexunit>when I get the time, I want to hack up a little Guile library that can manipulate cgroups <davexunit>has it's own special config format and stuff <davexunit>the kernel does all the work, the user just has to create the right files and bind mount the right directories <bavier>taylanub: there's substitute-keyword-arguments <paroneayea>"As an example of indirect usage, systemd assumes exclusive access to the cgroups facility" <paroneayea>I wonder if that's true or put up by somebody mad at systemd, or both <paroneayea>In the short-term future writing directly to the control group tree from applications should still be OK, as long as the Pax Control Groups document is followed. In the medium-term future it will still be supported to alter/read individual attributes of cgroups directly, but no longer to create/delete cgroups without using the systemd API. In the longer-term future altering/reading attributes will also be unavailable to userspace <paroneayea>applications, unless done via systemd's APIs (either D-Bus based IPC APIs or shared library APIs for passive operations). <paroneayea>does that mean systemd is saying "no" to docker also? <davexunit>in fact, I'd rather use systemd-nspawn instead of docker <paroneayea>but how will pro-docker people feel about that :) *paroneayea stocks up on popcorn <davexunit>I don't really see how doing that is useful. <taylanub>what is the reason? systemd needs to do book-keeping of cgroups or so? <davexunit>I wonder if the reason for it is because of how cgroups work. you are limited by where you can attach "subsystems" (network, cpu, memory, etc.) <paroneayea>it says something about want to propagate properties <taylanub>and why couldn't applications just do what systemd internally does? what stops applications from speaking to the kernel? <davexunit>"Any single subsystem (such as cpu) can be attached to at most one hierarchy. " <davexunit>if someone made their own cgroup hierarchy and attached subsystems, systemd could no longer use them. <davexunit>I guess a lot of the effort with this container stuff is creating a daemon to manage cgroups <davexunit>I will try to write a little guile script to create the cgroups, chroot, and launch a shell. <davexunit>grrr I hate docker, but they have 1 click installs now. <davexunit>just sucks to see so much energy put into a flawed system. <paroneayea>cehteh: davexunit: they have a lot of money, and marketing :) <paroneayea>but then, so did mongodb, and I think jsonb in postgresql killed that from a technical perspective <_`_>but how docker does that isn't a solution <_`_>unprivileged users can create containers, but the containers aren't unprivileged <_`_>it's not just cgroups it's also userns <_`_>with docker, unprivileged users can create containers, but the containers aren't unprivileged, and this is bad* <davexunit>what do you mean by "the containers aren't unprivileged"? <_`_>but docker is just trying to be an interface after all regardless of how it does it. <_`_>I mean that afaik, users in the containers share the same UIDs/GIDs on the host. <davexunit>and they should have their own users and groups. <_`_>no they should have their own mappings to a range of uids/gids <_`_>that should elaborate what I mean better <davexunit>so there would be a mapping to a range of uids/gids on the host system? <_`_>err my mistake, by “they” I meant the user on the host. it's not enough that the container's have their own users/groups, the uids/gids shouldn't match the uids/gids on the host, relative to the host. So if I ran an unprivileged container even as root, if they “hypothetically” break out the process would be uid/gid 100000 or so. <davexunit>so if they were to escape the container, the uid/gid would be invalid <davexunit>_`_: I'll look into this more sometime, going to focus on basic cgroup stuff for now. <rekado->if only we had as many people working on the Hurd ... I really want to be able to use all of these nice Hurd-native features in a stable, usable system. <_`_>davexunit: With recent shadow a range of IDs can be assigned to users as resources. When a user namespace is created a range of IDs can be set up as mapping. eg: add 100,000 to every UID in the container to get the “real UID”. In the linux kernel there are files names /proc/$PID/uid_map and gid_map which contain the mapping table for a process/container. <davexunit>would you like to help bring containers to guix? :) <_`_>davexunit: then newgidmap(1)/newuidmap(1) read the /etc files to see local policy/ownership, validates the request, and writes to the /proc files to set the ID mappings. <_`_>davexunit: that's how lxc does it afaik. and yeah I'd be interested, but I should probably familiarize myself with guix first, and time constraints preventing me a bit. <_`_>if I free up time, I'll look through what's currently done in guix, and see how it could be added in, then just submit any proposals I have. <davexunit>for starters, I'm going to build a standalone guile library that wraps up cgroups <rekado->I wonder how mandatory access control would work in GuixSD. I'd like to be able to restrict what applications can do (especially the browser). *rekado- looks at smack now. <rekado->apparmour rules work on paths, so the rules would break whenever a package is updated. <rekado->selinux is just too complicated to begin with. <_`_>davexunit: I initially intend to use guix as a local unprivileged user's package manager, on non linux kernel platforms, where the package management is less the satisfactory (e.g. FreeBSD). nix/pkgsrc were unsuitable for these use cases. <_`_>s/less the/less than/ *paroneayea almost has another game packaged during boring back and forths on a w3c call <paroneayea>main.c:41:21: fatal error: SDL_net.h: No such file or directory <paroneayea>but there's no ./configure, so I wonder if that's why the make is failing <davexunit>paroneayea: check the -I flags passed to gcc <davexunit>look for -I/gnu/store/...-sdl-net/include/SDL or whatever <paroneayea>gcc -Wall -O2 -ffast-math -funroll-loops -Dstricmp=strcasecmp -Dstrnicmp=strncasecmp -DUSE_SDL -DNDEBUG -I. `sdl-config --cflags` -DUSE_NET -DZLIB_SUPPORT -DBZLIB_SUPPORT -c -o filter.o filter.c <davexunit>_`_: cool. I wonder if any extra work is necessary for it work on BSD <paroneayea>main.c:41:21: fatal error: SDL_net.h: No such file or directory <davexunit>paroneayea: try replacing all sdl inputs with the sdl-union package <davexunit>it's a private variable, so you need to use (@@ (gnu packages sdl) sdl-union) to access it <paroneayea>davexunit: so I'd modify the Makefile and siwtch that out? <paroneayea>guess I also need to set the PREFIX so it knows where to install for "make install" <paroneayea>I keep picking packages that I think will be easy ... ;) <davexunit>no configure script is a good sign that it won't be easy *paroneayea commits to a branch and leaves jumpnbump for later <davexunit>paroneayea: you could always dump it on the ML if anyone wants to finish it up ;) <paroneayea>davexunit: yeah, I now have two incomplete games I could dump to the ML ;p <paroneayea>orbital-eunuchs-sniper has a somewhat alarming joke when compiling <paroneayea>I also tried packaging blackshades today and found out it was nonfree <paroneayea>"source available for educational purposes only" <paroneayea>submitted the incomplete game patches to the list <paroneayea>maybe someone else will feel like tying them together. <davexunit>it's a well written C program that implements cgroup and user namespace management, among other things. <phant0mas>hey mark_weaver could you have a look at the patches I sent on the ml? If you are okay with the changes I will push them to wip-hurd <taylanub>phew, messing with mesa and libva since freaking hours, finally got everything right I think <paroneayea>I thought I remembered something of that sort, but not sure where <paroneayea>when he's around again I'd love to get his feedback on the WEB_ASSET_PATH stuff. <Tsyesika>does the guixSD image you download have an X server installed? <cehteh>the image not, but running the system init should do so <cehteh>.. well and if you try, i failed to install it on a kvm <Tsyesika>i can't install it unfortunately, i just figured if i could start it from the usb then i could see if the bug i have still exists <cehteh>you could install X on it (actually in ram or on some backing store with the cow-store stuff) and start that up <Tsyesika>i can't, i only have two usb ports and since my laptop internal keyboard doesn't work i need 1 usb for my keyboard, 1 for the usb and then network would be the another but that's 3 ports i need <Tsyesika>bavier: yeah i'm considering buying one when i next go out <cehteh>what kernel, default linux-libre kernel? <Tsyesika>cehteh: macbook has a built in broadcom wireless card which needs a blob which guix doesn't ship with obviously <cehteh>as i saied some days ago i had problems with usb keyboards not recognized in the initrd with some more recent kernel <cehteh>the xhci_pci module was the culprit <Tsyesika>i checked apple hadn't got some strange propriatory blob to get the keyboard to work but it works just fine in trisquel so not that <cehteh>well as it turns out, guixsd doesnt work at all for me because of the kvm problem :/ <cehteh>on that side .. i am thinking since some time about providing a rather complete userland as initrd than crapped down unuseable stuff <cehteh>todays computers with SSD's and many GB of ram could load an initrd with few hundreds mb in size in a blink of an eye and the recovery and debugging options are so much more useful <Tsyesika>i tried booting guixSD on an SD card but it didn't like that <cehteh>here it installs fine but the initrd panics the kernel <cehteh>and having no gsd system running as all means i cant even get a look and feel how i would enter that and debug the suff <davexunit>cehteh: the initrd is a guile program, so your debugging options are *much* better than they would otherwise be <cehteh>not that i know guile very well .. but usual or debian way is somewhat pain <davexunit>cehteh: I think you might find it easier to experiment with guix and track down this issue if you use guix on top of your host distro <rekado->gah, mandatory access control is messy... I'd rather not work on it. Time's better spent working on the Hurd. <cehteh>davexunit: what host distro would you recommend? <cehteh>not when you play with the system stuff, grub, initrd and booting into guixsd <davexunit>you can use your host distro + guix to build vm images that you can run with kvm <cehteh>i am not that much used to guix that i can do that <davexunit>cehteh: see gnu/build/linux-initrd.scm for the initrd implementation <cehteh>well i am going to install a basic debian, build guix from source and then lets see to turn it into a guix only system <davexunit>cehteh: you won't be able to turn it into a guix only system like that. <Tsyesika>i wonder if i could get a basic installation of guix without network access and then boot into that with network and just do a reconfigure when i've set it up how i want it <Tsyesika>yeah without the internal keyboard working i need USB ports for: network, external keyboard and usb stick for guix <davexunit>perhaps there's just a kernel module that needs to be modprobed <davexunit>Tsyesika: so you get network access via a USB device? <davexunit>so you can have a keyboard or the network dongle connected, but not both? got it. <Tsyesika>no, i can't right? the docs suggest you need network <davexunit>Tsyesika: have you tried enabling any kernel modules? <Tsyesika>i haven't, i considered doing an lsmod on my current install and try and find one which might be the right one but i haven't yet <Tsyesika>davexunit: i'm wondering if it is a kernel module, i suspect there is only one for USB keyboards? maybe i'm wrong <cehteh>davexunit: what do you think about the fat-userland-initrd idea? <Tsyesika>the internal keyboard is a USB keyboard it shows up in lsusb <cehteh>some laptops use usb for their buildin keyboard <davexunit>cehteh: it should be easy to try that out. our maintainer made a boot-to-emacs image that launched emacs from the initrd <Tsyesika>i think internal usb is the decato way to do keyboards and mice in laptops <cehteh>haha i always wanted to make an EmacsOS .. where init=/bin/emacs .. to be released as any arbitary 1. April <davexunit>cehteh: it's *really* easy to make that with guix <davexunit>you can write a guile script that does just that and turn it into an initrd <cehteh>i know .. the hard part is to make it useful <davexunit>ludo does that in one of the guix talks on our home page <cehteh>aka writing init like stuff in elisp <cehteh>emacs would be nice if it had a jit compiler for the lisp engine :D <davexunit>to answer your question about adding kernel modules to the initrd, see 'base-initrd' in gnu/system/linux-initrd.scm <davexunit>cehteh: well if guile-emacs is successful, that would be a future possibility. <cehteh>ok hold on step by step . first i need to have a running system <cehteh>btw is there a facility to upgrade some library and all programs depend on it? walking the dependency graph backwards <davexunit>what do you mean by "upgrade"? there's many ways to interpret this question. *cehteh somehow has the feeling over time there will become a lot clutter on a guix system without such, one updates one program or a library, but a lot other programs stay to use the old version <davexunit>you can upgrade an entire profile at once with 'guix package -u' <cehteh>but i havent seen any help in only upgrading programs to just a new library version they use <davexunit>to do that you would update the package recipe <cehteh>for example some heart bleeds again, but i want my system mostly frozen and not upgrade everything, only install a new openssl and then find any package in the system using the old ssl and alter that package to use the new openssl version <davexunit>if foo depended on libbar, and you upgraded the libbar recipe, running 'guix pckage -i foo' would build libbar and foo <davexunit>cehteh: you would just upgrade the openssl recipe and upgrade your profiles <cehteh>yes .. is there any automatic help for this task? <cehteh>really? i have to check that out <davexunit>I don't think we're understanding each other. <davexunit>changing openssl, for instance, will trigger rebuilds of everything that uses it. <bavier>cehteh: guix also has support for "grafting", which I think is maybe what you're asking about <davexunit>if you upgrade openssl, and run 'guix package -u', everything in your profile that depends on openssl will be upgraded <cehteh>so if fooapp depends on openssl the installed version links exactly to that ssl version and none else it was configured with? <cehteh>if i now install a new ssl, then the fooapp still refers to the old ssl <cehteh>thats rewriting its package description to use the new ssl <davexunit>no, you only change the package object bound to 'openssl' in the distro <davexunit>this has the effect of changing the unique hashes for all package objects that are defined in terms of that variable. <davexunit>you likely won't have openssl installed in your profile, but you will have applications that depend on it. running 'guix package -u' will upgrade everything to latest versions, as defined by the package objects in the distro. <cehteh>well i may want to keep fooapp to the current (possibly older) version .. but using the new openssl <cehteh>you know people are sometimes reclutant to update stuff, never change a running system <bavier>cehteh: when upgrading, you can provide a package version <davexunit>but you said they wanted to keep the old version. <bavier>assuming there are two versions supported by guix <davexunit>an 'upgrade' doesn't necessarily mean upgrading to a new release of the software, but rather the latest build. <davexunit>in this case, a version linked against the fixed openssl <bavier>so, you'd upgrade to the same version, but the upgrade will bring with it the updated ssl dependency