IRC channel logs


back to list of logs

<rekado_>must be due to the new guile-config; I’ll check it. Thanks!
<rekado_>the move from bioconductor tarballs to git is complicated by the fact that the git repos don’t contain version tags
<rekado_>there are only *bioconductor release* branches
<rekado_>not sure how to abstract this away
<rekado_>no point in baking the release branch name into the bioc-git-origin thing when we need an explicit commit anyway
<rekado_>the updater could use the release branch to check for new commits, but aside from that we don’t have any use for the branch name.
<civodul>hmm i thought bioconductor was on svn, but i guess i'm just confusing things
<rekado_>it used to be on SVN a few years ago
<rekado_>then they migrated to git.
<rekado_>PurpleSym: I need to rename inputs in jasp (due to renaming in Guix), but it doesn’t build: Project ERROR: Unknown module(s) in QT: webchannel svg qml quick quickwidgets webengine
<PurpleSym>rekado_: Did you use the qt5 variants?
<PurpleSym>Hm, strange. It should just work. I haven’t looked at Maxime’s QT changes yet.
<PurpleSym>Do you have a patch to change input names?
<rekado_>I pushed it already
<rekado_>(because without it “guix pull” is broken)
<rekado_>I’m a little confused as to why the variable was just renamed like that without also adding a deprecation
<rekado_>we have all these nice features to minimize breakage, yet for some reason people don’t use them :-/
<PurpleSym>Ah, I see. Thanks for unbreaking guix-science. Not much time on my hands right now to work on Guix.
<PurpleSym>The unchanged qt variables refer to the qt@6 variants now, as far as I see.
<PurpleSym>That’s probably why the build system does not recognize them.
<rekado_>but I changed them to *-5
<rekado_>oh, the *other* qt variables…?
<PurpleSym>Yup, the other ones 😇
***civodul` is now known as civodul
<rekado_>I think we’re not realizing the full potential of Guix as it comes to containers.
<rekado_>state of the art is to bundle things up, upload to a registry, transfer the archive from the registry to the server, mount the file system it contains in a container, and then launch the application
<rekado_>Guix enables ephemeral file systems — we don’t even need to bundle things up first
<rekado_>but this also means that we don’t have a convenient unit of describing containers for consumption by existing tools and services
<rekado_>if I wanted to use any sort of commercial platform that lets me launch containers I’m almost inevitably restricted to using Docker: registries and images.
<rekado_>e.g. Amazon’s ECS (container service) and ECR (container registry), but also any kubernetes platform.
<rekado_>these things exist primarily because it’s no longer acceptable to start a new server just because you want to run a new service, nor is it okay to manually add a new service to an existing server.
<rekado_>why not “acceptable”? Because there are situations where you want to automate this and treat all your servers (whether VMs or physical) as an amorphous “computing substate”.
<rekado_>this goes beyond cattle vs pets. This is cattle vs ground beef.
<rekado_>this sounds a lot like HPC, actually
<rekado_>nodes and schedulers and tasks that are launched on any free node.
<rekado_>but also: tasks that are long lived and can be migrated to larger nodes.
<rekado_>so… what can we do with Guix in the realm of imageless container orchestration?
<efraim>I'm not entirely sure if you're looking at a "replacement" for dockerhub or something similar, currently I have a (system) user running shepherd who has some shell scripts which launch containers or environments
<efraim>ok, i'm reading it again and seeing you're looking for orchastration. would it need to run on top of Guix System?
<rekado_>it would not neet to run on top of Guix System.
<rekado_>it would however need to be able to make use of existing services
<drakonis>is this meant for giving guix a foothold into the cloud?
<drakonis>i'm currently working on a distributed daemon
<drakonis>but is this for providing containers generated by guix or something that runs a bit more higher up in the chain?
<rekado_>yes, my mind is in the clouds.
<rekado_>if you want to use cloud things today you can’t avoid Docker, and there’s a mismatch between what Guix does and what these services/platforms offer.
<drakonis>guix as it exists today is very machine local
<rekado_>so you could of course export docker images from Guix, but that’s such a degenerate use of Guix.
<rekado_>Guix has some real advantages in the container / cloud world, but we can’t realize them when we must “compile” things to independent compressed archives.
<drakonis>it isn't a driver in this environment
<drakonis>its not like it can do things like "i want to spin up a container environment with these rules"
<drakonis>at any moment
<drakonis>its a static thing
<rekado_>it doesn’t need to be static
<drakonis>that's what i meant
<drakonis>the current model operates under the assumption that you'll have input files and output files
<rekado_>I’m using Guix to build environments, transfer them (or their diff) to a new EC2 VM, and then launch services.
<drakonis>that's pretty much what i just said
<drakonis>you have your inputs and outputs, but they're always files
<drakonis>it doesn't involve repeating actions
<rekado_>I’d like to cut out the annoying intermediary steps and deploy not VMs but containers — just without registries or container images
<drakonis>that would also help
<rekado_>I think we might be miscommunicating
<drakonis>i see where you're going now
<rekado_>good good :)
<rekado_>(doesn’t help that all these cloud terms are nebulous)
<efraim>Like a combination of 'guix deploy' and 'guix system container' ?
<efraim>but with orchastration
<drakonis>that would require communicating with the target
<rekado_>efraim: yes, maybe.
<rekado_>drakonis: indeed.
<drakonis>and the target would have to be able to manage this
<drakonis>that's within the scope of what i'm doing right now
<drakonis>writing a distributed daemon with workers and managers
<drakonis>this requires changing the unit of work a bit
<rekado_>people who user kubernetes or docker *need* these docker images, because they can’t depend on reproducible, declarative package management.
<rekado_>using Guix under than assumption is self-defeating
<drakonis>indeed it is
<drakonis>but this definitely falls within the scope of what i'm trying to build right now
<drakonis>kubernetes is incredibly complicated though
<rekado_>so I think we need to do better and identify how we can leverage™ our advantages — without having to implement all these platforms and services
<drakonis>what i'm doing might lead into a general rewrite of guix down the road
<drakonis>the whole thing lol
<rekado_>yes, I’ve been looking around for simpler orchestration engines, but they all look like spaghetti to me.
<drakonis>including shepherd
<rekado_>as long as there’s a path to migrate incrementally I don’t see anything wrong with an effective rewrite.
<drakonis>mostly to accommodate for the idea that everything is a task that can be run
<drakonis>path forward is a yes.
<rekado_>this whole topic keeps poking me, because I feel that we keep on brushing against it
<rekado_>with GWL, “guix deploy”, “guix pack”, “guix shell -C”, but also with PiGx and that whole “run it in the cloud, somehow” thing.
<drakonis>i'm almost considering throwing in a shell rewrite
<drakonis>(actually, i already have, there's a lot of space for interesting trickery and subterfuge with it)
<drakonis>(not an immediate concern yet)
<drakonis>guix deploy wants a better way to communicate with the target
<drakonis>guix pack needs a nicer way to store data
<drakonis>guix shell -C wants a way to lie that doesnt need shell scripts
<drakonis>or C
<drakonis>the ongoing problem with guix right now is that it relies on a lot of shell trickery that could be avoided
<drakonis>builds right now run in the filesystem with a bunch of extra users
<drakonis>ah, i got off track here
<drakonis>rekado_: how is GWL implemented?
<drakonis>it does not go through the daemon in any form, right?
<rekado_>GWL uses the daemon for environments, but the processes and their inputs and outputs are managed outside of the daemon.
<drakonis>i see
<rekado_>there’s a lot of parallel evolution
<drakonis>the build coordinator and cuirass as well
<drakonis>also i can't help but think that ditching the current store model would help with some other pains
<drakonis>the only thing that's stored on sqlite right now is the node graph
<rekado_>with GWL there are two reasons for keeping out of the daemon: 1) we don’t want to store huge input or output files in /gnu, which is both precious and inconvenient to work with; 2) we want to run workflows without *needing* an active Guix installation on the executing nodes. We just need (declared parts of) /gnu as a read-only file system slice.
<drakonis>those are good reasons.
<drakonis>i also want to get rid of grafts as they are today
<drakonis>its too fragile and requires more thinking than it needs to
<drakonis>its only useful if you have a build farm
<drakonis>for grafting packages that are way too high in the tree
<drakonis>otherwise the rebuild cost is negligible
<drakonis>anyways, ditch sqlite and encode more useful data into the store
<drakonis>there's an abundant amount of work that can be done to improve everything in the ecosystem
<drakonis>i'll work on this
***kir0ul4 is now known as kir0ul