IRC channel logs

<rekado>I feel compelled to pick up my sledge hammer for some reason

<rekado>I often wonder about how we can bridge the gaps between user habits and our vision of computing

<rekado>there often is a real convenience cost in doing things the way we’d like, because *we* don’t provide services and platforms but others do. And those other platforms are incompatible with our paradigm.

<rekado>it seems quaint to still talk about Docker in 2023, but its early popularity and later continued entombment in the foundations of new computing habits tells us a story that may hold lessons for us

<civodul>re convenience cost, it's maybe not all that obvious

<civodul>getting started with one of these platforms is not easy either

<civodul>it takes training, you have to get an account on an existing one or build a whole new team in your IT department to deploy one

<rekado>people aren’t *excited* about Docker anymore; but it has won its place among the ranks of foundational technologies to a point where services consider the software deployment problem to be solved by it

<civodul>true

<rekado>at this point one cannot meaningfully combine Guix with, say, kubernetes or an openstack installation

<rekado>because most of its value is lost in the translation from high level abstraction to dumb Docker blobs

<rekado>store deduplication, garbage collection, provenance

<civodul>in a way we're kind of "forced" to develop our own vision, as opposed to plugging in existing tool sets

<rekado>yes

<rekado>and I don’t think that scales well

<civodul>yeah

<rekado>ACTION tries not to look at gwl

<civodul>but! it's also more satisfactory, from an intellectual standpoint

<rekado>oh sure

<civodul>we prolly need to think about bridges

<rekado>but that (understandably) doesn’t matter to many people who aren’t used to wading knee deep in the muds of package management

<civodul>yeah

<civodul>"guix pack -f docker" is one such bridge

<civodul>in that it lets you say: "i'll stop here with Guix and use that other tool for the rest of my trip"

<civodul>the repo2docker thing was an attempt in that direction too

<rekado>“we need to build this application, with these dependencies” (me: starts packaging lots of python stuff) “oh, we need it to be scalable, and we can’t host it on site, and it shouldn’t be too expensive, so make it use kubernetes with auto-scaling on AWS EC2 instances with spot pricing…” (me: scrambles)

<rekado>usually I run into early resistance

<rekado>oh, it’s difficult to package this, eh?

<rekado>but it exists in conda

<rekado>well, let’s just skip this complication of using Guix then, shall we?

<civodul>heh

<rekado>(and then I observe chat messages about some libc symbol versioning error messages because the conda thing was built on Ubuntu and not RHEL…)

<civodul>yeah, that makes sense i guess

<civodul>:-)

<civodul>i'm lucky i work with hard-core/old-style HPC folks who're not into k8s aws wtf

<rekado>if we aren’t getting any benefit out of using Guix for the whole thing (not just to provide an environment) then what’s the use anyway?

<rekado>that’s the sentiment

<rekado>I think the docker output is a bridge too soon

<civodul>what would be the right level?

<rekado>no idea

<rekado>even when using rented ad-hoc machines I want to be able to bring my own software, but I don’t appreciate the harsh downgrade from Guix to Docker.

<rekado>I want to be able to use my Guix things with kubernetes if I must deploy scalable services, but I don’t want to be forced to convert it all to blobs

<civodul>and installing Guix there is inconvenient?

<civodul>ok

<rekado>shared storage on kubernetes is considered very unusual

<rekado>and installing Guix on short-lived EC2 instances is also not great, because it takes time and i/o

<rekado>I’d love to just use Guix System, but I can’t just boot it on EC2

<rekado>lots of little problems like that

<rekado>we can’t specify Guix System OS definitions in a way that any of the established tools in the “cloud” realm understand

<rekado>Guix System support for cloud-init would be helpful, I suppose

<rekado>and it probably wouldnt’t be too hard to implement

<civodul>ok

<civodul>this was discussed a while back (with you?) but i forgot the details

<rekado>maybe

<rekado>I’m very forgetful when it comes to this stuff

<rekado>I sometimes have radical ideas about adapters that are specific to a couple of popular tools or platforms

<rekado>e.g. a Guix service for kubernetes and deeper integration so that guix-based services could be implemented *without* actually shipping a whole container blob (instead getting the needed store items from the deployed Guix service in the same kubernetes cluster).

<rekado>I have the impression that we attract the kind of people who are generally not embedded in those communities

<rekado>so our development efforts generally don’t overlap with whatever these communities do

<rekado>and since we don’t talk to one another there’s really not even a miniscule influence we can exert on the direction of change in those communities

<rekado>and since the guix journey ends with “guix pack -f docker” for users of those systems there’s no incentive for influence in the other direction either.

<civodul>ACTION nods

<rekado>I feel like we’re missing out on a lot of potentially valuable contributions because of a lack of bridges between *communities*

<rekado>but I really have no good ideas on how to change that

<rekado>(and how to avoid the opposite problem of bending ourselves out of shape to fit in where we can’t)

<civodul>i guess we need to look at concrete cases, like the cloud-init example

<rekado>yes

<rekado>it’s also not clear to me if efforts of integration with kubernetes would be useful at all — or if it wouldn’t be better to provide a leaner alternative

<rekado>even if that seems like NIH syndrome — at least we’d have (re)invented something in ways that *would* work for us.

<rekado>we have many of the pieces we’d need already

<rekado>shepherd, guile-ssh, metabash even, and of course guix system and guix deploy

<rekado>attempts to replace kubernetes with something simpler (but with a compatible interface) are not uncommon

<rekado>minikube, k3s, etc

<civodul>i don't have those needs, i barely know what k8s is meant to solve :-)

<civodul>(that's part of the problem too: that core contributors don't really care what happens outside)

<rekado>:)

<civodul>or put differently, we're developing an insular culture

<PurpleSym>Perhaps the lack of overlap in communities stems from the lack of overlap in values. From an ideological standpoint it’s not desireable to use „the cloud“ (which, in practise, is often very proprietary) when rooting for free and open hardware and software.

<rekado>PurpleSym: while I think that that’s true to some extent, we also have perfectly fine self-hosted solutions that are free software and that are not integrated with Guix

<rekado>ganeti is a welcome exception, but our local expertise isn’t exactly overwhelmingly spread out among contributors

<rekado>openstack and its grab bag of libraries and tools is another candidate

<PurpleSym>Sure, OpenStack and Kubernetes (afaik) are free software, but running it yourself undermines the reason to use the cloud in the first place: Not having to employ someone to run it.

<rekado>kubernetes makes it easy to have different on-demand scalable services sharing the same ‘computing substrate’ (= the kubernetes cluster)

<rekado>this often *is* run on rented machines, because that’s where that seemingly limitless capacity resides

<rekado>but even if we see it as a cloud-first technology it has the very welcome effect of reducing vendor lock-in as you don’t need some proprietary scaling solution that locks you into one hoster

<zimoun>hi!

<civodul>o/

<civodul>welcome back, zimoun! :-)

<zimoun>piouf, too much news to read…

<civodul>:-)

<civodul>i'll be talking tomorrow before an HPC audience that includes Conda users

<civodul>what should i say wrt. Conda?

<civodul>(i have ideas but i'm not the most knowledgeable, so i'm outsourcing the effort :-))

IRC channel logs

2023-06-19.log