<zimoun>indeed, it was when I gave a look at the debconf program :-) <drakonis>in my opinion, the most important thing about consfigurator is how unassuming it is <drakonis>i would like to eventually get to make my own thing in cl <drakonis>guix inherits too many assumptions from nix <drakonis>the presence of a specific kind of store located on a toplevel subdirectory, derivations, daemon <rekado_>“too many” sounds like it’s bad – that’s why I asked :) <drakonis>the ones that are indeed bad have been excised long ago <drakonis>1.4.0 is shaping up to be the best release so far <zimoun>About “too many”, using a functional lens, I do not how it could be different: you need a central memory and a daemon. *civodul is skeptical about Propellor and Consfigurator <civodul>a solid layer on top of a flaky foundation <zimoun>I do not think manifest should be pinned. The pin is the channels.scm :-) Otherwise, it already exists: it is inferior. :-) Well, I do not get what “nix flakes is. <drakonis>nix flakes is a fancy nix thing for channel management and pinning <drakonis>consfigurator is pretty much ansible slash fabric <drakonis>the thing i like the most about it, is being able to resume execution on the target host <rekado_>civodul: it can be good enough, especially when it improves significantly over doing things the old way <zimoun>nix flakes is the same thing feature as inferior, IIUC? <drakonis>regarding the existence of a central memory and daemon, the bad thing that's being inherited with the daemon is that it is opaque for debugging and inspecting <rekado_>it’s no Guix, but it’s how I like to manage configurations for non-trivial system deployments <drakonis>its not like you can have it pause execution on breaking <rekado_>drakonis: that’s something that we really should address. <rekado_>I keep imagining a future in which the daemon is written in Guile and we can open it up a little more easily <drakonis>indeed, but the only way out is by axing the current daemon implementation <zimoun>well, I am more often annoyed by Guile error than by daemon error. Even with Geiser, the debuging is poor… and I usually end using good old ’pk’ :-) <rekado_>there was some momentum there to replace the daemon. But the failed GSoC project and the protracted university project to continue it kinda swallowed all momentum <drakonis>since you have to rewrite more pieces of guix and guile to support the improved debugging features <drakonis>i spent a little bit of time trying to understand how to follow the daemon but it requires more thorough changes <drakonis>perhaps running it on a networked multithreaded repl would be a good starting point <drakonis>provided the builds can be already invoked from the repl <rekado_>perhaps it would be a good idea to write a bunch of tests to have an executable specification of the daemon <drakonis>is there any way to find the boundaries between the daemon and guile? <rekado_>because we already have a whole lot of features that the daemon needs; but I wouldn’t feel comfortable cobbling them together without tests. <rekado_>the interface between daemon and Guix is the derivation <drakonis>last i checked it was spread across guix/store/ <drakonis>is it too large a goal to ditch the derivation interface? <rekado_>yes, there’s some communication there, but the stuff that we want the daemon to do is all encoded in the derivations <zimoun>by “executable specification” I have read a Coq formal thing ;-) <rekado_>IIUC it was targeted because it would allow reuse of the nix daemon <rekado_>zimoun: my goals are way more primitive :) <rekado_>there’s no *need* to use nix derivations <rekado_>especially now that nix and guix have diverged so much <rekado_>but I guess we’d still want to be able to work with “old” derivations in a store <drakonis>find the piece that serializes the expressions into derivations <rekado_>things like write-list, for example. <rekado_>write-derivation pulls it all together <rekado_>inferiors are a much later addition. They are all about evaluating expressions in “guix repl”. <rekado_>e.g. to look up packages in an older version of Guix <drakonis>basically, inferiors that predate the new daemon would either require spinning up a new instance or simply consume the definitions <drakonis>there's plenty of considerations here if i want to overthink about it <rekado_>I guess it would also be fine to do a hard break and offer a migration command <drakonis>in my terrible opinion, maybe it'd be fine <rekado_>but I don’t think the on-disk derivation file format is a major obstacle to overcome <drakonis>saving it in s-exp form would be nice for loading <rekado_>we can read and write these files already, so I don’t think it’s urgent to move away from that format <drakonis>but the main urgency is to replace the scheduling bits? <drakonis>its been a little while since i've checked what the daemon still uses <drakonis>what still needs to be done to actually engage on replacing it? <drakonis>the only patches not merged are the ones that (guix environment) <civodul>rekado_: would be interesting to see what can be learned from the interfaces of Propellor/Consfigurator and applied to 'guix deploy' <drakonis>there are a few things i can name right now, being able to deploy multiple hosts <drakonis>resume execution on the target host, though i think that exists in a milder form? <drakonis>you dont have to change sudoers to use sudo ***alex__ is now known as alexdesiqueira
<zimoun>it is interesting because they do not mention funflow by Tweag which is a layer of the top of Nix managing kind of workflows. <zimoun>Well, the interesting part, IMHO, is about workflow and type. <zimoun>What could be interesting is to have common “records” (BAM, Fastx, etc.) and be able to type check beforehand to know if the workflow makes sense or not. Instead of rely on kind of filename and string. <zimoun>But I do not know if it is possible with Guile without introducing many macros or similar. <rekado>zimoun: yes, I heard of bionix. (I think there was a paper?) <rekado>I think it would be good to move away from file names, but it’s not obvious how to do it. <rekado>early GWL had no cache and no assumption about files being used as inputs or outputs <rekado>so you could have a process that updates a database record, for example <rekado>I don’t think types are necessarily the right answer, though. <rekado>I don’t like the habit of encoding process progress information in file names. <rekado>but I don’t know of any obviously better way <rekado>nextflow does it well, I think, by focusing on streams instead of files <rekado>that’s something I’d like GWL to do as well <zimoun>rekado: CWL (and ccwl) has type too. <zimoun>well Bistro (workflow on the top of OCaml) has type too. To me the issue with types is that you have to define them. <zimoun>Well, few are aroud (BAM, FASTx, etc.) but it can be difficult when it is not provided by default. <rekado>with programming languages I can see types prevent annoying problems, because composition of functions to purely functional programs can be very complex <zimoun>Or if the process outputs more and you are not interested by the default type but another file. <rekado>but for workflows I just don’t see the immediate benefit <rekado>I guess typing inputs and outputs would allow for certain input/output filters <rekado>the benefit of type checking the workflow seems not to be really practical to me <zimoun>Well, Python is a raw workflow because it glues many things. And I can tell it is really annoying when the error is at the end. But indeed, GWL does breakpoints by writing the intermediate result to some files. So it becomes possible to just re-run the part which failed. *zimoun has to go. Will read the log. :-) <rekado>…and this is also why it’s hard to move away from files <rekado>but I’d really like to go the nextflow route and annotate inputs and outputs as streams, files, or plain values <rekado>and then connect processes over the network, piping inputs and outputs from one process to another, and doing the equivalent of “tee” to optionally cache streams <rekado>on the other hand this would make process scheduling and execution across nodes a lot more complicated <rekado>all of a sudden you’d have to make sure that nodes can talk to one another over the network, or that they can talk to a coordinator node, which then would become a bottleneck <zimoun>Extend pipe with an equivalent of “tee” for optionally caching the streams appears to me a route <zimoun>I mean, a route that I would like GWL would explore. :-) <rekado>my problem is that I’m nothing but ashes. Burned to a crisp. Can’t muster enough motivation to hack on anything :-/ <rekado>I’m using this down time to read more about biology. That’s fun. <rekado>guess I’ll need to find someone to co-maintain the GWL <zimoun>I totally understand. I recently have tried to reorganize myself and tweak my workflow. Well, structured procrastination.