IRC channel logs
2017-07-24.log
back to list of logs
<paroneayea>rekado_: if you're looking to look at lexical environment and etc <paroneayea>rekado_: maybe a combination of current-dynamic-state and the-environment may provide something <paroneayea>I'm guessing even if somehow these were to give you the right info, and I don't feel confident they would <paroneayea>you'd need to write serializers for all the information <paroneayea>rekado_: I suspect anything of the sort would be fragile though <paroneayea>rekado_: I guess maybe this depends on whether you're talking about doing so ahead of time or during runtime maybe too if you're thinking ELF and etc apply? <paroneayea>but I am pretty naive, just trying to think out loud. <rekado_>paroneayea: my goal here is to create frozen slices of Guix, so that the state of the package graph can be restored from that slice at a later point without having to have a copy of the old Guix. <rekado_>I want the graph to live on, even after the dead build artifacts have been installed and Guix has moved on. <rekado_>I want to be able to just run Guile against any store item and explore the package closure from there. <rekado_>but without having to store *all* of Guix for each item. <rekado_>This was actually requested a couple of times here at this conference. <rekado_>People were easily sold on the idea that having programmatic control over the package graph is very useful. <paroneayea>it feels to me like traversing all the bits you need in the guix codebase might end up traversing all of guix itself <rekado_>and they were kinda bummed to hear that this is essentially lost when Guix “moves on” after installation <rekado_>take (gnu packages games) for example. I need none of that stuff when I install “samtools” from (gnu packages bioinformatics) <rekado_>I really just want to dump the closure – or maybe even *less* than that <paroneayea>rekado_: well, I suspect if you pull it off, you'll manage to learn a lot more about how guile works! :) <rekado_>I feel that the easiest way would be to have one Guile module per Guix package. <paroneayea>at that point, maybe we really shouldn't be using define-module <rekado_>and then install that module with the store item. <paroneayea>rekado_: I don't know what you're talking about, I love that python.scm has all the python stuff I need in one place! ;D <paroneayea>rekado_: storing a module with a store item would be interesting and maybe even could work, but <paroneayea>rekado_: it might also break if we change our apis under the hood <rekado_>well, I wouldn’t want to install python.go to a store item that doesn’t need it. <paroneayea>rekado_: but, I guess the challenge of those modules breaking is just the same as channels breaking <paroneayea>you now have a core guix with package bits disconnected from the core <rekado_>re changing apis: the Guix API may change; we would just install (gnu packages) and (guix packages) with the module. <paroneayea>the core machine could change and the surrounding bits break... it's a risk <paroneayea>rekado_: I mean things like the actual arguments and etc <rekado_>I would hope that Guile’s ELF format wouldn’t change again, so I’d assume that this is stable and future-proof <paroneayea>say we refactored all the python modules to remove package-with-python-2 or whatever it's called <rekado_>package-with-python2 creates a package object <rekado_>and I don’t want the code to depend on the *current* guix <paroneayea>rekado_: so as long as you aren't "linking to" a procedure we've removed... <paroneayea>guix seems mostly stable-ish on that stuff, but I guess we'd *really* find out if it were if we had this feature ;) <paroneayea>and that's why I'm saying it's the same challenge that channels would have <paroneayea>because if we link to procedures removed, etc, or arity / argument order changed etc <rekado_>I got really excited about this when I played with it – I just didn’t notice that I had GUILE_LOAD_PATH and GUILE_COMPILED_LOAD_PATH set, so I was actually just using the *current* Guix instead of my “serialized” blob. <rekado_>I guess I’d need to figure out a way to resolve modules at compile time and inline the parts that we’re using. <paroneayea>rekado_: wait a moment though, even if we packaged every package in its own ~module <paroneayea>you'd still need to include all dependency ~modules <rekado_>but they’d be in their own store item <rekado_>so if I wanted to look at the closure of a package I’d load up the modules of the union of the store items that make up the package closure. <paroneayea>what about when we do fancy things like (substitute-keyword-arguments (package-arguments another-package)... <rekado_>each of these modules is incomplete on its own, but taken as a union they would be sufficient to describe the package closure as a “live” graph <rekado_>paroneayea: these are evaluated when the value for the package object is computed <rekado_>it’s only a problem if you have delayed computations somewhere <rekado_>and the arguments themselves are just data anyway. <paroneayea>rekado_: maybe you don't even need to serialize the environment in some cases <paroneayea>and can actually serialize the package object itself <paroneayea>I suspect you'll ahve to solve the closure problem anyway <rekado_>we already do something custom, actually <rekado_>all the references are recorded in the database <rekado_>taken together this allows us to traverse the closure <paroneayea>rekado_: sounds like you've got some fun hacking to od <rekado_>but what I’d really like is to be able to use, say, “package-source” on an actual package object, long after the thing has been installed. <rekado_>maybe we could even replace the database at some point … <paroneayea>ShedDweller: see the docs, they're pretty good on this subject <ShedDweller>paroneayea: Alright, I might check the docs out, thanks <paroneayea>ACTION reading "how to write ports in C" part of guile docs <paroneayea>I need to get better at C to understand how to better contribute to Guile's internals <rekado_>I create a new environment, define a value “foo” in it, then compile “foo” using the environment … <rekado_>… and this just results in a *lookup* for the value “foo” <rekado_>it does not pre-evaluate “foo” in that environment, nor does it embed the environment itself. <rekado_>it only generates: #<tree-il (toplevel foo)> <rekado_>of course, loading this all on its own is going to fail, because “foo” is undefined. <rekado_>Am I misunderstanding the env argument of “read-and-compile”? <ennoausberlin>Hello, I wrote a few lines of code to understand the for-each procedure but at some point I get an error <lloda>ennoausberlin: file-exists? expects a string and you're passing symbols <ennoausberlin>Wrong type argument in position 1 (expecting open file port): todays-dir <lloda>do (list todays-dir ...) instead of '(todays-dir ...) <lloda>`(,todays-dir ,log-dir ...) would also work <ennoausberlin>lloda: I guess, I have to reread the quote chapter in my study book. <janneke>paroneayea: #*&(&#!@@@@@#! E: [pulseaudio] main.c: Daemon startup failed. <janneke>i only use audio/video once in a while, so sad how brittle that is <paroneayea>well, it's William Byrd (of minikanren, Little Schemer, etc) talking about "Maxwell's Equations of Software", but not *your* maxwell's equations of software specifially... talking about lisp in lisp, and showing all the mind blowing ways you can alter it <janneke>of course my mes is just one incarnation of The "Maxwell's Equations of Software" <janneke>i'm working very hard on getting tcc to compile and run <janneke>ACTION google says to remove state: rm -r ~/.config/pulse <paroneayea>ACTION comes back to suspending custom i/o ports <paroneayea>wingo: okay, so I don't know how you feel about this, but my feeling on how to make custom binary i/o suspenable: <paroneayea> - add read_wait_fd/write_wait_fd to the custom_binary_port struct, and make use of it in the way that file ports are used <paroneayea> - add keyword arguments to make-custom-binary-(input,output,input/output)-port for read-wait-fd and write-wait-fd <paroneayea>in the case of https, we'll actually just be passing through the underlying port's information <paroneayea>however it doesn't appear possible to do define* type things in C, unless I'm misreading <paroneayea>which means that I'd probably have to have make-custom-binary-[io]-port stuff be in scheme, passing in those arguments or #f. <paroneayea>or does anyone else with enough ports internal knowledge have an opinion before I spend time on this? (maybe civodul ?) <civodul>paroneayea: i think at this stage you know more than me about suspendable ports :-) <paroneayea>well I don't know much about C and guile's C internals! <paroneayea>I think that's what's making me so afraid to hack it <civodul>perhaps you can do things in Scheme like you wrote? <paroneayea>civodul: hm I don't think so, the custom-binary-i/o stuff is all in C, and slots there basically need to be exposed <paroneayea>civodul: so once the slots are added in C, I think yeah, you can actually do the wiring of this in scheme <civodul>i have to go (really, that's not because i want to escape problems ;-)) <civodul>let's talk about it at a later point if you want <rekado>I’m playing with serialisation and I gotta say: I’m such a noob! <rekado>I get easily confused with nested quoting. <rain1>i just had a thought, would it be possible to turn the lambda in your code to some kind of data structure? and calling it with an apply function <rain1>the ones that needed serialized <rekado>what I’m doing now is this: a) get the struct to be serialized, b) go through all its field, c) generate an S-expression, d) feed that to (compile e #:from 'scheme #:to 'bytecode) <rekado>the S-expression I generate has to be sufficient to reconstruct the state of the struct. <rekado>I find it sad that that’s not so easy. I always found printers that produced something like #<package ...> ugly. <rekado>When the reader encounters “#<package ...>” it won’t know what to do with it. <rekado>this is why I need to go the extra mile and make sure that the S-expression is just a bunch of constructors. <rain1>hmm this is an interesting problem.. <rain1>i wonder if there could be a writer that throws an exception when if it has to print something which cant be read back <rekado>maybe I can just overwrite the writers. <rekado>(or are they called “printers” in Guile?) <paroneayea>rekado: I've been thinking about making a serializer package for guile that uses something along the lines of canonical s-expressions <paroneayea>rekado: the really nice idea that canonical s-expressions has is its "display hints" field <paroneayea>which I think is a good place to give extra type info <rekado>what conditions need to be fulfilled before (procedure-source some-procedure) will return something other than #f? <rekado>I enabled source copying with (read-enable 'copy), but I cannot seem to get the source code for other modules or even local procedure definitions. <rain1>rekado i am curious what this is for ? maybe its some code online I could see? <rekado>rain1: I’d like to figure out if we can store “slices” of the package graph with the build artifacts <rekado>this would make it possible for a user to look at relevant parts of the package graph at a later point without having to have an old version of Guix. <rekado>I’d like to have a lispier alternative to the Guix store database. <rekado>the database keeps a record of package references, but that’s not nearly as nice as exploring the package graph in a REPL. <rekado>one way to do this is to just store *all* compiled Guile modules that make up the particular Guix version and store or link those for every installed package. <rekado>on the other hand it seems much less like a hack <rekado>I guess I’ll have to get the sources by reading from source files. <rekado>maybe “procedure-source” should be removed from the documentation, or a warning be added that you’ve got to put something in there before retrieving it. <rekado>now that I’m motivated to look at linker things I think I’ll take a closer look at the first task <paroneayea>rekado: I actually want to work on the "opposite" one, basic register allocation :) <paroneayea>really because I want to add more support for debugging info <rekado>how does this work? The scope of interest is only known at debug time, isn’t it? So how can one know ahead of time what values need to be kept alive? <paroneayea>but that's what wingo says I need to look at to make it happen :) <paroneayea>right now guile has an in-theory-amazing debugging suite at the repl <paroneayea>but in reality everyone uses (pk) because ,locals tells you often nothing <rekado>many of the tasks on that list sound like they’d be fun to work on. <paroneayea>I need ot overcome my fear of hacking guile itself <paroneayea>also, you can tell I'm a stereotype of this channel: I'm compiling guile's stable-2.2 branch on my server from the guix environment that I'm running my project out of because I need some of the recent commits <paroneayea>I really need to find time to work on guix-deploy too but that won't happen until next year <davexunit>I want struct field unboxing, but the real task is huge I found out. <wingo>ACTION will have some guile time over the next couple weeks <wingo>been zero guile time over this last month :P <sneek>Welcome back wingo, you have 1 message. <sneek>wingo, paroneayea says: so as you may have seen, I now have 8sync running entirely on top of fibers... almost! Everything works but I don't have a way to halt an actor's event loop to shut it down. I thought I had a solution with conditions, but it seems that watchers don't get removed after they "move on" by something else being selected in a choice-op. I wonder what the right solution is, and if you have thoughts? <rekado>ACTION reads the recommended series of blog posts on linkers <wingo>i don't know what you mean by "watchers" in that note <paroneayea>wingo: sorry that I've kinda overwhelmed your backlog possibly :) <paroneayea>wingo: so say you've got a condition `stop?' (I have no idea how bad of abuse it is to use a ? postfix on conditions) <paroneayea>basically so that when your async queue fiber is looping around and doing <paroneayea>watcher here actually refers to the use of the term in the conditions code I think <rain1>the fibers post was really awesome I learned a lot from it... I had been thinking continuations were not really a "useful" thing more like a puzzle thing to play around with/induce headaches so it smashed that & it's really cool to see them used in a practical way <wingo>anyway is good to define terms :) <paroneayea>wingo: so the problem with this "solution" is that every time you loop a new waiter gets consed on <paroneayea>eventually the condition grows way outta control... yow! <paroneayea>if they were going to work, they'd have to "unsubscribe" when another choice happens <wingo>usually what you need to do is arrange for somebody to do "gc" <wingo>basically, every so often, tidy up the set of waiters <paroneayea>would be to have stop? actually be an atomic box, checked right before perform-op at the top of the loop <wingo>so like every Nth wait you go through the whole set of waiters and prune out the events that are already in 'S state <wingo>but also GC when a waiter is woken up <paroneayea>wingo: aha, maybe the condition itself could do that? <wingo>or when you wake up a waiter <paroneayea>wingo: want me to give that a try and send you a patch? <paroneayea>wingo: btw we shouldn't miss out the real meat of the news on this though! <paroneayea>which is that 8sync on fibers otherwise seems to work :) <paroneayea>so 8sync can become an optional layer of actors and etc <paroneayea>performance-wise I couldn't test because the bug we just talked about killed performance so bad that testing anything else, I dunno :) <paroneayea>I also took an opportunity to spiff up the way that actors "reply to" each other <paroneayea>by two things: you no longer explicitly "reply" to a message, that happens automatically by the system replying with the return values <paroneayea>and! that gets passed back to the original actor's continuation <paroneayea>so "sending a message that will wait for a reply" involves continuations in more ways than one :) <paroneayea>but actors can handle other correspondence while waiting for replies on the messages they send with (<-wait ...) <paroneayea>and results in way less verbose code than before <paroneayea>wingo: I suspect it'll be peformant enough, since I also cut out the "delivery actor" in most cases, the Hive <wingo>do you take advantage of multicore? <paroneayea>actors run in their own fiber (well technically two fibers, one for the queue and one for the actor) <paroneayea>wingo: so the other bit of news is that I'm trying to make custom binary i/o ports be suspendable <paroneayea>wingo: heh, well I need to get up a version by tomorrow but I can have that one only half working. so really, as soon as I can... I'm trying to do it this week, and I think I can <paroneayea>I just want to run whether or not what I'm doing makes sense by you <wingo>but i will read it in the morning :) <wingo>happy hacking and good luck! <rekado>paroneayea: is this for the web module you’ve been working on? <paroneayea>rekado: not for guile-webutils (though that's used) but for Pubstrate <paroneayea>which is the federated social networking software and standard test suite <rekado>have you published guile-webutils yet? <rekado>I’ll have to write another web thing and it would come in very handy, I’m sure. <paroneayea>rekado: I have a repo, but no release yet, because deadlines <paroneayea>I'm making git-based packages adding to the guix.scm I'm using it in :) <magnicida>hi, here i am again trying to find a nice way to do unit tests in guile <catonano>so tonight it was about linking, debugging and, tangentially, unboxing <catonano>magnicida: look, I'm the last fool here. But for a toy project of mine I ust used the basic vanilla test runner. It worked <catonano>magnicida: I may be misinterpreting yor needs, though <magnicida>catonano: what is the basic vanilla test runner? <catonano>but it's a valid template for binding a C library AND running some tests <magnicida>thanks a lot, actually i did not realise srfi-64 was included in guile <magnicida>aha! the guix.scm in that repo also solves another problem of mine: defining an guix.scm to put in the repo that uses the current checkout <magnicida>i was trying to find a way to use guix inside travis but i could not find anything easy <magnicida>wow, this is weird, so if I use the (test-group macro) to define all my tests I get this error in the line where the macro is first used: <magnicida>In procedure struct_vtable: Wrong type argument in position 1 (expecting struct): #f <magnicida>but if I do define one test before using (test-begin) and (test-end) everything works <magnicida>mmm ok it seems that there is no test-runner-current <magnicida>one is created by test-begin but not by test-group... looks like a bug to me but not sure 100%