IRC channel logs


back to list of logs

***sneek_ is now known as sneek
***tolk`` is now known as tolk
<dsmith>sneek, later tell wingo v2.1.0-1098-gaf95414 failed in FAIL: test-language
<sneek>Got it.
<mark_weaver>dsmith: I think I fixed that.
<mark_weaver>you might try pulling again.
<dsmith>Ya, just did
<dsmith>Exciting times!
<dsmith>mark_weaver, Yes, I think it's gotten beyond that point
<dsmith>Totals for this test run:
<dsmith>passes: 38822
<dsmith>failures: 0
<dsmith>unexpected passes: 0
<dsmith>expected failures: 7
<dsmith>unresolved test cases: 569
<dsmith>untested test cases: 1
<dsmith>unsupported test cases: 10
<dsmith>errors: 0
<mark_weaver>yay :)
<nalaginrut>morning guilers~
<mark_weaver>hi nalaginrut!
<mark_weaver>fyi, wingo uploaded some interesting code to the wip-cps-bis branch.
<dsmith>sneek, later tell wingo And v2.1.0-1107-g062888f Passes make check. Yey!
<sneek>Got it.
<nalaginrut>alas~the times issue yesterday was hurd's bug
<nalaginrut>fortunately ;-P
<mark_weaver>it's not exactly automatic, but I just compiled a little procedure to RTL, assembled it, and ran it :)
<dsmith>I wonder how wingos wip-cps-bis relates to Noahs wip-rtl-cps
<dsmith>mark_weaver, Cool!
<nalaginrut>I'm reading rtl code last night
<nalaginrut>but haven't tried any example
<nalaginrut>well, bad tense
<mark_weaver>it's very preliminary. it can compile simple procedures with '+', but if I use '-' or '*' I hit a bug :)
<mark_weaver>well, maybe not so much a bug as some primitive arity tables not yet fully populated or something.
<nalaginrut>hmm...thanks for mention it, I'll show '+' to frighten other guys ;-P lol
<mark_weaver>ah, I just found the table that I need to extend. excellent :)
<nalaginrut>what's the most proper way to do AOT job? maybe translate rtl bytecode to ASM?
<nalaginrut>hmm..but there's assembler in master
<nalaginrut>I've no idea how to do with it
<dsmith>nalaginrut, It all happens with compile
<mark_weaver>on the wip-cps-bis branch, you can do this: (use-modules (system base compile) (system vm assembler)) (define* (my-compile x #:key (env (current-module))) ((assemble-program (compile x #:env env #:to 'rtl))))
<mark_weaver>and then do things like this: (my-compile '(lambda (foo x y) (+ x y)))
<mark_weaver>and get back a procedure that's implemented using the RTL VM, as you can see by using ,x on the returned value.
<mark_weaver>and you can run it.
<mark_weaver>however, the set of primitives that is currently supported is very minimal right now.
<mark_weaver>actually, + is about the only thing I've found so far that works :)
<nalaginrut>I'll try it soon, compiling it
<nalaginrut>only for wip-cps-bis?
<mark_weaver>in my private tree, I just successfully added support for (- * / 1+ 1-) :)
<nalaginrut>so the 'assemble-program' assembles the code to RTL-bytecode, right?
<nalaginrut>not native ASM code?
<nalaginrut>so my question is what's the most proper way to convert it to native ASM code?
<nalaginrut>IMO, just translate it
<nalaginrut>my purpose is to write an external tools for bytecode->native-asm
<mark_weaver>wingo and I will be working on this very soon.
<nalaginrut>mark_weaver: alright, it's interesting, now I believe it's real registerVM lol~
<nalaginrut>where is the definition of (version) ?
<nalaginrut>ok, got it
<nalaginrut>I realized PACKAGE_VERSION was generated by git-version-gen, which will be UNKNOWN if it's not a git repo
<nalaginrut>it's strange that guile2 depends on git to generate version, it causes the independent tarball can't get version correctly
<nalaginrut>well, seems the magic is to generate .tarball-version file to hold the actual version
<youlysses>I asked the other-night, but I'm pretty-sure I got no response -- has there been anymore progress to/on this ?
<youlysses>Even if such-a project, while cool, is even worthwhile -- with Wayland just around the corner. :^P
<nalaginrut>well, I'm not familiar with X ;-)
<youlysses>I'm very interested, but I'm afraid of investing myself into something that's more-or-less not based on and/or some-factor of compatible with "latest and greatest" tech. For a fairly related example, the promise of Wayland and the potential of a Guile-based Compositor based on-top of it, has limited my long-term investment into Stumpwm -- though I love the WM.
<Chaos`Eternal>I've tried his guile-xcb
<Chaos`Eternal>and the window-manager in the samples
<Chaos`Eternal>it works
<Chaos`Eternal>at least on guile-git
<nalaginrut>is guile-xcb strong enough to write a nice window-manager?
<Chaos`Eternal>not sure, since I am not familiar with X either
<nalaginrut>maybe it's worth writing one for Hurd, since Hurd could use only WM now
<youlysses> nalaginrut: Could you rephrase the latter part, the "since Hurd could use only WM now"?
<nalaginrut>I should use 'at this time' not 'now'
<nalaginrut>well, I should refactor my English
<youlysses>nalaginrut: Ah; Personally I'd like to see a sort of "Emaculate DE". Basically a collection of applications centered around the emacs-like methodology of use. Which, for the most-part -- seems like the tools are slowly but surely coming together. :^)
<Chaos`Eternal>well, I've had thought of write a tiling window manager using guile
<nalaginrut>yes, the emacs-like methodology is the guile-way (how puzzled philosophy it is) ;-)
<nalaginrut>Chaos`Eternal: so do it
<youlysses>Chaos`Eternal: Which again, while exciting/cool -- I fear it might be a waste of efforts in the long-term, if one doesn't go the Compositor for Wayland route. :^P
<Chaos`Eternal>but i found it is more complex than i can handle
<nalaginrut>agreed, considering wayland is better
<youlysses>nalaginrut: Well it's more-or-less universally agreed that it's going to be "the thing" for the next 20 years or-so. :^P
<nalaginrut>I think based on given WM lib and extent it
<nalaginrut>youlysses: well, I didn't say Wayland is better, but as I know, many main-stream distros choose to compatible with Wayland
<nalaginrut>hmm...I'm too utilitarianism
<youlysses>nalaginrut: I don't think I implied you did? I meant that it's pretty-much decided to be the big-thing that most major distros and related communities are actively moving to -- and from what I understand, for some good reasons.
<nalaginrut>youlysses: hah ;-)
<nalaginrut>but, if guile-emacs works then, maybe no one wants WM under hurd...
*nalaginrut is kidding..
<youlysses>nalaginrut: I actually lived under *exclusively* under Emacs for a month or-two, so it's possible. That being said, I think the emacsy route will probably when-out in the end, but I'd be glad to see Emacs edge closer to a LispM... :^)
<Chaos`Eternal>lispmachine.. the anceint curse
<youlysses>Chaos`Eternal: More-like powerful, lost magic.
<Chaos`Eternal>that's why scheme is a language of ... the foresaken
<youlysses>Chaos`Eternal: There's an overwhelming negative connotation assigned to Scheme...?
<nalaginrut>well, we have FPGA now, so lispmachine is not a dream this day, and it could be practical
<nalaginrut>the problem is what's the most practical scene of lispmachine?
<youlysses>nalaginrut: "scene"?
<nalaginrut>“scene" means the need suitable to work with lispmachine under certain environment
<Chaos`Eternal>there is some practice on bootstraping scheme upon ARMcpu
<nalaginrut>picobit could run on ARM
<mark_weaver>wingo: is there something I could read that might help me understand your cps representation?
<wingo>morning :)
<sneek>Welcome back wingo, you have 2 messages.
<sneek>wingo, dsmith says: v2.1.0-1098-gaf95414 failed in FAIL: test-language
<sneek>wingo, dsmith says: And v2.1.0-1107-g062888f Passes make check. Yey!
<mark_weaver>greets :)
<wingo>mark_weaver: have you read kennedy's "compiling with continuations, continued" ?
<mark_weaver>ah, not yet, will do :)
<wingo>np :)
<wingo>it's not exactly the same but it's not dissimilar
<mark_weaver>I'm trying to get the rtl compiling working a bit more :)
<wingo>super :-)
<wingo>it's pretty easy to stumble on an unimplemented bit :)
<mark_weaver>indeed, it took a few tries to find the implemented part, haha
<wingo>or a buggy piece...
<mark_weaver>I added support for * / - 1+ 1- easily enough. then stumbled onto a bug in the handling of conditionals.
<wingo>was it in compile-rtl.scm or in the allocator?
<mark_weaver>it's in compile-cps.
<wingo>exciting ;)
<wingo>so about that
<mark_weaver>(make-$continue kif (list test)) in the <conditional> case of 'convert'
<mark_weaver>that (list test) should instead be a <cps> something or other.
<wingo>indeed, wonder how that happened
<wingo>i think it should be (make-$var test)
<wingo>the idea is this:
<mark_weaver>seems reasonable
<wingo>you compile to (let ((test TEST)) (if test (kt) (kf)))
<wingo>that would compile to a br-if-true
<mark_weaver>I was just trying to figure out what the 'make-$letk*' did exactly, and why I see no variable names in there.
<wingo>but at some point we should run an optimization pass that would allow some tests to be evaluated directly with the $kif as their continuation
<wingo>which would allow them to be compiled as br-if-tc7, br-if-eq, etc
<wingo>but we start with the conservative general approach and then specialize the cases we know we can handle
<wingo>so that the compiler can be straightforward, and we don't have to have so much logic in the compiler itself
<wingo>ok bound variables
<wingo>variables are bound by continuations
<wingo>specifically $kargs continuations
<wingo>so a (let ((x 1)) x) becomes
<wingo>($ $letk (($ $cont SRC K ($ $kargs (x) (x-sym) ($ $continue 'ktail ($primcall 'return (x-sym)))))) ($ $continue K ($ $const 1)))
<wingo>at least, the "return" is added in there by the arity conversion pass
<wingo>so you see here that there is one $letk term that defines one continuation (the list of one $cont) and has a body
<mark_weaver>okay, thanks for the hints! I think I will have to read that paper to fully grok what's going on here.
<wingo>the body continues to K with 3
<wingo>and K binds that value to X-SYM
<wingo>and K's body continues to 'ktail with the return primcall
<wingo>i think i've read that paper like 10 times
<wingo>it's good but a little dense :)
<wingo>you can think of this example as (let ((k (lambda (x) (return x)))) (k 3))
<wingo>except that continuations are represented using a separate type -- they are not first-class lambdas
<wingo>anyway, enjoy the paper :)
<mark_weaver>ah, so the variable bound by the 'let' is K within the cont.
<mark_weaver>okay, that helps.
<wingo>yes and no -- that variable is a continuation, not a value
<wingo>so it's a label, not a slot in a stack frame
<wingo>but in a scoping sense, that's what's going on, yes
<mark_weaver>fair enough, and understood.
<mark_weaver>yeah, I meant "variable" in the more general sense as in something being bound.
<wingo>indeed the dfg.scm treats them uniformly
<wingo>a data flow-graph on uses and definitions of continuations is a control-flow graph :)
<mark_weaver>I'm a bit puzzled by the fact that the continuation consists of both $cont and $kargs within. my first instinct is that the $cont is essentially part of the description of the outer $letk.
<wingo>yes that is an arbitrary decision
<wingo>i think it turns out you often want to deal with a continuation and its name
<wingo>and each continuation is named precisely once
<mark_weaver>I could imagine that it somehow makes the structure easier to work with.
<wingo>so it makes sense to join them in one structure instead of having to correlate multiple fields in the $letk
<mark_weaver>do $cont nodes appear anywhere else?
<wingo>the only appear in two places
<wingo>in $letk
<wingo>and as the body of a function
<wingo>because it's useful to give a label to the body of a function, and continuations seem to be a good way to do that
<mark_weaver>and in that case, what is the meaning of the 'K' in the $cont ?
<wingo>in the function case?
<wingo>it is bound by the $fun, not the $cont
<mark_weaver>is it the continuation that returns from the function?
<wingo>it is the entry continuation
<wingo>that's also the only place $kentry appears
<wingo>you will note there is no $fun-case or lambda-case thign
<wingo>that is because the body of a function is ($ $cont src k ($ $kentry arity body alternate))
<mark_weaver>ah, so within a $fun, there is always a $cont and within that a $kentry?
<wingo>also, as a hack, the tail continuation in a function is just named "ktail"
<mark_weaver>what is the scope of 'k' ?
<wingo>an unhappy hack, but there you go -- it seems to be fundamental
<mark_weaver>if it is the entry continuation, that makes me think it will be referenced by those who wish to call the function.
<mark_weaver>is 'K' ever referenced from within the function?
<mark_weaver>sorry for all the questions, I should probably just read the paper :)
<wingo>mark_weaver: those that call the function reference the function -- unless you somehow manage to prove that you are calling a known function, in which case you could jump directly to k (possibly after linking a call frame)
<wingo>no, they are great questions and i'm happy to explain
<wingo>so a number of the closure optimizations in orbit and in that O(0) closure optimization paper are about known closures
<wingo>contification is one of the optimizations, but that is separate
<wingo>sometimes you still need to push on a call frame, but you can still optimize known calls
<mark_weaver>*nod* I'm familiar with the opportunities of known closures :)
<wingo>and you would do that using some branch-linked instruction we don't have, effectively to do a call but to a known label
<wingo>a self-call within a closure (tail or not) could use the label
<wingo>tricky at the top level, given redefinition, but lexically it can work
<mark_weaver>ah, nice!
<wingo>and it would be nice to have a better module compilation story
<wingo>and you can express that self-call optimization in the language -- you replace a call to a lexical with a $continue K ($ $values ...)
<wingo>having names for everything has been wonderful
<wingo>also for known closures it's possible to think about eliding "slot 0", the procedure slot
<wingo>note that in the rtl vm, slots start from 0, and slot 0 is the procedure (in the standard calling convention), not the first arg
<wingo>or you could replace slot 0 with something else
<mark_weaver>sounds good :)
<wingo>anyway, all the usual orbit tricky
<mark_weaver>I've been doing: (define* (my-compile x #:key (env (current-module))) ((assemble-program (compile x #:env env #:to 'rtl))))
<mark_weaver>is there a better way that I missed?
<wingo>backtraces get more tricky, but we will always be able to map from IP to function, so that's ok
<wingo>mark_weaver: no, that's what i use
<wingo>actually i usually stop after the rtl, look at the code, then assemble it
<wingo>the whole thing needs a test suite, eh...
<mark_weaver>do you have anything written here to evaluate tests in a value context, or is that on the todo?
<wingo>on the todo
<mark_weaver>okay, just wanted to make sure I didn't miss it :)
<mark_weaver>well, it's very exciting stuff!! :)
<wingo>i'm stoked you're excited about it :)
<wingo>lots of work yet to do, but hopefully we should be able to compile eval.scm soon enough
<wingo>not sure how much time you have, but help is very very much appreciated --
<wingo>it was quite difficult up to now, with things changing so much
<wingo>but now that the languages are more or less settled and the compiler works we can start working on pieces that make things better
<wingo>like the optimization passes for example
<mark_weaver>I'd be glad to help
<mark_weaver>actually, I was going to ask about that. do you plan to run some of the existing tree-il passes on the tree-il before converting to cps?
<mark_weaver>I noticed that you wrote code to handle letrec nodes.
<mark_weaver>or maybe it was noah who wrote it
<wingo>yes see compile-cps.scm
<mark_weaver>but I think the fixing-letrec pass should probably come before conversion to cps.
<wingo>before converting we run the tree-il optimizer
<wingo>which does the fixing-letrec
<mark_weaver>ah, good.
<wingo>and indeed, it happens before cps
<wingo>another thing
<wingo>cps has $letrec, but that's rewritten in the closure-conversion pass
<wingo>the idea is that $letrec binds mutually-recursive first-class functions
<wingo>as from <fix>
<mark_weaver>ah, okay
<wingo>but that a pass should run early to contify all possible functions
<mark_weaver>makes sense.
<mark_weaver>so cps $letrec is constrained to lambda expressions?
<DerGuteMoritz>all *typical* functions!
<wingo>and afterwards in the closure-conversion pass $letrec is rewritten as effectively let + set!
<mark_weaver>(or the cps equiv)
<wingo>DerGuteMoritz: in this case it's a lexical thing, not a global thing, so "possible" is the right word here ;)
<wingo>mark_weaver: yes
<mark_weaver>I wonder if the fixing letrec algorithm would make sense to use there as well.
<mark_weaver>fixing-letrec-reloaded I mean.
<DerGuteMoritz>wingo: hehe, dunno, I was just referring to the nice diagram you tweeted yesterday or so :-)
<wingo>DerGuteMoritz: hehe, that's the thing i should be hacking on right now ;)
*DerGuteMoritz puts on BOSS hat
<wingo>mark_weaver: dunno, it seems that if we can do that at the tree-il level, we should
<wingo>getting scheme's letrec semantics into cps is tricky, and any prior simplification is helpful
<mark_weaver>oh, we definitely need to do it before conversion to cps.
<wingo>ah ok
<wingo>then yes, i think fixing letrec reloaded would be great
<mark_weaver>but then you mentioned the closure-conversion pass turning letrec into let + set!
<wingo>it opens up more contification possibilities
<mark_weaver>and I wonder if the set!s could sometimes be avoided, and if there's a benefit to doing so (not sure)
<wingo>well... really what it turns them into is let ((f (make-closure label nslots)) ...) (set-free-var! f 0 val) ...
<mark_weaver>but maybe the earlier fixing-letrec-reloaded pass will take care of all (most) of those opportunities already.
<wingo>which is what the tree-il compile-glil does, incidentally -- nice to be able to take that logic out of the rtl emitter
<mark_weaver>ah, I see
<wingo>but yes: more <fix>, more contification, better code
<wingo>so fixing-letrec-reloaded can help
<mark_weaver>I'll dust off my code for that soon.
<mark_weaver>well, it can compile (lambda (x y) (if x y x)) now.
<wingo>excellent :)
<wingo>small steps ;)
<mark_weaver>replacing that 'if' condition with something else like 'null?' still doesn't work though.
<mark_weaver>s/null?/(null? x)/
<wingo>what if you replace with "pk"
<wingo>if (pk x) ...
<wingo>just because it's not one of the special primitives that if is trying to recognize
<mark_weaver>when calling the procedure produced by compiling (lambda (x y) (if (pk x) y x)), 'pk' gets called with no arguments.
<wingo>probably something wrong with call frame allocation
<wingo>the idea is that a call frame has space for all calls that it makes
<wingo>so making a call means moving the procedure and arguments into place
<wingo>possibly shrinking the stack using reset-frame
<wingo>(that lets the callee know how many arguments are given, because it checks based on the sp)
<wingo>and then calling
<wingo>when the called function returns,
<wingo>it moves its return values into place, but doesn't reset the stack
<mark_weaver>what's wrong with that rtl code?
<wingo>i mean it resets the stack to hold just the correct number of values
<wingo>the first return value is in slot 1
<wingo>relative to the called procedure
<mark_weaver>I still have much to learn about the RTL.
<wingo>so then the calling procedure deals with arguments, and resets the stack back to its size
<wingo>so the sp dances a bit around calls
<wingo>shrinking before the call to indicate the number of arguments
<wingo>shrinking before a return to indicate the number of return values
<wingo>and possibily growing again after dealing with a return to reset to the frame's "natural" size
*wingo looks
<wingo>mark_weaver: i think (call 7 1) should be (call 7 2)
<wingo>because the called frame will have two locals: the procedure and the argument
<wingo>thank you :)
<wingo>feel free to push whatever you like to that branch btw
<mark_weaver>okay, will do :)
<wingo>we should rebase before merging or presenting it, but it's fine to work incrementally for a while
<mark_weaver>I guess in (emit `(call ,proc-slot ,nargs)), that 'nargs' should be '(+ nargs 1)' (compile-rtl.scm line 186 or so)
<mark_weaver>maybe also on line 276
<mark_weaver>(those lines are in my changed version, maybe a bit off)
<wingo>or we could change the instruction
<wingo>dunno, that was a very recent change -- i went back and forth on calling convention for a while...
<mark_weaver>which do you think is better?
<wingo>i have no idea :)
<wingo>i would just patch the compiler for now to be consistent
<wingo>and if we decide later to do something different, we fix all the things
<mark_weaver>well, that fixed it :)
<wingo>excellent :)
<Chaos`Eternal>breaking breaking
<Chaos`Eternal>may i ask whether we have performance comparisons between guile-2.2 and guile2.0?
<mark_weaver>something is wrong here, because in (if (null? x) y x), 'null?' is still getting compiled in a value context.
<nalaginrut>well, you dig so carefully ;-)
<wingo>Chaos`Eternal: not while 2.2 is broken :)
<wingo>mark_weaver: null? shouldn't be present at all, i though it would be compiled as a primcall
<Chaos`Eternal>broken? ...omg...
<mark_weaver>we're ending up with:
<mark_weaver>(letk ((k kif123 (kif k124 k125)))
<mark_weaver> (let k128 (tmp v129
<mark_weaver> (continue k128
<mark_weaver> (primcall null? #{x 113}#)))
<mark_weaver> (continue kif123 (var v129))))
<Chaos`Eternal>i switched my whole environment to guile-git...
<nalaginrut>I think master is fine
<wingo>so the cps call pessimizes the (if TEST (KT) (KF))
<mark_weaver>when it's compiled to rtl, I guess the 'null?' call is being evaluated in value context not test context.
<wingo>to (let ((TMP TEST)) (if TMP (KT) (KF)))
<wingo>and we are missing the additional pass to re-inline the primcalls we know can be evaluated directly in $kif context
<wingo>like null?, and others
<wingo>what to do about null? in a generic value context is another question...
<wingo>we could compile to a little (if (null? foo) #t #f), or to a call-out
<wingo>i guess the former would be best
<mark_weaver>seems sensible
<wingo>or we could add an opcode for null?
<mark_weaver>not just null?, but all the others.
<wingo>would seem to me to be much less common than null? in test contexts though
<wingo>and for native compilation you will want to compile to the (if (null? foo) #t #f) dance, so i would do that
<wingo>that would be a thing to add to the arity-conversion pass btw
<wingo>if you happened to focus on that ;)
<mark_weaver>I don't know what 'arity conversion' is :)
<wingo>well in scheme all expressions return values
<wingo>but not all vm ops make values
<wingo>so the arity conversion says, ok, i'm calling this primcall op that i know makes one value but it's in the head of a $seq
<wingo>so i need to convert it to be in a $kargs so that i know where to allocate the return value
<wingo>s/$seq/$kargs with no values/
<wingo>and likewise, some primops like set-car!, in value context, should be transformed to continue with the unspecified value
<mark_weaver>understood, thanks :)
<wingo>(language cps arities) for the code
<wingo>so it just needs some more cases
<mark_weaver>and the arity conversion pass is the place to convert (null? foo) in a value context to (if (null? foo) #t #f) ?
<wingo>yes i think so
<mark_weaver>the later cps pass won't pessimize it again?
<wingo>around arities.scm:244, add a case for branching primops
<wingo>no, pessimization happens on tree-il->cps conversion
<mark_weaver>ah, right, of course.
<mark_weaver>wingo: can a primcall be an argument to $continue?
<wingo>mark_weaver: a primcall can only be an argument to $continue
<wingo>$var, $void, $const, $prim, $fun, $call, $primcall, $values, and $prompt can only appear as the "exp" of $continue
<wingo>the semantics being "call this continuation with FOO"
<wingo>where FOO is $void, the results of a $call, etc
<wingo>and if the continuation is 'ktail, it's in tail position
<mark_weaver>I'm trying to figure out what the generated CPS should look like for (if (null? x) #t #f)
<wingo>so the whole thing has a continuation, call it kval
<wingo>(make-$letk (list (make-$cont #f KT (make-$continue kval (make-$const #t))) <same for KF and #f> (make-$cont #f KIF (make-$kif KT KF))) (make-$continue KIF <the null primcall>))
<wingo>surely we can make some kind of macro that can help us build cps better
<wingo>as it is, it's horrible
<mark_weaver>thanks much!
<wingo>np :)
<mark_weaver>that example helps a lot. I think I'm starting to get a handle on this representation.
<wingo>so one thing to note is the letk defines mutually recursive continuations, logically
<wingo>and we use the scope chain as an approximation to the dominator relationship
<wingo>so, and i don't know if this is really important or not, but you might want to bind the $kif in an inner $letk
<wingo>indicating that its body dominates the bodies of the kt and kf continuations
<wingo>but i have no idea, more serious optimizations have yet to come, and they have to deal with this kind of thing anyway
<mark_weaver>makes sense, will do.
<wingo>i'm curious to see how CSE will work out -- fluet & weeks found that you had to rewrite cps quite a lot to do good cse
<wingo>anyway, idle thoughts
<wingo>, if you haven't read it, is really interesting
<wingo>i come back to it every year or so ;)
<wingo>their system is different from ours (e.g. they lift all lambdas early, so everything is first-order), but it's good to think about
<mark_weaver>I'll add it to my queue :)
<mark_weaver>do you think that all occurrences of branching primcalls should be converted to the (if <primcall> #t #f) thing? and then they'll be optimized later?
<wingo>hum, that sounds like a step backwards
<wingo>i guess we need to do the optimization at the same time
<wingo>so for each branching primcall, either we inline it to an existing $kif, or we materialize a new $kif
<wingo>i suppose you could do both and rely on dead-code elimination to remove the needless materialization
<mark_weaver>I wonder if branching primcalls should be handled differently in the tree-il -> cps pass.
<wingo>yes, perhaps
<wingo>that would be another way to do it
<wingo>i really don't know :)
<wingo>i guess it's best not to make garbage code to begin with, and if we can do that in the ->cps pass, that would be ok
<mark_weaver>seems like it would be good to avoid creating lots of bloat only to be optimized later, where it can be done without making things too ugly.
<mark_weaver>yeah, I think this will be better.
<mark_weaver>wingo: btw, in that <conditional> case of 'convert' that was broken before, you had bound two variables with 'let' that didn't end up being used: 'kifvar' and 'var', both gensyms. any idea what you might have been planning to do with those?
<mark_weaver>(around line 418 of compile-cps.scm)
<wingo>looks like i simply didn't finish that pessimization that i wanted to do
<mark_weaver>okay, this is where I need to handle branching prims differently anyway, so I'll rip those out.
<wingo>yeah sounds good
<wingo>yes, i guess convert-arg already handled converting to a variable
<wingo>which is why i didn't need those extra vars
<wingo>so i think you would change to just use "convert" instead of "convert-arg", for branching primitives
<mark_weaver>hmm, okay.
*mark_weaver looks more carefully
<wingo>convert-arg gives you the gensym of a lexical, bound to the result of converting an expression
<wingo>convert gives you a term
<mark_weaver>it appears that convert-arg also gives terms
<wingo>well yes it evaluates to a term
<wingo>but i guess by "give" i was referring to the expansion-time continuation
<mark_weaver>oh, I see.
<wingo>yeah, it's a twisty algorithm
<wingo>it seems to be the standard way of doing things, though
<wingo>i looked at a couple of papers and they do it in the same way
<mark_weaver>it looks fine, it'll just take some time for it to become familiar
<mark_weaver>wingo: I wonder if the use of the special symbol 'ktail', which is not unique, will make analysis more difficult.
<mark_weaver>I can see how it is convenient when generating rtl code.
<wingo>yeah, dunno
<wingo>we could bind a $cont _ K ($ktail) or something in a function
<mark_weaver>I think something like that would be wise.
<wingo>and then all lookup-conts succeed for all K, and we distinguish tail continuations as being known as $ktail
<wingo>sure, we can do that
<wingo>s/known as/represented as/
<wingo>i guess we'd add a field to $fun for the tail continuation
<mark_weaver>even that might be constraining.
<mark_weaver>using the name "$ktail" to detect the tail continuation, that is.
<wingo>mark_weaver: i mean a new $ktail type
<wingo>not a name
<mark_weaver>what if some manipulation interposes a different ktail, or something. maybe it can't happen, dunno.
<mark_weaver>oh, maybe. hmm.
<mark_weaver>adding the field to $fun sounds good though.
<wingo>i can do that, or you can if you like
<wingo>if you do it, do it in a separate patch please
<wingo>the field should probably be called "tail" and it should probably hold a ($cont #f K ($ktail))
<wingo>where K is unique to the function
<mark_weaver>is there a reason that the assembler should just keep track of the name of the tail continuation, instead of relying on it being specially marked somehow?
<wingo>no reason in particular, no
<wingo>in fact having it be distinguished by type instead of by name would be easier
<mark_weaver>I guess I don't know what you mean by "type". isn't it just a gensym?
<mark_weaver>I have vague worries about treating this continuation specially causing complexity or bugs somewhere else.
<mark_weaver>but maybe I worry too much :)
<wingo>so look at compile-rtl 286
<wingo>there we would match on ($ $ktail)
<wingo>instead of #f
<wingo>that's what i mean by matching on type
<mark_weaver>I can see how it's more convenient here, at rtl generation, of course.
<wingo>likewise arities.scm:78 would remove the first clause of the and
<mark_weaver>but it seems to me the more important question is: how will it interact with all the analysis and transformation that is done in the cps representation.
<wingo>because each function's tail continuation would have a unique name, instead of just "ktail" as it is now, it would work better for interprocedural analysis
<wingo>and the only way to name a continuation is via $cont
<mark_weaver>so one issue is, how does it interact with read-only analyses.
<mark_weaver>and that's fully addressed by using gensyms.
<wingo>so you need a kind of continuation, hence, $ktail
<mark_weaver>the other issue is: what kind of transformations will be done on the cps code. where you are generating new cps code based on the old, and maybe renaming continuations and so on.
<wingo>yes, like contification
<mark_weaver>well, I'll take your word on it :)
<wingo>in that case you replace the ($continue K $call) with a ($continue KENTRY $values), and replace the KTAIL in the fun with K
<wingo>by "replace the KTAIL" i mean rewrite all uses of KTAIl to instead use K
<wingo>where KENTRY and KTAIL are the gensyms for entry and exit of the fun being contified
<mark_weaver>wingo: somewhere (letk ((k kif1 (kif k1 k2))) (continue kif1 (primcall null? x))) needs to be specially handled, I guess? the arity conversion pass is choking on this, because it can't determine the arity of 'null?'.
<mark_weaver>do you think this should be handled by simple matching, or is there a need for deeper analysis?
<wingo>yes that needs to be handled specially in arities.scm and in compile-rtl.scm
<wingo>i would do simple matching
<mark_weaver>okay, sounds good, thanks!
<wingo>thank you!
<mark_weaver>welcome :)
<mark_weaver>so I guess subtrees of that form needn't be changed at all by arity analysis, right? (assuming that the primitive is a branching type)
<mark_weaver>the arguments of the primcall are simple enough that they don't have to be recursed on?
<nalaginrut>mark_weaver: I've heard that CPS is difficult to translate to ASM?
<mark_weaver>nvm, it's obvious enough
<wingo>nalaginrut: not the case at all!
<nalaginrut>so it's not? ;-P
<wingo>emitting asm from cps is easy
<mark_weaver>yeah, I can compile (lambda (x y) (if (null? x) y x)) now and it works :)
<wingo>mark_weaver: excellent!
<mark_weaver>nice code too :)
<wingo>disassembly? :)
<nalaginrut>I'm reading a comparison between ANF/SSA/CPS, and encounter such a rumor ;-P
<mark_weaver> 0 (assert-nargs-ee/locals 3 0) ;; 2 args, 0 locals
<mark_weaver> 1 (br-if-null 1 #f 3) ;; -> L1
<mark_weaver> 3 (return 1)
<wingo>nalaginrut: the rumor is incorrect :)
<mark_weaver> 4 (return 2)
*nalaginrut recording it
<wingo>yes that is very nice code
<nalaginrut>well, seems such a slide was written by a ANF fans, and ANF seems perfect in it...
<wingo>i wrote an anf converter for guile and it didn't convince me
<nalaginrut>well, I'm glad you made the decision after compared all of them (actually SSA is not the case)
<nalaginrut>hmm, the slide final rate CPS as 1st, I speak too quick
<nalaginrut_>mark_weaver: where did cps insert? between tree-il and glil?
<mark_weaver>tree-il --> cps --> rtl
<mark_weaver>the rtl compiler doesn't use glil
<nalaginrut_>OK, you answered my second question ;-)
<mark_weaver>it's not inserted. basically, everything below tree-il is different
<nalaginrut_>ok, so it's replaced a bit
<nalaginrut_>I just saw there's glil
<nalaginrut_>so I thought it may be inserted
<mark_weaver>wingo: compute-dfg currently can't cope with references to free variables.
<wingo>mark_weaver: it should run after closure conversion, so there shouldn't be free variables per se
<wingo>i suppose we could make it more general
<wingo>probably a good idea
<wingo>that might involve making it have two different modes
<mark_weaver>well, it's just that compiling things like (lambda (x) (lambda (y) (+ x y))) currently fails.
<wingo>one where it recurses into nested lambdas, and one otherwise
<wingo>mark_weaver: dunno, closure conversion should have done its job so everything is local...
<mark_weaver>okay, thanks for the hint.
<wingo>obviously a bug somewhere :)
<mark_weaver>wingo: I guess the problem is that 'compute-dfg' is applied to the function body, just inside of the $fun, where the free variables are listed. 'def!' is never applied to those free variables in 'compute-cfg'.
<wingo>mark_weaver: not sure i understand you
<wingo>in (lambda (y) (+ x y)), the reference to "x" should... ah.
<wingo>ok i get you now.
<wingo>so, free-ref is particular in some ways
<wingo>"x" gets compiled to (free-ref <self> x)
<mark_weaver>I guess I'll add the list of free variables to the argument lists of 'emit-fun-body', 'allocate-slots', and 'compute-dfg'.. maybe?
<wingo>and the intention was to compile that to (free-ref <self> idx), where idx is the index of x in the free vars
<wingo>mark_weaver: it's not really a reference for the dfg, though
<wingo>it's an index into the closure's free variables
<wingo>so i guess closure conversion should rewrite that to actually take the index as an argument
<mark_weaver>oh, so the free variable references should be ignored by compute-dfg?
<wingo>i mean i guess the real thing is to residualize free-ref primcalls that take indices as arguments
<wingo>in the meantime it would be ok to hack around that in compute-dfg
<wingo>does that make sense?
<wingo>apologies for that
<mark_weaver>no worries, I'd like to fix it properly.
<wingo>sounds good
<mark_weaver>so basically the representation of free-ref and free-set! will change during closure conversion?
<wingo>then the proper fix is in closure conversion
<wingo>well free-ref and free-set! get introduced in closure conversion -- the fix is to make them take the free var as an index, not by name
<mark_weaver>oh, I see, okay.
<mark_weaver>looks like some code makes assumptions about the arguments to primcalls. will a bare number as an argument be well handled?
<wingo>so in cps all values are bound to variables, constants included
<wingo>you'd need to introduce the constant via $continue k $const to a $kargs with a name
<mark_weaver>I wonder if free-ref and free-set! should have dedicated nodes.
<mark_weaver>well, I guess maybe it's better to cope with the bloat than to make the code more complex.
<wingo>that's a possibility, but primcalls work, so we should probably continue with that...
<wingo>probably best to get experience with this style before possibly deciding on another one -- and in particular with optimizations using this style
<mark_weaver>makes sense.
<wingo>kennedy likes it this way because you don't need a general notion of substitution -- you always just replace variables by variables
<wingo>but there are many other papers and systems that don't bind names to constants
<mark_weaver>I like the uniformity.
*wingo too
<wingo>i don't like the verbosity, but cps is just that way ;-)
<mark_weaver>I guess I'll have to do this as a second pass within 'convert-closures'.
<wingo>yes, sounds sensible to me
<wingo>unless you add to the free list imperatively
<mark_weaver>yeah, I thought of that.
<wingo>nastiness all around :)
<mark_weaver>I think I'll go with a second pass :)
<taylanub>I wonder if it's intentional that (/ 1.0 0.0) and (/ 1 0.0) are +inf.0, but (/ 1.0 0) throws an exception ?
<nalaginrut_>I think it's intentional
<nalaginrut_>#ifndef ALLOW_DIVIDE_BY_EXACT_ZERO
<nalaginrut_>but I wonder what's the meaning of it
<mark_weaver>yes, it is intentional
<mark_weaver>(/ x 0.0) is the limit of (/ x y) as y goes to zero from above.
<mark_weaver>(/ x 0) is undefined
<nalaginrut_>so it's something to do with limitation/approximation when it's 0.0
<nalaginrut_>well, I see
<mark_weaver>in the realm of real numbers, you can make sense out of (/ x 0) only in terms of limits. but only the inexact signed zeroes and infinities are evaluated as limits.
<mark_weaver>you can ask about the limit of (/ x y) as y approached zero, but the answer depends on whether it approaches from above or from below.
<mark_weaver>when you evaluate the limit of (/ x y) as y approaches zero, you never actually evaluate (/ x 0). you only evaluate (/ x y) for very small y.
<mark_weaver>(/ x -0.0) is the limit of (/ x y) as y approaches zero from below, btw.
<nalaginrut_>I have a question, maybe meaningless, how to enable ALLOW_DIVIDE_BY_EXACT_ZERO?
<mark_weaver>I don't recommend it, and quite likely something is broken in the handling of that.
<nalaginrut_>just curious
<mark_weaver>probably you'd add -DALLOW_DIVIDE_BY_EXACT_ZERO=1 to the CFLAGS or something, dunno.
<nalaginrut_>OK, I thought there's a way to enable it on the fly
<mark_weaver>personally, I'd like to get rid of the option. it's just dumb.
<nalaginrut_>I have to go to bed, night guilers~
<mark_weaver>g'night nalaginrut!
<mark_weaver>wingo: I found the problem. 'init-closure' is creating a 'free-set!' to initialize the closure slots, but that code is of course ending up outside of the closure body itself.
<wingo>mark_weaver: yes that should be fine though
<wingo>the free-set! takes the closure from a local slot
<wingo>or it should anyway
<mark_weaver>it might be with your code. but the code I wrote to change the free references to indices assumed that they would be accessed in scope :)
<wingo>it's not like the old vm where the closure is an implicit argument
<mark_weaver>it's okay, I'll fix it :)
<wingo>heh, ok
<wingo>for free-set! you just need to change the var to be an index
<wingo>probably there is a value as well that doesn't need to be changed
<wingo>anyway, i'm sure you're on it :)
<mark_weaver>yeah, I'll fix it. just a mistaken assumption, that's all :)
<mark_weaver>I didn't anticipate free variable accesses out of scope. though you did warn me about this before.
<wingo>which var is accessed out of scope?
<mark_weaver>the free variable in 'free-set!', created by 'init-closure'
<wingo>in (free-set! closure slot value), which one of those three?
<wingo>the "slot" one i guess? surprising, since the slot label corresponds to a var that the closure captures, it should be bound at that point
<mark_weaver>in the code I'm looking at, it looks like there are only two fields in free-set!
<mark_weaver>it's bound, but not as a free variable yet.
<mark_weaver>or not in that scope. it's bound using kargs there.
<wingo>yes it looks like free-set! is missing an argument, doesn't it?
<mark_weaver>so the problem is, how to determine which $fun is being referred to.
<mark_weaver>since that same variable may be found in more than one nested $fun, the answer is not unique.
<mark_weaver>I cannot simply have a global lookup table keyed by gensym.
<wingo>right, that's crazy
<wingo>ok so i think (1) the free-set! residual calls are bogus
<wingo>they need three args
<wingo>the closure, the slot, and the value
<wingo>the closure should be a bound variable from a make-closure primcall
<wingo>the slot index should be the index of the slot var within the function's free list
<wingo>which you can get directly as another parameter to the fold in init-closure
<wingo>and then that leaves "free", which is probably the value that you need
<wingo>so you leave that one as is, it should be bound
<wingo>does that make sense?
<wingo>you add an (iota (length free)) to that fold, and then there you are
<mark_weaver>okay, makes sense, thanks!
<wingo>apologies again!
<mark_weaver>no need to apologize, I said I was eager :)
<wingo>well, thank-yous then :)(
<wingo>:) i mean
<taylanub>sneek: Later tell nalaginrut C pre-processor macros like #if and #ifdef change the code that is compiled at all, so the produced executable doesn't have any of the code that was excluded, so one cannot switch between the options on-the-fly, or even by restarting. Only by recompiling.
<sneek>Got it.
<taylanub>Are things like variable-set! and set-car! guaranteed to be atomic during parallelism, or should I overhaul my false intuition for parallelism ?
<amk9>hi :)
<amk9>I'm trying to create a stack of context
<amk9>basicaly a FILO of hash tables
<amk9>I did this
<amk9>environment-new-context! does't edit the list in place as it should instead it returns a new list which should actually by env
<ijp>that code is just so wrong
<ijp>first off, contrary to popular misunderstanding reverse! and append! do not necessarily mutate the list
<ijp>secondly, why are you reversing, appending, and then reversing?
<ijp>lisp lists are linked lists, so you are doing three traversals where you could have done one cons
<ijp>stacks _are_ linked lists
<didi>ijp: Please, don't hold back.
<amk9>the thing is that I will always walk the stack from last to first
<amk9>that's why I always reverse it
<ijp>(reverse (append (reverse l) (list x))) == (cons x l)
<ijp>didi: I know it comes off harsh, but I'm just very confused by it
<amk9>how do I do to mutate it ? I mean do the thing in-place
<ijp>three ways: 1, dont (my preferred method). 2, mutable box containing the list, or 3
<amk9>what it 3 ?
<ijp>I'm just writing it
<ijp>(define (push! x ls) (if (null? ls) (throw 'pick-your-favourite-behaviour-here) (let ((tmp (car ls))) (set-cdr! ls (cons tmp (cdr ls))) (set-car! ls x))))
<ijp>actually I should double check that first, since code written directly in your IRC buffer is invariably wrong
<amk9>ues of course
<amk9>why do you say 1) don't do it ?
<amk9>I mean, is mutation nota good thing ?
<ijp>I haven't mutated a _list_ in years
<ijp>nor a string
<amk9>so you do «let» stuff all around ?
<amk9>all around the code ?
<ijp>functional programming 101: don't mutate a structure, return a new one
<amk9>I assume yes
<ijp>but if I wanted a mutable stack, I'd do 2
<ijp>which avoids the ugly '() case
<amk9>ok I take into account 1) for the future but I'll go 2 I think
<ijp>actually, that's one of the reasons why append! and reverse! and co. don't guarantee mutation
<ijp>because you can't mutate an empty list
<ijp>well, for reverse! that doesn't matter, since there is only one empty list
<amk9>this doesn't make sens to me
<ijp>so the correct pattern for append! and reverse! is (set! bar (append list bar)) (set! foo (reverse! foo))
<amk9>the doc says procedure! does the thing «in place»
<ijp>pretty sure it says "linear update"
<amk9>which means ?
<amk9>«append! modifies the given lists to form its return. »
<ijp>if that is in 2.0 docs, it's just wrong
<amk9>the thing is that I already use the return value of the functions to do some stuff
<amk9>if I had to go the functional way, I had to return several values
<didi>Can sneek eval?
<ijp>didi: no
<didi>Aw. Bummer.
<amk9>that said it's not impossible
<amk9>aka. it's possible ;=
<didi>The docs should probably say "append! might modify the given lists to form its return."
<amk9>is it ok to use multiple return values, it always seemed to me a bit akward in Python
<didi>amk9: Really? I like them in Python.
<didi>So easy to use.
<ijp>bleh, stupid emac