IRC channel logs


back to list of logs

<amz3>can I inspect a module ?
<amz3>also if I understand correctly, one *has* to export things to be able to use them from another module
<amz3>I think, maybe because in Python it's otherwise, it should be open by default and restricted upon keywords
<mark_weaver>it's possible to get a private bindings in a module, using the (@@ (module name) symbol) syntax.
<mark_weaver>from the REPL, you can use the ,m command to "enter" a module, so the REPL is within the module.
<mark_weaver>(everything you evaluate will be evaluated within that module)
<mark_weaver>modules are first-class objects too. see
<mark_weaver>there's also 'module-map' and 'module-for-each' that will iterate over all the bindings in an arbitrary module.
<mark_weaver>amz3: what are you trying to accomplish?
<mark_weaver>I strongly disagree that modules should export everything by default.
<mark_weaver>anyway, it's too late to change that.
<mark_weaver>but in normal usage, you need to explicitly export the bindings that should be exported.
<mark_weaver>you could make a macro to do that automatically though.
<mark_weaver>actually, there already is one: it's called 'define-public'
<mark_weaver>just use that instead of 'define' and it'll take care of exporting it.
<amz3>so it's good, I just need to recall to use that
<amz3>I'm investigating what do to next
<amz3>I would like to generate documentation from modules
<amz3>that's all :)
<amz3>but I'm not sure
<mark_weaver>amz3: you keep changing your nick :)
<amz3>yes ^^ amk9 make me think of to dogs too much
<mark_weaver>it helps for me to have context about who you are.
<mark_weaver>it's okay
<amz3>I've been lurking around couchbase, another nosql database, it looks nice
<amz3>it's the new couchdb
<amz3>that resulted from the merge of couchdb and membase
<mark_weaver>does this have any relation to guile?
<amz3>no sorry I stop there
<mark_weaver>regarding generating docs from modules...
<mark_weaver>if you have an idea for a basic approach of how such modules would be written, I might be able to help you accomplish that.
<mark_weaver>do you just need to scrape the docstrings out of all the procedures?
<mark_weaver>(do you know about docstrings?)
<mark_weaver>another approach would be to use macros that would contain the doc snippets, and they would expand into nothing when loaded as code, but expand into something else when loaded by the doc generator.
<mark_weaver>or the doc generator could use 'read' to read the source file directly, and look for certain known top-level constructs.
<amz3>I will use 'module-map' or 'module-for-each' and retrive the docstring
<mark_weaver>'read' makes things pretty easy because it handles the parsing entirely.
<amz3>yes but I have to extract the infos
<mark_weaver>the infos?
<amz3>the docstrings
<amz3>whereas if I load a module I have access to the docstrings directly
<mark_weaver>the only thing that makes me worried about the docstrings is that in core guile, we're going to interpret those as texinfo at some point in the future.
<mark_weaver>if you build something that assumes they'll be something else, there will be conflict.
<mark_weaver>but procedures can have arbitrary meta-data associated with them, and it's possible to make custom 'define' macros that will store additional meta-data in the procedure.
<mark_weaver>so you could have your own separate key in the meta-data for the info you need.
<mark_weaver>although I tend to think that having a doc generator read the source directly might be best. then you'll be able to put stuff in there that isn't associated with a procedure.
<mark_weaver>for example, variables don't have docstrings.
<amz3>for all this reasons I'm still thinking about this particular feature
<mark_weaver>'read' gives you each top-level form as a nested data structure, as if you put a quote in front of it.
<amz3>pretty amazing btw
<mark_weaver>so it's super easy to just look at the 'car' of each of those things being read.
<mark_weaver>the 'car' is the first element in the list, so it would usually be 'define' or 'define-public', or your own custom 'define' macro that includes a slot for your doc snippets.
<mark_weaver>'read' is actually what the compiler and interpreter use to parse the file, btw.
<amz3>in the previous version of my static blog generator I was using it directly
<mark_weaver>you'll probably want a way to add docs for things like structure definitions as well.
<mark_weaver>and advanced scheme code tends to make their own custom 'define' constructs as well.
<mark_weaver>so you should take all of this into account.
<mark_weaver>I think associated the doc snippets with the run-time objects, and then scraping them from the module at run-time, is a lose for what you want.
<mark_weaver>IMO anyway
<mark_weaver>but if you really want the runtime approach, guile also has something called "object properties" that let you associate arbitrary meta data about any non-immediate object.
<mark_weaver>(implemented as a weak-key hash table)
<amz3>also I was thinking that from the user (of the documentation) perspective that she would rather look directly at the (highlighted) code like
<mark_weaver>'read' loses whitespace and comments, and some other details like whether you wrote 'a or (quote a), etc.
<mark_weaver>so if you want to preserve the exact source, you'll need to use lower-level I/O (maybe read-line) instead.
<mark_weaver>but a hybrid approach is also possible, where you use 'read' in a first pass, and then read the exact text in a second pass.
<mark_weaver>you can connect them together using the filename/line/column information that is associated with every non-immediate datum returned by 'read'.
<amz3>thx for the idea
<mark_weaver>happy hacking!
<mark_weaver>(note that older versions of guile only recorded source information for lists, but even that would probably be enough for what you need.)
<mark_weaver>(we improved that sometime in the last year or so)
<mark_weaver>amz3: btw, if you don't know about (ice-9 match), you should. it makes for very elegant code, and would be useful for pattern matching on what is 'read'.
<mark_weaver>e.g. you can do things like (match datum (('define (proc . args) docstring . body) (format #t "I found a procedure named ~a~%" proc)) (('define name expr) (format #t "I found a variable named ~a~%" name)))
<mark_weaver>it can be used to match SXML also, although for many purposes sxpath might be better for that.
<mark_weaver>for a real-world example of its use, see the code for Guile's simplifying decompiler:;a=blob;f=module/language/scheme/decompile-tree-il.scm;h=fad857d33dd9a939520da788889b8761199e8c8d;hb=stable-2.0
<mark_weaver>(it's what generates the output of the ,optimize and ,expand REPL commands, among other things)
<mark_weaver>pattern matching via match, together with templating by quasiquotation, is a nice combination.
***tolk`` is now known as tolk
<mark_weaver>shanecelis: I've come up with a more efficient and cleaner 'make-trampoline' for you: (define (make-trampoline module name) (let ((var (module-variable module name))) (lambda () ((variable-ref var)))))
<mark_weaver>this uses vastly less memory per trampoline.
<mark_weaver>sneek: seen ijp
<sneek>ijp was here Aug 14 at 10:25 pm UTC, saying: unless they've been fiddling with it again while I haven't been paying attention.
<mark_weaver>sneek: seen cky
<sneek>cky was here Jul 14 at 03:02 am UTC, saying: cluck: You see, my car plate is CALL/CC, and if I'm going to upgrade it with call/cc's successor, it needs to be just as instantly recognisable as call/cc. ;-).
<mark_weaver>sneek: seen civodlu
<sneek>I last saw civodul on Jul 19 at 12:16 pm UTC, saying: though QuasiConf looks really nice too :-/.
<mark_weaver>wow, sneek corrected the spelling. nicely done.
<mark_weaver>sneek: botsnack
<mark_weaver>sneek: later tell shanecelis: here's an implementation of 'make-trampoline' that uses vastly less memory per trampoline, and avoids loading the compiler: (define (make-trampoline module name) (let ((var (module-variable module name))) (lambda () ((variable-ref var)))))
<sneek>Got it.
<mark_weaver>good morning wingo!
<wingo>good morning :)
<wingo>how goes?
<wingo>did contification break anything? :)
<mark_weaver>yes. have you been receiving my emails?
<wingo>i haven't been reading email
<mark_weaver>ah :)
*wingo looks
<mark_weaver>The relevant one for the moment is "Problem with the new dfg code"
<wingo>ah, yes
<wingo>i tried to tighten up some pattern matching, but it seems I borked it
<mark_weaver>I have a newfound appreciation for the importance of regression testing :)
<wingo>could be it was the old dfg code that was borked, and we just didn't know
<mark_weaver>well, before this contification commit, my continued fractions code seemed to compile and work correctly.
<wingo>yes of course :) but it could be that there was a bug in the dfg from before that was simply ignored before
<wingo>anyway, i'm on it
<mark_weaver>many thanks :)
<mark_weaver>I'm tempted to add the code to the test suite, except that it's not really ready for publication.
<mark_weaver>if I sent you an easy-to-run script that tested the 1000 digits of pi thing, would you use it? :)
<wingo>sure :)
<mark_weaver>excellent! I'm on it...
<wingo>fixed the dfg thing, i think
<mark_weaver>alas, the continued fractions code still fails to compile.
<mark_weaver>I've got the test script ready.
<wingo>woo, contified a recursive fn
*youlysses wonders how hard it would be to port guile-sdl over to 2.x.
<youlysses>2.x is on my todo-list, for guix packaging.
<add^_>youlysses: guile 2.x or SDL 2.x ?
<youlysses>add^_: The latter.
<add^_>ah ok
<add^_>Someone on #lispgames said it'd probably be easier to make that ground up, because of loads of changes.. But I don't know for sure.
<youlysses>While it looks like it'd be a pain at this-point, I suppose it's better than making significant gains supporting via the 1.2.x branch while more-and-more projects are porting over to 2.x. :^P
<youlysses>Oh, well -- ultimately I suppose it's on Wingo and/or anyone willing to do the work. :^P
<wingo>mark_weaver: heya :)
<wingo>i realize i broke your thing a bit more, will poke at fixing it
<wingo>heya dsmith
<dsmith>wingo, How goes cps?
<wingo>dsmith: good!
<wingo>i have contification mostly working
<wingo>we got as far as getting one of mark_weaver's larger programs working, but then i broke it
<wingo>but it's looking good
<wingo>with contification we can hope to be faster than the old vm, but we'll see ;)
<wingo>lots of bugs to fix tho
<sjamaan>Hi all!
<sjamaan>I'm having a look at the R7RS spec, and adding some test cases from it
<sjamaan>Are these truncate/, floor/ etc procedures originally from Guile?
<wingo>heya sjamaan :)
<sjamaan>hi wingo :)
<wingo>i believe those procs are from Riastradh
<sjamaan>Guile has a few which Riastradh doesn't list (the "centered" ones), so I wasn't sure
<wingo>yes, we argued to include that one in r7rs but it didn't make it
<wingo>i think there is a wiki page on the r7rs trac that might have more info, dunno
<sjamaan>Thanks for the info
<wingo>np, happy hacking :)
*sjamaan now needs to decide whether to support all of them in "numbers" or not..
<wingo>combinatorics :/
<sjamaan>This stuff is confusing :/
<sjamaan>Especially because the "native" bignum ops support only one type and the others are derived from those in my implementation
<sjamaan>It's kind of dumb :)
<wingo>one day it would be nice to do all non-fixnum, non-flonum arithmetic with inline caches
<add^_>I got the book "The Lambda Calculus, its syntax and semantics" But I still need to read up on more math to be able to read it more.. efficiently.
<wingo>mark_weaver: i think i fixed it now
<sjamaan>add^_: IIRC there's a good Lambda Calculus introduction by Barendregt
<add^_>sjamaan: That book is made by him.
<sjamaan>This is the intro I was referring to:
<sjamaan>ISTR it was pretty gentle
<add^_>I've read that one
<sjamaan>Excellent :)
<add^_>It's good
<add^_>But I wanted a more in-depth work on lambda calculus, so I bought that book, knowing it could be over my head (hopefully only for a while)..
<add^_>Oh well, what I can understand is cool though
<add^_>But most of it were covered by that paper xD
<add^_>I'm having trouble coming up with a hobby project which isn't *too* hard for me to complete. I usually don't complete my projects because they get out of hand complex and huge..
*sjamaan knows that problem too well :(
<add^_>for all the fish
*add^_ looks at sjamaan's parting message and smiles.
<mark_weaver>wingo: the recursive contification is great! I tried compiling some loops and recursions and the code is much nicer! :)
<mark_weaver>the main problem I see is all the redundant moves before calls.
<wingo>mark_weaver: yeah! i think with that pass we have a hope of being faster
<wingo>better slot allocation is needed but tricky
<wingo>and there are some cleanups needed (e.g. contification results in a return-values always, not return)
<mark_weaver>as for breaking things, I feel bad for bugging you about that. I think I came off differently than I meant to. All the work you're doing is very exciting, and I hope you know how much I appreciate it :)
<mark_weaver>one other area I noticed that could be improved has to do with 'find-constant-value'. It could trace back through multiple-value continuations, but last I checked it didn't.
<wingo>but really, if it can trace back, that means there is one predecessor, so the MV continuation could be turned into a number of single-valued continuations
<wingo>not sure what the right thing is there
<mark_weaver>atm, all of the half-assed benchmarks I've tried are slower than the stack VM, but I think that's probably because of the redundant moves.
<wingo>i doubt it's that, i suspect it is because of the higher cost for subr calls
<wingo>because it recurses into the old vm
<mark_weaver>well, these benchmarks have not gone into the old vm.
<wingo>they don't call subrs?
<mark_weaver>recursive fibonacci, for example.
<wingo>that is one that stack vms do very well at
<wingo>would be nice to beat the stack vm on that one :)
<mark_weaver>(fib 40): 17.8 seconds in the stack VM, 25.7 in the RTL.
<mark_weaver>though that might also be affected by the fast ASM math in the stack VM that's not in RTL.
<wingo>would be nice to give that one some closure optimization...
<mark_weaver>so it's hard to say. still, for such an early stage, I'm quite pleased with how it works and performs.
<mark_weaver>anyway, there are two non-debugging things I'd like to do soon: (1) get fixing-letrec-reloaded cleaned up and pushed, and (2) working on the arithmetic support in the RTL VM. (add1/sub1 -> add-immediate; more instructions)
<mark_weaver>does that sound reasonable?
<mark_weaver>I guess there are more important things to do, but I get the impression you're more qualified to do the big items.
<wingo>sounds great to me
<wingo>not sure about the "qualified" comment :P
<wingo>but anyway, i think i have to put this hack down for the next month or so
<wingo>i'll probably be able to poke at things but it is a packed week at work, i have a family visit, and a presentation to prep for, that i haven't done any of the software work on (!)
<wingo>on the debugging side obviously the unparse-cps stuff is a total mess
<wingo>it's also going to be important to get source info into the elf fils
<wingo>should the muse strike you, that would be a really helpful addition
<mark_weaver>*nod* thanks so much for doing all this! I expected to wait longer for contification. It was like an early holiday present to get it so soon :)
<wingo>also as you probably saw, the contification pass could do with some SCC love ;)
<mark_weaver>It's probably because I'm rusty, but I'm blanking on what SCC is.
<mark_weaver>source code cleanup?
<wingo>strongly connected components
<mark_weaver>I haven't looked at the contification code yet in any depth. I still need to read that paper.
<mark_weaver>ah. well, fixing-letrec-reloaded also needs SCC, so it'll be there. my code, while not the most elegant, performs very well.
<mark_weaver>(which was probably overkill for fixing-letrec, but maybe more important for contification)
<mark_weaver>wingo: briefly, what's involved in getting the source info into the elf files? does that involve dwarf?
<wingo>i think it involves dwarf, yes
<wingo>so there are a few things
<wingo>one is adding an interface to the assembler to allow you to put in source annotations
<wingo>(source SRC) would do
<wingo>you would emit that like a label
<wingo>then the assembler should collect all those in the "meta" for the current function
<wingo>and somehow pack them into a .debug_lines section
<wingo>wip-rtl-dwarf has a dwarf parser
<mark_weaver>okay, sounds reasonable.
<wingo>there is no dwarf serializer as of yet
<wingo>i guess it's a little more involved than that if we wanted to be more properly DWARF -- we'd have to define minimal debugging information entries (DIEs) in a separate section
<mark_weaver>dwarf sounds very nice. I never looked at until one of your blog posts.
<wingo>probably you don't want to page it all in though :)
<wingo>mentally i mean
<wingo>so the DIEs would be the link from functions to debugging info
<mark_weaver>yeah, there would be a cost there. otoh, it would mean one more part of guile that I'm familiar with.
<wingo>the functions are sorted by address in the ELF image in the DIE entries
<wingo>if you do that, i recommend starting with the DWARF 2.0 spec -- it's much simpler than the rest, and as compatible
<mark_weaver>I definitely consider good error messages, backtracing, and debugging to be a high priority. I feel that we aren't very good in that regard at present.
<mark_weaver>(we're not bad either, but imo improvement is needed)
<mark_weaver>toward that end, I'd also like to implement the MIT scheme chain-of-rings thing, and source information for immediates.
<mark_weaver>wingo: okay, thanks for the pointers. I'm not sure if I'll work on that or not, but if I do, these hints will be very helpful.
<wingo>yes that also sounds like great stuff to work on!
<wingo>of course getting rtl to compile all of guile would also be a nice thing :)
<wingo>so many things!
<mark_weaver>I find that to be a good thing to work on for me, yes. There some drudgery to the debugging, but somebody has to do it, and your time is probably better spent on the big items. and I learn about the code as I go, so that's good too.
<mark_weaver>part of the reason why I was motivated to get non-trivial code working is that I know from experience that in big projects like this, it can be hard to get past the initial hump. once things are working well enough to be somewhat useful, it gets much easier.
<mark_weaver>I guess it's a motivational thing more than anything else.
<mark_weaver>wingo: well, good luck getting caught back up on your work/presentation/etc. not to mention time with your partner, etc :)
<ArneBab>mark_weaver: the wip-cps-bis works - cool!
<mark_weaver>ArneBab: did you try it?
<ArneBab>mark_weaver: the test is running right now
<mark_weaver>(not that I recommend doing so)
<ijp>so, I finally have an uptodate guile, after X months
<mark_weaver>well, that's not using the RTL VM.
<mark_weaver>ArneBab: even on that branch, the stack VM is still used by default.
<ArneBab>mark_weaver: base run: (define (forlooptest stop)(let loop ((i 1) (x 42))(unless (>= i stop)(loop (+ i 1) (/ i 2)))))(forlooptest 100000000)
<mark_weaver>ArneBab: it's not ready for benchmarks. the new compiler is missing several important optimization passes.
<ArneBab>mark_weaver: RTL run: (use-modules (system vm assembler) (system base compile) (language cps))(define* (my-compile x #:key (env (current-module))) ((assemble-program (compile x #:env env #:to 'rtl)))) (my-compile '(define (forlooptest stop)(let loop ((i 1) (x 42))(unless (>= i stop)(loop (+ i 1) (/ i 2))))))
<ArneBab> (forlooptest 100000000)
<mark_weaver>ArneBab: and the new compiler isn't used unless you manually pass code through it.
<mark_weaver>ah, okay.
<ArneBab>I read my backlog :)
<mark_weaver>me too :)
<ArneBab>(I meant the past-tense: that’s how I found the code :) )
<ArneBab>(english is terribly unclear on that :) )
<mark_weaver>ah, right
<mark_weaver>a lot of code won't compile yet. it's still very much a work in progress.
<ArneBab>I don’t always read the backlogs in all channels - normally only when I expect to find something I need or something I would have to react to.
<ArneBab>mark_weaver: does it actually use the GCC as backend?
<mark_weaver>it doesn't generate native code. that's still a ways off.. but this is an important step toward that goal.
<ArneBab>what does it use? a self-written compiler? → do you have a pointer?
<mark_weaver>it compiles to a VM, just like before. but it's a totally new VM; one based on registers, not a stack.
<mark_weaver>what do you mean by "self-written compiler" ?
<mark_weaver>Guile 2.0 has a compiler written in Scheme, which compiles Scheme into VM bytecode.
<mark_weaver>most of the compiler is in module/language/tree-il
<mark_weaver>but there are bits in module/language/
<mark_weaver>the new parts of the compiler are in module/language/cps
<mark_weaver>ArneBab: if you want to learn more about this, the "Guile Implementation" chapter of the manual is a start.
<mark_weaver>(it's about 2.0, but 2.2 still uses many of the same concepts)
<mark_weaver>anyway, I have to go afk for a while. ttyl!