<amz3>can I inspect a module ? <amz3>also if I understand correctly, one *has* to export things to be able to use them from another module <amz3>I think, maybe because in Python it's otherwise, it should be open by default and restricted upon keywords <mark_weaver>it's possible to get a private bindings in a module, using the (@@ (module name) symbol) syntax. <mark_weaver>from the REPL, you can use the ,m command to "enter" a module, so the REPL is within the module. <mark_weaver>(everything you evaluate will be evaluated within that module) <mark_weaver>there's also 'module-map' and 'module-for-each' that will iterate over all the bindings in an arbitrary module. <mark_weaver>I strongly disagree that modules should export everything by default. <mark_weaver>but in normal usage, you need to explicitly export the bindings that should be exported. <mark_weaver>you could make a macro to do that automatically though. <mark_weaver>actually, there already is one: it's called 'define-public' <mark_weaver>just use that instead of 'define' and it'll take care of exporting it. <amz3>so it's good, I just need to recall to use that <amz3>I'm investigating what do to next <amz3>I would like to generate documentation from modules <amz3>yes ^^ amk9 make me think of to dogs too much <amz3>I've been lurking around couchbase, another nosql database, it looks nice <amz3>that resulted from the merge of couchdb and membase <mark_weaver>if you have an idea for a basic approach of how such modules would be written, I might be able to help you accomplish that. <mark_weaver>do you just need to scrape the docstrings out of all the procedures? <mark_weaver>another approach would be to use macros that would contain the doc snippets, and they would expand into nothing when loaded as code, but expand into something else when loaded by the doc generator. <mark_weaver>or the doc generator could use 'read' to read the source file directly, and look for certain known top-level constructs. <amz3>I will use 'module-map' or 'module-for-each' and retrive the docstring <mark_weaver>'read' makes things pretty easy because it handles the parsing entirely. <amz3>yes but I have to extract the infos <amz3>whereas if I load a module I have access to the docstrings directly <mark_weaver>the only thing that makes me worried about the docstrings is that in core guile, we're going to interpret those as texinfo at some point in the future. <mark_weaver>if you build something that assumes they'll be something else, there will be conflict. <mark_weaver>but procedures can have arbitrary meta-data associated with them, and it's possible to make custom 'define' macros that will store additional meta-data in the procedure. <mark_weaver>so you could have your own separate key in the meta-data for the info you need. <mark_weaver>although I tend to think that having a doc generator read the source directly might be best. then you'll be able to put stuff in there that isn't associated with a procedure. <amz3>for all this reasons I'm still thinking about this particular feature <mark_weaver>'read' gives you each top-level form as a nested data structure, as if you put a quote in front of it. <mark_weaver>so it's super easy to just look at the 'car' of each of those things being read. <mark_weaver>the 'car' is the first element in the list, so it would usually be 'define' or 'define-public', or your own custom 'define' macro that includes a slot for your doc snippets. <mark_weaver>'read' is actually what the compiler and interpreter use to parse the file, btw. <amz3>in the previous version of my static blog generator I was using it directly <mark_weaver>you'll probably want a way to add docs for things like structure definitions as well. <mark_weaver>and advanced scheme code tends to make their own custom 'define' constructs as well. <mark_weaver>I think associated the doc snippets with the run-time objects, and then scraping them from the module at run-time, is a lose for what you want. <mark_weaver>but if you really want the runtime approach, guile also has something called "object properties" that let you associate arbitrary meta data about any non-immediate object. <mark_weaver>'read' loses whitespace and comments, and some other details like whether you wrote 'a or (quote a), etc. <mark_weaver>so if you want to preserve the exact source, you'll need to use lower-level I/O (maybe read-line) instead. <mark_weaver>but a hybrid approach is also possible, where you use 'read' in a first pass, and then read the exact text in a second pass. <mark_weaver>you can connect them together using the filename/line/column information that is associated with every non-immediate datum returned by 'read'. <mark_weaver>(note that older versions of guile only recorded source information for lists, but even that would probably be enough for what you need.) <mark_weaver>amz3: btw, if you don't know about (ice-9 match), you should. it makes for very elegant code, and would be useful for pattern matching on what is 'read'. <mark_weaver>e.g. you can do things like (match datum (('define (proc . args) docstring . body) (format #t "I found a procedure named ~a~%" proc)) (('define name expr) (format #t "I found a variable named ~a~%" name))) <mark_weaver>it can be used to match SXML also, although for many purposes sxpath might be better for that. <mark_weaver>(it's what generates the output of the ,optimize and ,expand REPL commands, among other things) <mark_weaver>pattern matching via match, together with templating by quasiquotation, is a nice combination. ***tolk`` is now known as tolk
<mark_weaver>shanecelis: I've come up with a more efficient and cleaner 'make-trampoline' for you: (define (make-trampoline module name) (let ((var (module-variable module name))) (lambda () ((variable-ref var))))) <sneek>ijp was here Aug 14 at 10:25 pm UTC, saying: unless they've been fiddling with it again while I haven't been paying attention. <sneek>cky was here Jul 14 at 03:02 am UTC, saying: cluck: You see, my car plate is CALL/CC, and if I'm going to upgrade it with call/cc's successor, it needs to be just as instantly recognisable as call/cc. ;-). <sneek>I last saw civodul on Jul 19 at 12:16 pm UTC, saying: though QuasiConf looks really nice too :-/. <mark_weaver>sneek: later tell shanecelis: here's an implementation of 'make-trampoline' that uses vastly less memory per trampoline, and avoids loading the compiler: (define (make-trampoline module name) (let ((var (module-variable module name))) (lambda () ((variable-ref var))))) <wingo>did contification break anything? :) <wingo>i haven't been reading email <mark_weaver>The relevant one for the moment is "Problem with the new dfg code" <wingo>i tried to tighten up some pattern matching, but it seems I borked it <mark_weaver>I have a newfound appreciation for the importance of regression testing :) <wingo>could be it was the old dfg code that was borked, and we just didn't know <mark_weaver>well, before this contification commit, my continued fractions code seemed to compile and work correctly. <wingo>yes of course :) but it could be that there was a bug in the dfg from before that was simply ignored before <mark_weaver>I'm tempted to add the code to the test suite, except that it's not really ready for publication. <mark_weaver>if I sent you an easy-to-run script that tested the 1000 digits of pi thing, would you use it? :) <wingo>fixed the dfg thing, i think <mark_weaver>alas, the continued fractions code still fails to compile. <wingo>woo, contified a recursive fn *youlysses wonders how hard it would be to port guile-sdl over to 2.x. <add^_>youlysses: guile 2.x or SDL 2.x ? <add^_>Someone on #lispgames said it'd probably be easier to make that ground up, because of loads of changes.. But I don't know for sure. <youlysses>While it looks like it'd be a pain at this-point, I suppose it's better than making significant gains supporting via the 1.2.x branch while more-and-more projects are porting over to 2.x. :^P <youlysses>Oh, well -- ultimately I suppose it's on Wingo and/or anyone willing to do the work. :^P <wingo>i realize i broke your thing a bit more, will poke at fixing it <wingo>i have contification mostly working <wingo>we got as far as getting one of mark_weaver's larger programs working, but then i broke it <wingo>with contification we can hope to be faster than the old vm, but we'll see ;) <sjamaan>I'm having a look at the R7RS spec, and adding some test cases from it <sjamaan>Are these truncate/, floor/ etc procedures originally from Guile? <wingo>i believe those procs are from Riastradh <sjamaan>Guile has a few which Riastradh doesn't list (the "centered" ones), so I wasn't sure <wingo>yes, we argued to include that one in r7rs but it didn't make it <wingo>i think there is a wiki page on the r7rs trac that might have more info, dunno *sjamaan now needs to decide whether to support all of them in "numbers" or not.. <sjamaan>Especially because the "native" bignum ops support only one type and the others are derived from those in my implementation <wingo>one day it would be nice to do all non-fixnum, non-flonum arithmetic with inline caches <add^_>I got the book "The Lambda Calculus, its syntax and semantics" But I still need to read up on more math to be able to read it more.. efficiently. <wingo>mark_weaver: i think i fixed it now <sjamaan>add^_: IIRC there's a good Lambda Calculus introduction by Barendregt <add^_>sjamaan: That book is made by him. <add^_>But I wanted a more in-depth work on lambda calculus, so I bought that book, knowing it could be over my head (hopefully only for a while).. <add^_>Oh well, what I can understand is cool though <add^_>But most of it were covered by that paper xD <add^_>I'm having trouble coming up with a hobby project which isn't *too* hard for me to complete. I usually don't complete my projects because they get out of hand complex and huge.. *sjamaan knows that problem too well :( *add^_ looks at sjamaan's parting message and smiles. <mark_weaver>wingo: the recursive contification is great! I tried compiling some loops and recursions and the code is much nicer! :) <mark_weaver>the main problem I see is all the redundant moves before calls. <wingo>mark_weaver: yeah! i think with that pass we have a hope of being faster <wingo>better slot allocation is needed but tricky <wingo>and there are some cleanups needed (e.g. contification results in a return-values always, not return) <mark_weaver>as for breaking things, I feel bad for bugging you about that. I think I came off differently than I meant to. All the work you're doing is very exciting, and I hope you know how much I appreciate it :) <mark_weaver>one other area I noticed that could be improved has to do with 'find-constant-value'. It could trace back through multiple-value continuations, but last I checked it didn't. <wingo>but really, if it can trace back, that means there is one predecessor, so the MV continuation could be turned into a number of single-valued continuations <wingo>not sure what the right thing is there <mark_weaver>atm, all of the half-assed benchmarks I've tried are slower than the stack VM, but I think that's probably because of the redundant moves. <wingo>i doubt it's that, i suspect it is because of the higher cost for subr calls <wingo>because it recurses into the old vm <mark_weaver>well, these benchmarks have not gone into the old vm. <wingo>that is one that stack vms do very well at <wingo>would be nice to beat the stack vm on that one :) <mark_weaver>(fib 40): 17.8 seconds in the stack VM, 25.7 in the RTL. <mark_weaver>though that might also be affected by the fast ASM math in the stack VM that's not in RTL. <wingo>would be nice to give that one some closure optimization... <mark_weaver>so it's hard to say. still, for such an early stage, I'm quite pleased with how it works and performs. <mark_weaver>anyway, there are two non-debugging things I'd like to do soon: (1) get fixing-letrec-reloaded cleaned up and pushed, and (2) working on the arithmetic support in the RTL VM. (add1/sub1 -> add-immediate; more instructions) <mark_weaver>I guess there are more important things to do, but I get the impression you're more qualified to do the big items. <wingo>not sure about the "qualified" comment :P <wingo>but anyway, i think i have to put this hack down for the next month or so <wingo>i'll probably be able to poke at things but it is a packed week at work, i have a family visit, and a presentation to prep for, that i haven't done any of the software work on (!) <wingo>on the debugging side obviously the unparse-cps stuff is a total mess <wingo>it's also going to be important to get source info into the elf fils <wingo>should the muse strike you, that would be a really helpful addition <mark_weaver>*nod* thanks so much for doing all this! I expected to wait longer for contification. It was like an early holiday present to get it so soon :) <wingo>also as you probably saw, the contification pass could do with some SCC love ;) <mark_weaver>It's probably because I'm rusty, but I'm blanking on what SCC is. <wingo>strongly connected components <mark_weaver>I haven't looked at the contification code yet in any depth. I still need to read that paper. <mark_weaver>ah. well, fixing-letrec-reloaded also needs SCC, so it'll be there. my code, while not the most elegant, performs very well. <mark_weaver>(which was probably overkill for fixing-letrec, but maybe more important for contification) <mark_weaver>wingo: briefly, what's involved in getting the source info into the elf files? does that involve dwarf? <wingo>i think it involves dwarf, yes <wingo>one is adding an interface to the assembler to allow you to put in source annotations <wingo>you would emit that like a label <wingo>then the assembler should collect all those in the "meta" for the current function <wingo>and somehow pack them into a .debug_lines section <wingo>wip-rtl-dwarf has a dwarf parser <wingo>there is no dwarf serializer as of yet <wingo>i guess it's a little more involved than that if we wanted to be more properly DWARF -- we'd have to define minimal debugging information entries (DIEs) in a separate section <mark_weaver>dwarf sounds very nice. I never looked at until one of your blog posts. <wingo>probably you don't want to page it all in though :) <wingo>so the DIEs would be the link from functions to debugging info <mark_weaver>yeah, there would be a cost there. otoh, it would mean one more part of guile that I'm familiar with. <wingo>the functions are sorted by address in the ELF image in the DIE entries <wingo>if you do that, i recommend starting with the DWARF 2.0 spec -- it's much simpler than the rest, and as compatible <mark_weaver>I definitely consider good error messages, backtracing, and debugging to be a high priority. I feel that we aren't very good in that regard at present. <mark_weaver>(we're not bad either, but imo improvement is needed) <mark_weaver>toward that end, I'd also like to implement the MIT scheme chain-of-rings thing, and source information for immediates. <mark_weaver>wingo: okay, thanks for the pointers. I'm not sure if I'll work on that or not, but if I do, these hints will be very helpful. <wingo>yes that also sounds like great stuff to work on! <wingo>of course getting rtl to compile all of guile would also be a nice thing :) <mark_weaver>I find that to be a good thing to work on for me, yes. There some drudgery to the debugging, but somebody has to do it, and your time is probably better spent on the big items. and I learn about the code as I go, so that's good too. <mark_weaver>part of the reason why I was motivated to get non-trivial code working is that I know from experience that in big projects like this, it can be hard to get past the initial hump. once things are working well enough to be somewhat useful, it gets much easier. <mark_weaver>I guess it's a motivational thing more than anything else. <mark_weaver>wingo: well, good luck getting caught back up on your work/presentation/etc. not to mention time with your partner, etc :) <ArneBab>mark_weaver: the wip-cps-bis works - cool! <ArneBab>mark_weaver: the test is running right now <ijp>so, I finally have an uptodate guile, after X months <mark_weaver>ArneBab: even on that branch, the stack VM is still used by default. <ArneBab>mark_weaver: base run: (define (forlooptest stop)(let loop ((i 1) (x 42))(unless (>= i stop)(loop (+ i 1) (/ i 2)))))(forlooptest 100000000) <mark_weaver>ArneBab: it's not ready for benchmarks. the new compiler is missing several important optimization passes. <ArneBab>mark_weaver: RTL run: (use-modules (system vm assembler) (system base compile) (language cps))(define* (my-compile x #:key (env (current-module))) ((assemble-program (compile x #:env env #:to 'rtl)))) (my-compile '(define (forlooptest stop)(let loop ((i 1) (x 42))(unless (>= i stop)(loop (+ i 1) (/ i 2)))))) <mark_weaver>ArneBab: and the new compiler isn't used unless you manually pass code through it. <ArneBab>(I meant the past-tense: that’s how I found the code :) ) <ArneBab>(english is terribly unclear on that :) ) <mark_weaver>a lot of code won't compile yet. it's still very much a work in progress. <ArneBab>I don’t always read the backlogs in all channels - normally only when I expect to find something I need or something I would have to react to. <ArneBab>mark_weaver: does it actually use the GCC as backend? <mark_weaver>it doesn't generate native code. that's still a ways off.. but this is an important step toward that goal. <ArneBab>what does it use? a self-written compiler? → do you have a pointer? <mark_weaver>it compiles to a VM, just like before. but it's a totally new VM; one based on registers, not a stack. <mark_weaver>Guile 2.0 has a compiler written in Scheme, which compiles Scheme into VM bytecode. <mark_weaver>the new parts of the compiler are in module/language/cps <mark_weaver>ArneBab: if you want to learn more about this, the "Guile Implementation" chapter of the manual is a start. <mark_weaver>(it's about 2.0, but 2.2 still uses many of the same concepts)