IRC channel logs

2014-01-14.log

back to list of logs

<davexunit>hello again guilers.
<davexunit>a-ha! found the issue with the REPL. geiser is using a metacommand with C-x C-e
<davexunit>and that bypasses the stuff I've written.
<davexunit>now how to work around this...
***sneek_ is now known as sneek
<nalaginrut>morning guilers~
***linas_ is now known as linas
***sneek_ is now known as sneek
<civodul>Hello Guilers!
<ArneBab>hi civodul
<mark_weaver>hi civodul!
<mark_weaver>civodul: btw, I saw your comments about CBIPs being broken because of the internal buffering, and I must disagree.
<mark_weaver>I don't see anything in the R6RS that would prohibit the buffering, and they would be painfully inefficient if not for the internal buffering.
<mark_weaver>I don't think using a CBIP as a wrapper to compute hashes of exactly what you read is a viable strategy.
<civodul>i'll write more about that tonight
<civodul>but i don't think implementations can buffer things in practice
<civodul>otherwise 'get-position' cannot work
<civodul>anyway, how would you compute hashes of what gets read?
<mark_weaver>'get-position' can take into account the buffer position.
<mark_weaver>if you look at the ports code, its efficiency depends crucially on the fact that the fast path (when there's enough data in the buffer) doesn't depend on any dispatches on the port type, or calls to function pointers, or anything like that.
<civodul>i know
<mark_weaver>what is this for?
<civodul>for the offload thing
<civodul>at one point it reads an archive from a port, and at the end it must make sure its hash matches the expected hash
<civodul>where its hash = hash of everything deserialized up to this point
*civodul goes grab coffee :-)
<mark_weaver>so why can't it compute the hash as it reads, instead of expecting the port to do it behind the scenes?
<mark_weaver>or it could write it to a file and then compute the hash of the file.
<civodul>heheh
<civodul>not particularly elegant ;-)
<civodul>the problem is that there are several APIs that read from the port
<civodul>and they're not "hash aware" so to speak
<civodul>it feels natural to wrap the port as one would do in an OO fashion
<mark_weaver>well, I can sympathize with your desire for an elegant solution, but I don't see a way to do it without killing the efficiency of ports.
<civodul>perhaps you forget the scm_c_read efficiency hack :-)
<civodul>if you actually do binary I/O on the CBIP, then you really read by app-level chunks
<civodul>so my guess is that performance wouldn't be too different in practice, but that has to be checked, of course
<civodul>(besides, when doing this kind of wrapping, one is already on the slow path)
<mark_weaver>okay, but that's just one API. if we were to add hooks to the single-char read function (which is inlined), it would be terrible.
<mark_weaver>take a look at scm_get_byte_or_eof in inline.h, for example.
<civodul>yes, clearly
<civodul>but here this is the binary port
<civodul>so in practice it is accessed with scm_c_read
<civodul>the problem you mention shows in Guile's historical "soft ports"
<civodul>which are necessarily painfully slow
<mark_weaver>well, you could put a wrapper around scm_c_read, but then what would you do with scm_get_byte_or_eof?
<civodul>well if you do textual I/O on the CBIP, then read! is called for every single byte
<civodul>i think i'll post something for discussion later today
<civodul>i need to do pay-the-bill work now :-/
<mark_weaver>okay
*dje42 can relate
<ArneBab>I wish, more free software hackers had their hacking as pay-the-bill work…
<ArneBab>(we’d need more people to do regular donations…)
<jemarch>hi
<lloda`>hey wingo
<wingo>morning lloda`! happy new year!
<wingo>and... god i hate doing this again, but apologies
<wingo>i had much less time than expected to do guile things in december :/
<wingo>terrible
<wingo>i plan on reviewing and merging your stuff this week; fingers crossed, and i don't want to make a fool out of myself again by promising, but hey, maybe this is the time...
<lloda`>i just want to have it done so I can attack bytevectors.c :-/
<lloda`>let me know if I can help in any way
<wingo>yeah understood
<wingo>ok will do
<lloda`>... and happy new year to you too! :-)
<mark_weaver>civodul: I'm embarrassed to admit this, but I have no idea what I was talking about earlier, regarding CBIPs. We just need to make sure the buffer is always empty, which we can do if you switch the port to unbuffered mode. Does that not work?
<mark_weaver>(well, to be more precise: we need to ensure that the buffer is always empty after any read)
<mark_weaver>ah, looks like CBIPs make some bad assumptions about the buffer. I guess you were right all along. sorry for the noise.
<mark_weaver>ugh, setvbuf only works on file ports.
<wingo>yay ports
<civodul>mark_weaver: exactly: that's what i wanted to do, make the port unbuffered
<mark_weaver>ah, even R6RS has no API for setting the buffering mode for non-file ports.
<civodul>that way, scm_c_read will pass the actual user buffer
<mark_weaver>that's such a drag.
<civodul>i'll look into it tonight, hopefully
<civodul>yeah
<civodul>that port API is terrible
<wingo>just make setvbuf do the right thing i guess?
<civodul>for CBIPs that'd be insufficient
<mark_weaver>our current implementation of CBIPs assumes that the input buffer will never be changed.
<wingo>ah. humm :)
*wingo ignorant, shuts up
<mark_weaver>but that could be fixed, of course.
<mark_weaver>civodul: what would be insufficient?
<mark_weaver>I feel very torn about this, because in many cases it's very helpful to have buffering. If you make a CBIP and do a lot of get-u8's on it, then it would be vastly faster if buffered.
<mark_weaver>but sometimes you want unbuffered. it's a shame that there's no portable API to set the buffering mode on the custom ports.
<mark_weaver>I tend to agree with wingo that we should make 'setvbuf' work on CBIPs.
<mark_weaver>and then there's the question of what the default buffering mode should be on CBIPs.
<mark_weaver>I guess the default should probably be unbuffered.
<mark_weaver>dje42: did you use R6RS custom binary ports in your gdb/guile work?
<civodul>mark_weaver: wait wait, we'll discuss tonight :-)
<civodul>and i'll write everything in a message
<mark_weaver>okay
<civodul>IRC doesn't work so well here ;-)
*civodul goes home
<mark_weaver>wingo: your "$(GOBJECTS): ice-9/psyntax-pp.go" rule to optimize the build in master has the unfortunate side effect of forcing a full rebuild of everything whenever psyntax changes. There must be a better way, no?
<wingo>mark_weaver: dunno!
<wingo>a number of gross possibilities come to mind
<wingo>fwiw the same thing has always been the case with eval.scm
<mark_weaver>ah, is that for the same reason?
<mark_weaver>well, I guess it's not so bad to temporarily comment out that rule when I work on psyntax.
<mark_weaver>relatively few people hack on eval.scm or psyntax.scm, so it's probably not worth worrying about.
<wingo>that was my thought
<dje42>mark_weaver: rs: r6rs custom binary ports. No, I didn't. At least not for the current two instances: memory and gdb stdio.
<mark_weaver>dje42: okay, thanks.
<dje42>fwiw, I've played with the port code a fair bit (e.g. I found that setvbuf only works on file ports too :-)).
<dje42>I think some cleanup could be useful.
<mark_weaver>agreed
<davexunit>mark_weaver: I found the issue with my REPL last night. geiser was sending a metacommand called ,geiser-eval and the REPL processes metacommands differently than other input.
<mark_weaver>makes sense. actually, I suspect that the newer version of the REPL stuff you did (where the reading is done in a different thread) would not have any problem with that.
<davexunit>yeah that would work.
<mark_weaver>davexunit: speaking of the REPL, do you remember my proposal that we add Guile support for running a REPL within an event-loop-style main thread?
<davexunit>yes
<mark_weaver>would you like to work on a patch that does that for stable-2.0?
<davexunit>sure.
<davexunit>did you have any rough ideas for how to implement it?
<mark_weaver>I think the API should be roughly as follows: one procedure to create a new REPL, with the reading done in another thread, and another procedure that you call periodically from the main thread to run any pending commands.
<bu^>hi, how long is guile git compilation time ?
<bu^>its already been 8 hours :)
<davexunit>bu^: it takes a while.
<davexunit>8 hours is far longer than I've ever heard of.
<mark_weaver>bu^: what branch are you on?
<bu^>master
<mark_weaver>I think master bootstrap time has gotten notably slower recently.
<bu^>might have messed things by passing make -j8 ?
<mark_weaver>we'll probably need to take a look at that.
<bu^>what takes time is building the .go files
<mark_weaver>you might be happier with stable-2.0 for now.
<bu^>otherwise its ok
<bu^>I use 2.0
<bu^>but wanted to look for 2.2
<mark_weaver>well, it's the bootstrap in particular. compiling the files before the compiler itself has been compiled.
<mark_weaver>so what's happening is that you're using a (rather slow) evaluator to interpret the compiler, while it compiles itself.
<mark_weaver>after the most important components of the compiler have been compiled, it's faster.
<mark_weaver>davexunit: you'd do it similar to what you're doing now, but separated from the coroutines stuff that you built.
<mark_weaver>so it would just use partial continuations directly.
<davexunit>mark_weaver: okay. I'll have to read over the code I wrote again since I haven't actually used it in some time.
<bu^>ok, no way to skip this part ? :) I'll see tomorrow morning if it finished
<mark_weaver>if you find it too difficult, I can do it. I was just hoping to delegate some of my TODO list.
<davexunit>hopefully I can avoid duplicating too much code from (system repl server)
<davexunit>mark_weaver: I'll at least give it a good try before I decide that it's too difficult.
<mark_weaver>bu^: well, the good news is that once you've built from git, you can do later "git pull" and recompile only what changed, which is much faster.
<davexunit>I already have a rough draft to reference.
<mark_weaver>davexunit: you shouldn't duplicate much code. generalize the existing code as needed.
<davexunit>roger.
<davexunit>I won't be able to hack tonight but hopefully tomorrow I can start. :)
<mark_weaver>no rush. I'm still hoping to get R7RS into 2.0.10, which will take some time.
<davexunit>okay.
<wingo>bu^: what libgc are you using?
<wingo>with git libgc i have to compile with GC_MARKERS=1
<wingo>otherwise the parallel marking totally gets stuck
<wingo>haven't tried to figure that out yet
<bu^>wingo, hum, I'm using "default" gc 7.4.0
<bu^>was R7RS published ?
<wingo>they released 7.4!
<wingo>i didn't know that
<wingo>well that would probably include the issue
<bu^>just updated my distro :)
<bu^>sorry for that ^^
<wingo>bu^: i suggest running with GC_MARKERS=1
<wingo>bu^: what distro, for the record?
<bu^>Ok, i'll note it
<bu^>gentoo
<wingo>i guess we should hack around this in guile until we figure out the issue
<mark_weaver>bu^: R7RS-small was published, yes.
<mark_weaver>ratified, I mean.
<bu^>not on http://scheme-reports.org/ yet
<bu^>hum, ok it's been a long time...
<bu^>they forgot to update their website
<wingo>well, the end of the r7rs process was a bit strange
<wingo>a kind of trailing off rather than a full stop
<mark_weaver>there's also gc-7.2e now, which is what they're currently recommending as "the most stable version".
<mark_weaver> http://www.hpl.hp.com/personal/Hans_Boehm/gc/ doesn't yet mention 7.4
<davexunit>mark_weaver: so my prototype "cooperative" REPL uses your mvars module, which saves me some work, but I won't have access to that when I write a patch for stable-2.0.
<davexunit>I guess I'll just write a few procedures that implement the mvar features that I used.
<mark_weaver>davexunit: you can build your patch on top of my mvars patch. I'm pretty sure something like mvars will make it in.
<mark_weaver>we can fix it up later.
<davexunit>mark_weaver: okay. sounds good. thanks.
<bu^>inkscape is also broken because of gc api changes
<bu^> http://www.hpl.hp.com/personal/Hans_Boehm/gc/gc_source/?C=M;O=D
<mark_weaver>bu^: fwiw, I just built master from clean in about 105 minutes on a Core 2 Duo.
<mark_weaver>I didn't pass -j to make, and that's using libgc-7.2d
<bu^>ok...
<bu^>gonna look for this tomorrow
<bu^>the computer is an 8 cores xeons...
<bu^>ah yes ! I remember now
<bu^>gc had issues wingo was right
<bu^>with "mark"
<bu^>I'll look for the proper message
<bu^>GC_printf1("Failed to grow mark stack to %lu frames\\n"
<bu^>when building the .go files
<dsmith-work>mark_weaver: Are you having any problems (still) with make -j ?
<dsmith-work>mark_weaver: On master
<wingo>i haven't seen make -j issues recently, and did fix some bugs that could have caused them; though it's hard to tell
<wingo>the only compilation issue i see is the GC_MARKERS=1 issue with libgc>=7.3
<mark_weaver>dsmith-work: I haven't tried recently.
<dsmith-work>mark_weaver: Just curious because of "<mark_weaver> I didn't pass -j to make,"
<mark_weaver>to be honest, I simply forgot to :)
<dsmith-work>Been working fine for me since wingo did those fixes
<mark_weaver>nice!
<mark_weaver>hmm, I wonder what to name the read/print options for supporting the R7RS |...| symbol syntax.
<mark_weaver>currently, I've named it 'r7rs-symbols', but I'm not sure that's the best.
<mark_weaver>thoughts, anyone?
<dsmith-work>pipe-escapes, R7RS-symbol-escapes
<mark_weaver>or maybe 'pipe-symbols'
<mark_weaver>xb
<dsmith-work>Yeah, they are not really escapes,
<dsmith-work>mark_weaver: Would they be used instead of or in addition to old style #{ }# symbol?
<dsmith-work>s
<dsmith-work>Doesn't CL use those too?
<wingo>r7rs-symbols, pipe-symbols, whatever sounds fine to me
<mark_weaver>dsmith-work: yeah, I think so.
<wingo>our current #{} thing is understandable but silly
<mark_weaver>I think perhaps we should switch the read/print default options to use pipe symbols by default (when escapes are needed).
<mark_weaver>in 2.2, that is.
<mark_weaver>WDYT?
<mark_weaver>dsmith-work: we'll continue to support the #{ }# syntax, of course.
<wingo>mark_weaver: agreed, i think
<wingo>the only loss for me is treating | as a lone symbol, but that was never scheme-portable, so oh well
<wingo>*lone | as a symbol
<mark_weaver>sounds good
<dsmith-work>Hmm. What I meant was, with he pipe-symbols reader option enabled, does that disable #{}# symbols? Can they coexist?
<mark_weaver>dsmith-work: no, the option doesn't disable #{}# symbols. they can coexist.
<dsmith-work>Cool.
<mark_weaver>the only thing the option disables is bare symbols containing |
<mark_weaver>actually, not even that much. just bare symbols that _start_ with |
<taylanub>mark_weaver: Should (foo| bar| baz) not read `foo', ` bar', and ` baz' ?
<taylanub>(Spaces before bar and baz.)
<mark_weaver>hmm. well, maybe it should return 'foo', ' bar', and 'baz'. (space before bar)
<taylanub>Oh right, that's what I .. meant to mean.
<mark_weaver>yeah, good point. in R7RS, '|' is a delimiter.
***linas_ is now known as linas
<mark_weaver>I'll fix it.
<mark_weaver>dsmith-work: so yeah, the option will disable '|' anywhere in bare symbols.
<mark_weaver>which means you either have to use a hex escape or write it as |\\||
*dsmith-work looks for r7rs
*taylanub looks forward to the day Scheme will drop the symbol type and just define them as implicitly-interned immutable strings ...
<taylanub>If I'm not mistaken, they already are that, except that you can't use operations on them that you could otherwise use on immutable strings.
*davexunit wishes guile's strings were immutable.
<mark_weaver>well, for one thing, strings and symbols are guaranteed to be different types.
<mark_weaver>lots of code implicitly assumes that.
<mark_weaver>R6RS uses | in its numerical syntax, to specify a precision in bits.
<taylanub>Ugh, that's a nasty R6RS/R7RS clash.
<mark_weaver>when we support that syntax, it's going to be a slight pain, because right now when we read a number we stop at any delimiter.
<mark_weaver>and that's kind of important, given that if converting to a number fails for any reason, then it's interpreted as a symbol.
<mark_weaver>so we'll have to do something nasty like read as far as we can, including pipe symbols, and then try that as a number. if it fails, then unget everything from the first pipe and then treat that as a symbol.
<mark_weaver>or, we could just diverge from R7RS on this point, and make | not be a delimiter.
<mark_weaver>so (foo| bar| baz) would read as 'foo|', 'bar|', and 'baz'.
<taylanub>I think I'd agree with that, sane code shouldn't use | as the sole delimiter anywhere anyway IMO.
<mark_weaver>yeah, I'm leaning toward making '|' not be a delimiter at all.
<mark_weaver>the nasty hack I described would lead to things like 10|blah| being read as two symbols: |10| and blah.
<mark_weaver>hmm, well, I guess it could be made to work with an even nastier hack, where you first try parsing the whole thing including pipes as a number, and if that fails, try everything before the first pipe as a number, etc.
<mark_weaver>whether to make pipe a delimiter could be a separate read option, I suppose.
<ArneBab>mark_weaver: is there a way to tell the R7RS writers about the problem?
<ArneBab>(I’d call that a bug-report…)
<mark_weaver>it's too late
<taylanub>ArneBab: Do you mean the big R6RS clash ?
<mark_weaver>anyway, they don't consider clashes with R6RS to be bugs.
<taylanub>The atmosphere I've gotten was that big clashes were avoided as much as possible .. in fact, does Chibi not somehow implement both R6 and 7RS at the same time ? Maybe we could check how that deals with this ?
<mark_weaver>the mailing list is scheme-reports@scheme-reports.org though.
<mark_weaver>I don't think Chibi supports R6RS.
<ArneBab>taylanub: yes
<taylanub>mark_weaver: It might have been Sagittarius instead. There was some "R6RS/R7RS compatibility proof of concept" implementation, I think by jcowan.
<taylanub> https://code.google.com/p/sagittarius-scheme/
<taylanub>Hrm, maybe that was just about the library syntax after all. :\\
<taylanub>I'll ask jcowan if I catch them, to be sure I'm not missing anything.
<mark_weaver>thanks!
<taylanub>np
<ArneBab>mark_weaver: no bugs… :(
<mark_weaver>ArneBab: huh?
<ArneBab>I mean it’s sad that they don’t consider clashes with R6RS as bugs
<mark_weaver>yes, it is.
<dsmith-work>They consider them bug *fixes*.