IRC channel logs

2014-02-04.log

back to list of logs

***dsmith_ is now known as dsmith
<dsmith>sneek, later tell wingo Yeah, debugging can be relaxing, if you don't have a deadline. It's like puzzle solving. Like playing an adventure game.
<mark_weaver>I must say, I rarely find debugging relaxing.
<mark_weaver>but I'm glad some people like it :)
<mark_weaver>hi zacts!
<mark_weaver>can you tell me which tests failed on freebsd?
<mark_weaver>davexunit: do you think you'll be able to update the coop-repl-servers patch to avoid mvars soon, or should I do it?
<zacts>hi
<mark_weaver>hi zacts
<zacts>mark_weaver: yeah, give me at least 30 min to go to my cafe and get started with my evening hacking. iirc, I first need to improve the devel/boehm-gc port before I can do a new guile-2.0 port.
<mark_weaver>okay, sounds good. no rush :)
<mark_weaver>fwiw, there's a gc-7.2e now. is that the version in the freebsd ports system now?
<mark_weaver>(there's also 7.4, but that might be too cutting edge, dunno :)
<davexunit>mark_weaver: how soon do you need it done by? end of the week?
<mark_weaver>that should be fine, yes.
<davexunit>I should be able to do that, then.
<zacts>mark_weaver: 7.2e is what is in FreeBSD ports
<mark_weaver>I'd like to aim to release 2.0.10 early next week, hopefully.
<mark_weaver>zacts: perfect!
<zacts>but the port needs to be vastly improved
<zacts>mark_weaver: oh cool about the 2.0.10 release, that should make things easier to port also.
<zacts>if it will be that soon, I may wait for 2.0.10. and I could focus my time on helping 2.0.10, and making sure it ports well to ports.
<mark_weaver>zacts: sounds good.
<mark_weaver>thanks :)
<zacts>brb to the cafe
<zacts>mark_weaver: have you fixed the #!/randomnixstuff/bin/sh lines in the git repo yet?
*zacts is checking out the repo now
<zacts>s/checking out/cloning/
<mark_weaver>that problem was never in the repo, or in guile.
<mark_weaver>it was only in the 2.0.9 tarball, created when a flawed 'autoconf' and 'automake' were run to create the tarball.
<mark_weaver>the problem was in the automake or autoconf package in Guix, and it's been long fixed.
<mark_weaver>(I don't remember which of those two packages it was)
<mark_weaver>zacts: you'll want the stable-2.0 branch.
<mark_weaver>zacts: also, in a few hours I'll push a big gnulib update to stable-2.0, which might have some relevance for freebsd.
<zacts>ok, I'm on stable-2.0 now
<zacts>oh, cool
<mark_weaver>testing after that update will be more useful than testing before.
<zacts>I'll wait for that
<zacts>in the meantime I think I'll work on this boehm-gc
<mark_weaver>also, after the gnulib update, it might be better for you to grab a tarball from http://hydra.nixos.org/job/gnu/guile-2-0/tarball/latest
<zacts>ah ok
<mark_weaver>since that will be closer to the 2.0.10 tarball than you'll get from using your own (possibly old) copies of 'automake', 'autoconf', and 'libtool' to bootstrap.
<mark_weaver>I'll let you know when a new tarball is up on hydra with the gnulib update.
<zacts>sweet
<zacts>mmm.. I just got a free scone at my cafe
<zacts>^_^
<mark_weaver>oh? how'd you manage that? :)
<zacts>they are walking around giving free samples
<zacts>they must have had extra, and so it's night now. so they are giving the extras away
<mark_weaver>nice!@
<zacts>mark_weaver: btw, mind if I /msg you my email?
<zacts>in case my irc client dies..
<mark_weaver>sure!
<zacts>cool.
<zacts>it's just with all of this freenode DDoS, you know..
<mark_weaver>indeed
<mark_weaver>zacts: fyi, I've pushed the gnulib update to stable-2.0. now it's just a matter of waiting until hydra builds a new tarball.
<zacts>oh, cool thanks!
<zacts>mark_weaver: http://hydra.nixos.org/build/8709831
<zacts>is that the build you were doing?
<mark_weaver>no, that one doesn't include the gnulib update.
<mark_weaver>it should be available by tomorrow at <http://hydra.nixos.org/job/gnu/guile-2-0/tarball/all>. You're looking for a release name that's guile-2.0.9.182-* or newer
<mark_weaver>anything after that will be an excellent test on freebsd. I don't expect any changes between .182 and the release that would affect portability on freebsd.
<zacts>ah ok. thanks!
<mark_weaver>how goes the boehm-gc work?
<zacts>boehm-gc works and builds on FreeBSD (as it always has), but I'm just working with a port maintainer on cleaning up the Makefile and possibly creating a new devel/boehm-gc-threaded-pthreads port or build option
<zacts>I guess guile seems to need the boehm-gc --with-threads=pthreads rather than --with-threads=posix options
<mark_weaver>ah, interesting. what's the difference between those two?
<zacts>I'm not really sure yet. I can't remember exactly. I did look into it a couple of days ago though..
<zacts>but when it = posix guile fails to build for sure.
<zacts>= pthreads it works.
<zacts>at least on FreeBSD-10
<mark_weaver>okay
***heroux_ is now known as heroux
<mark_weaver>zacts: it won't be long now. the tarball build is currently in progress. http://hydra.nixos.org/build/8715075
<zacts>cool
<mark_weaver>zacts: okay, here it is: http://hydra.nixos.org/build/8715075/download/4/guile-2.0.9.182-c8e83-dirty.tar.xz
<zacts>thanks, let me check it out
<mark_weaver>going offline for a while, ttyl!
<zacts>ok
<Onslauth>Hi guys. I ran into an assert in libguile, on a system that has been running for about a month or so without problems.
<Onslauth>The assert in question is hashtab.c line 137, and this is in guile-2.0.9
*mark_weaver looks
<mark_weaver>Onslauth: can you send an email to bug-guile@gnu.org about it?
<Onslauth>The system is forcing a garbage collect every so often. And it seems there might have been one just before this occurred
<Onslauth>I was actually hoping you might be able to suggest what caused the assert, as the only real error I can see in the logs is the assert output.
<mark_weaver>what version of libgc are you using?
<Onslauth>Everything up to that point looked normal.
<Onslauth>Let me find it for you
<zacts>mark_weaver: are you still going to be afk? I do have a question for you.
<mark_weaver>I'm not familiar with that part of the code. I could look into it at some point, but it might be more efficient to mention it to wingo.
<mark_weaver>zacts: what's your question?
<Onslauth>We are using bdw-gc
<Onslauth>version 7.2.7
<zacts>well, let me pastebin
<mark_weaver>Onslauth: that's a version number I'm not familiar with. The usual version numbers are 7.2d, 7.2e, etc.
<zacts>wow, my pastebin is down, let me do another one
<Onslauth>Hrm, ok
<mark_weaver>Onslauth: what OS/distro are you running?
<Onslauth>Sorry, version 7.2
<Onslauth>Solaris 10u10
<zacts>mark_weaver: http://nopaste.info/423f2db1ac_nl.html
<zacts>whenever you have time, no rush! :-)
<mark_weaver>Onslauth: can you send a bug report to bug-guile@gnu.org? mention the OS, libgc version, and the fact that you were forcing gcs manually. then I'll ask wingo to look into it.
<Onslauth>Sure
<mark_weaver>Onslauth: thanks!
<zacts>I'm using: export BDW_GC_LIBS="-L/usr/local/lib -libgc-threaded-pthreads" ; ./configure
<mark_weaver>zacts: so apparently HAVE_GC_SET_FINALIZER_NOTIFIER wasn't set, even though it seems to be there.
<mark_weaver>so i guess something went wrong with a configure test.
<mark_weaver>can you search config.log for "GC_set_finalizer_notifier", see if there are clues about what went wrong?
<zacts>yep
<zacts>doing that now
<zacts>/usr/bin/ld: cannot find -libgc-threaded-pthreads
<mark_weaver>Onslauth: also please mention any special flags you passed to Guile's configure.
<zacts>configure:51786: cc -o conftest -I/usr/local/include -g -O2 -D_THREAD_SAFE -I/usr/local/include conftest.c -L/usr/local/lib -libgc-threaded-pthreads -lcrypt -lm >&5
<Onslauth>Sure
<zacts>that was the complete error. two lines
<mark_weaver>zacts: oh, I guess you probably want -lgc-threaded-pthreads, right?
<zacts>mark_weaver: I don't think so
<mark_weaver>what's the .so filename?
<zacts>the FreeBSD ports install it as -libgc
<zacts>libgc-threaded-pthreads.so@
<mark_weaver>then you want -lgc-threaded-pthreads
<zacts>libgc-threaded-pthreads.so1
<mark_weaver>-l* turns into lib*
<zacts>oh, I see
<zacts>let's see if it works now
<zacts>thanks
<mark_weaver>np!
<mark_weaver>Onslauth: btw, did you run "make check" in bdw-gc?
<mark_weaver>did it pass the test suite?
<Onslauth>Sure, will do now
<mark_weaver>thanks
<Onslauth>Do you need all the configure flags?
<mark_weaver>you might as well include them all.
<mark_weaver>the configure flags passed to bdw-gc might also be relevant.
<mark_weaver>Onslauth: it would also be good to know which subversion of 7.2 you're using. 7.2e is available now, and is the recommended stable version.
<Onslauth>We using the 7.2.tar.gz release
<mark_weaver>ah, wingo!
<Onslauth>So the one from May 2012? I believe
<wingo>morning :)
<mark_weaver>wingo: Onslauth hit an assertion failure in guile 2.0.9 on solaris. hashtab.c line 137.
<mark_weaver>it's in vacuum_weak_hash_table. assert (removed <= len);
<wingo>uf
<wingo>that code...
<zacts>mark_weaver: ok it looks like it's going to build! \\o/ next I will send you a log of the grepped test failures
<mark_weaver>his system was forcing GCs manually, and it seems that the assertion failure might have happened immediately after a forced GC.
<mark_weaver>he said the system was running for a month or so without problems, and then this assertion failure.
<mark_weaver>zacts: excellent! :)
<wingo>perhaps that function isn't safe to call as a weak gc callback (via an async), as it accesses the fields of a weak pair without the alloc loc
<wingo>*lock
<wingo>as a workaround you could edit the source to remove all lines that look like " scm_c_register_weak_gc_callback (ret, vacuum_weak_hash_table);"
<wingo>in hashtab.c
<Onslauth>Ok, I am busy writing up an email with the bug report.
<wingo>i added those when we were experiencing inexplicable memory leaks and i thought it was weak tables themselves, but it turned out to be something different (weak entries whose value had a strong reference on the key)
<wingo>fwiw this particular bug should be fixed in master, as the weak hash tables are completely rewritten
<wingo>of course, there may be other bugs...
<Onslauth>Sorry, I havent been following the releases, but is 2.0.10 released yet?
<Onslauth>Or do you have a planned date for release?
<wingo>Onslauth: not yet, no
<mark_weaver>Onslauth: not yet, but I hope to make that happen next week.
<Onslauth>Ive sent the mail through now. The system we build with tries to tie down all libraries used.
<Onslauth>So that we never release and then find we have missed a lib
<wingo>cool
<Onslauth>Ive sent the mail with both the bdw-gc and guile-2.0.9 configure arguments used
<mark_weaver>Onslauth: thanks!
<mark_weaver>wingo: would you be willing to make the change you described to stable-2.0?
<Onslauth>Sure, however as wingo says I might as well just wiat for 2.0.10 which should solve the issue
<mark_weaver>Onslauth: no. he said the issue should be solved in master, which will become 2.2 eventually.
<wingo>mark_weaver: hum, dunno; i was looking again and it seems the vacuum routine is called within the alloc lock (via the before-gc hook)
<Onslauth>Ah
<mark_weaver>the stable-2.0 branch is what will become 2.0.10 soon.
<Onslauth>Which version would you suggest running ?
<Onslauth>Would moving from 2.0.x to 2.2.x be advised?
<mark_weaver>no, I wouldn't recommend that for a production system.
*wingo agrees with mark_weaver
<wingo>mark_weaver: to answer your question, it should be a harmless change but i don't understand why this error has happened
<mark_weaver>we should try to get to the bottom of this before 2.0.10 is released, if possible.
<Onslauth>So these are the entries in our logs:
<Onslauth>connection to database confirmed
<Onslauth>loading things from the database...
<Onslauth>connection to database confirmed
<Onslauth>loading things from the database...
<Onslauth>Assertion failed: removed <= len, file hashtab.c, line 137
<Onslauth>Sorry, this one has no timestamps
<wingo>i don't have time to debug it further, fwiw; other things are higher priority for me
<mark_weaver>Onslauth: fwiw, I would recommend switching to bdw-gc-7.2e
<Onslauth>And as you can see, at that point its doing nothing as its just spinning
<Onslauth>sure, not a problem
<Onslauth>i will try the 7.2e move
<mark_weaver>and run "make check" for both bdw-gc and guile.
<Onslauth>And this was the final gc message before the crash
<Onslauth>2014-02-01T17:28:00 T 12 gc: ((gc-time-taken . 250510000000) (heap-size . 74485760) (heap-free-size . 22282240) (heap-total-allocated . 13822504952) (heap-allocated-since-gc . 588512) (protected-objects . 113) (gc-times . 3171))
<Onslauth>All 10 tests passed
<Onslauth>our build script for bdw-gc runs a make check after compiling bdw-gc
<mark_weaver>it would be helpful to create a test program that tries to reproduce the problem, perhaps by repeatedly forcing GCs while adding lots of entries to weak hash tables that will be removed due to GC.
<Onslauth>Ok, will look into that
<Onslauth>Thanks for the help
<zacts>also, mark_weaver is it save to build guile-2.0 with gmake -j3 ?
<mark_weaver>at this point, it's not yet clear whether this only happens on solaris, or not. or whether it's related to the old version of bdw-gc.
<zacts>safe*
<mark_weaver>but fwiw, this is the first time we've received a report about this assertion failing.
<mark_weaver>Onslauth: you're welcome. thanks for the report. please let us know if it happens again, or if you can find a way to reproduce it.
<Onslauth>Will do.
<mark_weaver>zacts: should be, yes.
<zacts>mark_weaver: ok. sometimes the FreeBSD poudriere bulk building system (needed for testing ports), builds with multiple jobs at once.
<mark_weaver>although I'm not 100% sure that it will actually save time.
<zacts>well, it's kind of a default of poudriere on many bulk systems
<zacts>not really something I'm doing
<mark_weaver>because the most time-consuming part of the build process is compiling the first few .go files, and each one that's compiled makes the later ones compile faster.
<mark_weaver>that's fine.
<mark_weaver>anyway, it should be fine.
<zacts>good
<zacts>\\o/ ok it builds! \\o/ now let me pastebin test failures
<mark_weaver>zacts: excellent!
<civodul>Hello Guilers!
<mark_weaver>hi civodul!
<zacts>mark_weaver: http://nopaste.info/3544733a6c_nl.html
<zacts>^ fail.log
<mark_weaver>I want to try to release 2.0.10 early next week. I updated gnulib, and asked for access to the GCC build farm.
<zacts>(note: this was built with FreeBSD-10 clang and not gcc)
<mark_weaver>zacts: did you search for "ERROR" also?
<zacts>let me do that
<civodul>hello mark_weaver!
<civodul>mark_weaver: excellent, for 2.0.10
<zacts>mark_weaver: no lines with '^ERROR:'
<civodul>jemarch passed me a tiny fix for GOOPS, which i'll probably look at tonight
<taylanub>Any test version of 2.0.10 you'd like me to test on OS X and OpenBSD ?
<mark_weaver>zacts: okay.
<zacts>mark_weaver: ship it? (I'll wait for 2.0.10 for the actual port, but yeah..)
<mark_weaver>taylanub: yes, please! http://hydra.nixos.org/build/8715075/download/4/guile-2.0.9.182-c8e83-dirty.tar.xz
<civodul>or http://hydra.nixos.org/job/gnu/guile-2-0/tarball/latest :-)
<mark_weaver>yeah, I guess that's better :)
<mark_weaver>same thing now, but maybe not later :)
<mark_weaver>zacts: is there a package for GNU iconv?
<mark_weaver>in freebsd ports?
<mark_weaver> http://www.gnu.org/software/libiconv/
<mark_weaver>it seems that freebsd iconv is not very good.
<zacts>mark_weaver: yeah, let me install it
<zacts>do I need to re ./configure ; gmake ; gmake check?
<civodul>if there's a specific thing that it's known to fail at, we should add a configure test
<zacts>which of those?
<mark_weaver>zacts: thanks. also, better to grep the log for ^FAIL instead of FAIL
<zacts>mark_weaver: eek! sorry.
<mark_weaver>no worries :)
<taylanub>I seem to have libguile and libguilereadline left in ~/usr/lib after uninstalling Guile with make ..
<taylanub>Or maybe I didn't properly uninstall a previous version, I can't be sure.
<mark_weaver>civodul: yes, possibly, although I'm not sure any of these test failures are serious enough to warrant aborting the build.
<mark_weaver>but I should go to sleep very soon. I'll have to look into this more tomorrow.
<taylanub>(We also seem to leave around empty directory hierarchies but that's less bad I guess.)
<zacts>mark_weaver: gn, thanks for all the help
<mark_weaver>zacts: glad to help. thanks for working on this.
<mark_weaver>taylanub: can you send a report to bug-guile@gnu.org?
<mark_weaver>taylanub: well, if you confirm it, anyway.
<taylanub>Yeah, I'll try to see if it really was Guile's Makefile that left around those.
<taylanub>sleep well :)
<mark_weaver>thanks :)
*mark_weaver --> zzz
<zacts>would you guys also be interested in compiler warnings? (not errors, but warnings).
<mark_weaver>zacts: oh, and yes, you'll need to rerun configure after installing GNU iconv.
<zacts>yeah, I figured it out, but thanks
<mark_weaver>zacts: sure.
<zacts>ok cool
<mark_weaver>okay, now I'm really going to sleep :)
<zacts>gn
<civodul>good night! :-)
<Onslauth>Sorry, I have one last question, besides SICP, are there any other good Scheme books?
<zacts>Onslauth: yes
<zacts>Onslauth: which are you starting with?
<zacts>I know of several
<taylanub>Onslauth: The "Schemer series", "The Little Schemer", "The Seasoned Schemer", .., and then HtdP.
<zacts>the little schemer, the seasoned schemer, and the reasoned schemer
<Onslauth>Thanks
<zacts>Onslauth: if you want a book to read before SICP
<zacts>I'm reading 'Scheme and the Art of Programming'
<zacts>great book. by Daniel Friedman
<taylanub>(With "and then" I didn't mean to say you should read HtDP last.)
<zacts>it's a good intro to SICP
<taylanub>(HtDP is more like general programming, taught via Scheme, IIRC.)
<Onslauth>Thanks, i'll look at them
<taylanub>Let me note that I haven't read any of these myself. :P
<zacts>my only problem with the little schemer books is that I have to use a page opener to read them with my computer
<zacts>Onslauth: I have read the little schemer, and I'm almost 50% way through scheme and the art of programming.
<zacts>both are good
<Onslauth>Ok great, i'll see if I can get a copy somewhere
<zacts>Onslauth: one more thing..
<Onslauth>Yeah?
<zacts>for Scheme and the Art of Programming, you need to add a guile procedure to make the test functions work
<zacts>mark_weaver knows more, or I can give you that procedure when needed.
<Onslauth>ok, thanks
<Onslauth>Does anyone know if "The Little Schemer" is available as ebook from anywhere?
<zacts>not that I know of
<zacts>anyone know how I can install all files and libs into a local dir without having to ./configure again?
<zacts>DESTDIR=~/guile-stuff make install # or something like this?
***mario-go` is now known as mario-goulart
***Shozan is now known as SHODAN
<jemarch>hi
<civodul>'lo jemarch!
***DerGuteM1 is now known as DerGuteMoritz
<civodul>i'll look at the #:supers thing tonight :-)
*wingo thinks jemarch has a "hi on join" script :)
<brendyn>implemeted in braincode
<jemarch>wingo: come on, I am an artisan..
<wingo>:)
*civodul likes to think it's a genuine, friendly "hi" :-)
<add^_>Ohai
<jemarch>civodul: did you try bugz-mode?
<civodul>jemarch: not yet!
<civodul>i only have to use BZ when in despair, like when reporting a test suite failure for some obscure GLib-related package
<civodul>;-)
<civodul>but now that i have it around, my life will be much brighter for sure
<zacts>I answered my question myself with my question. :p
<civodul>heh, good
<zacts>I've got a new channel #nubsd, for a project wanting to deblob the BSDs to gnu free software guidelines.
<zacts>also, it might be cool to try a gnu/kfreebsd guix eventually
<civodul>i'll accept any patches in that direction ;-)
<civodul>but the first step is to get Debian's patches in upstream libc
<zacts>neat
<civodul>(good luck)
<zacts>thanks!
<jemarch>argh, GSOC again
<jemarch>civodul: btw, if you want sparc bootstrap binaries for guix feel free to send me an email with information about how I could provide them
<civodul>ahah, which hat do you have on? ;-)
<civodul>but yeah, i think it's clearly critical for SPARC to be able to run Guix
<taylanub>Should I worry about unresolved test-cases ? http://sprunge.us/IYdi
<jemarch>hat?
<jemarch>civodul: I am just a concerned citizen :)
<civodul>very well :-)
<civodul>taylanub: no, these are test cases that couldn't run or provide a definite answer for some reason
<mark_weaver>civodul: what is the "supers" thing? I have a vague recollection of fixing something having to do with supers already.
<mark_weaver>zacts: how goes guile on freebsd?
<zacts>mark_weaver: good, I'm getting a turkish language failure
<zacts>I'm trying to figure out how to get turkish iconv stuff installed on FreeBSD
<civodul>mark_weaver: GOOPS's 'make' supports #:dsupers but not #:supers
<mark_weaver>civodul: see 7cfcb60bc16263ee48323781a57b1007a27380d0
<mark_weaver>it was a documentation error.
<mark_weaver>zacts: are those turkish tests the only remaining failures after using GNU iconv?
<zacts>mark_weaver: yeah
<mark_weaver>zacts: excellent! it sounds like the port is in great shape!
<civodul>mark_weaver: that's one way to look at it, but jemarch (rightfully IMO) suggested that it should be #:supers; WDYT?
<mark_weaver>well, the thing is, it has always been #:dsupers (for direct supers), and presumably there exists code that uses #:dsupers.
<mark_weaver>so it seems to me a bad idea to change it.
<mark_weaver>WDYT?
<jemarch>What makes sense is to support either #:dslots AND #:dsupers or #:slots AND #:supers
<mark_weaver>I think we'd really have to support both.
<civodul>yes sure, that would mean supporting both
<civodul>well, no strong opinion here
<mark_weaver>I don't have any strong objection to supporting both, but it might be a slight pain to implement.
*mark_weaver looks
<zacts>mark_weaver: http://nopaste.info/9d1b21ab2f_nl.html
<jemarch>my patch keeps backwards compatibility, or at least that was the intention :)
<mark_weaver>well, I guess it wouldn't be too bad, but I think you'd have to search for both and merge them if they both exist.
<mark_weaver>or at least it's not clear to me that you could get away with not doing that.
<mark_weaver>well, the other thing is that existing code might look for #:dsupers and assume that there's no #:supers.
<mark_weaver>and also, the order of the dsupers is important, so how do you interleave them if you merge them?
<mark_weaver>I dunno, the whole thing seems worrisome to me. I agree that the naming is inconsistent, but I have worries about the implications of supporting both.
<mark_weaver>thoughts?
<zacts>I bbl, going to take a morning nap.. mark_weaver if you see anything scary in that paste let me know. whenevers. :-)
<mark_weaver>zacts: that looks great to me!
<mark_weaver>zacts: sleep well!
<mark_weaver>civodul: btw, the new 'makeinfo' produces a bunch of warnings about incorrect sectioning in the guile manual. I looked, but I don't see what's wrong. might you be able to take a look and give me a hint?
<mark_weaver>for example: ./api-utility.texi:519: warning: node `C Hooks' is next for `Hook Reference' in menu but not in sectioning
<mark_weaver>oh, maybe it's the "@subsubsection Handling Scheme-level hooks from C code" on line 613?
<mark_weaver>I'm not sure how this is all supposed to work. should the menu list things that have a @subsubsection but no @node?
*mark_weaver is still fairly clueless about texinfo.
<wingo>i think menu things need nodes
<mark_weaver>right, that's what I assumed. and yet it's complaining. maybe that @subsubsection should be a @subsubsubsection or something.
<mark_weaver>except that there's no such thing apparently :-(
<wingo>maybe you can't go that deep actually
<mark_weaver>ugh
<wingo>in that case perhaps the menu is inappropriate and the thing should be sectioned differently
<mark_weaver>well, I'm tempted to punt on this until 2.0.11.
<mark_weaver>(if not later :)
<wingo>:)
<mark_weaver>too much TODO...
<civodul>+1
<mark_weaver>davexunit: I'm tempted to make the changes to your repl-coop-servers patch myself. any objection? I'd leave you as the author, and just add a "Modified-by" line in the commit log.
<davexunit>mark_weaver: go for it.
<mark_weaver>okay, thanks!
<davexunit>I started it last night, but I won't have time to finish until tomorrow night.
<mark_weaver>davexunit: I'll send you the modified patch when it's ready. if you could do some basic testing with it, that would be helpful.
<davexunit>sure thing.
<mark_weaver>cool :)
<mark_weaver>wingo: I'd like to change 'close-socket!' in system/repl/server.scm to close the underlying file descriptor instead of the port. the reason is that closing the port while another thread is using it is not thread-safe, and davexunit has actually observed segfaults when using 'stop-server-and-clients!'.
<mark_weaver>wingo: WDYT?
*wingo updates to have latest code
<mark_weaver>hmm, well, I see that 'close-socket!' is used from a couple of different places. maybe not all of them should be changed.
<wingo>right
<mark_weaver>actually, i guess 'stop-server-and-clients!' is the only one that should be changed.
<mark_weaver>if you have a better idea of how to fix it, let me know.
<wingo>serve-client does need to flush the port
<mark_weaver>*nod*
<wingo>the "interrupt" case is a little silly given the removal of call-with-sigint, though a call to (quit) still throws 'interrupt; dunno
<mark_weaver>well, do we want (quit) to kill the entire server, or just that one REPL?
<wingo>anyway, as long as serve-client flushes the port, things will be fine i think
<mark_weaver>I guess that should be caught somewhere else.
<mark_weaver>(if it's not already; I haven't tested it)
<wingo>C-c doesn't kill the repl
<wingo>it raises 'interrupt, which is caught
<wingo>aah, quit throws 'quit, not 'interrupt
<wingo>sorry, was confused
<mark_weaver>I think C-c doesn't work from the repl server, and I don't see how it could really.
<wingo>in that case the 'interrupt case can be removed from the accept-new-client thing
<mark_weaver>sounds good
<mark_weaver>so I guess 'stop-server-and-clients!' can't call 'close-socket!'. it needs to do something else. the question is: what?
<wingo>mark_weaver: racy things with the fd? i don't know :) my answer to those things is on master ;)
<mark_weaver>I can think of three ideas: (1) close the underlying FD, (2) cancel the thread, (3) get a message to the thread to shut down cleanly somehow.
<mark_weaver>wingo: I wonder if even master fixes this properly. davexunit sent me a backtrace of the crash. one thread was in get_codepoint, and tried to access the scm_t_port structure through a pointer that had been nulled out by the close.
<mark_weaver>maybe the close would block indefinitely, until the read finished?
<mark_weaver>anyway, I think we still need to fix bugs in 2.0.
<wingo>i guess that would deadlock, yes... humm
<mark_weaver>it seems to me that closing the underlying FD isn't such a terrible solution. if we're closing another thread's port, it's not going to be particularly clean no matter what. WDYT?
<mark_weaver>maybe we can come up with a better solution later in master.
<wingo>for this case, sure
<mark_weaver>cool, thanks!
<wingo>it's impossible to do correctly though without a shutdown/wakeup channel
<mark_weaver>yes, I agree that would be best.
<mark_weaver>though I don't know how to wake up another thread from a blocked read.
<mark_weaver>(without being able to send a signal to a specific thread, which cannot be done portably afaik)
<wingo>select
<wingo>or poll, or whatever
<mark_weaver>well, yes. that's the ultimate :)
<wingo>:)
<wingo>poll the ultimate :)
<mark_weaver>it would be good to get rid of all blocked syscalls in guile, also to support green threads.
<wingo>yes, i would like to implement more of ports in scheme
<mark_weaver>some day :)
<madsy>ecology friendly green threads :D
<davexunit>what are green threads exactly? I thought they were just a term for coroutine.
<wingo>hand-crafted artisanal concurrency
<davexunit>twice the price of usual concurrency.
<wingo>coroutines are laced with pesticides, is the thing
<davexunit>buy it at Whole Foods and such.
<davexunit>hahaha
<mark_weaver>:)
<davexunit>are green threads USDA organic? :P
<mark_weaver>davexunit: http://en.wikipedia.org/wiki/Green_thread
<davexunit>mark_weaver: I understand it slightly more now, thanks.
<davexunit>how would guile implement these?
<mark_weaver>there are a few options, not sure which is best. one option is could implement it in the VM. but regardless, it would require avoiding blocking system calls everywhere.
<davexunit>mark_weaver: do you think this is a "must have" feature or something that would be nice but is low priority?
<mark_weaver>what currently blocks would instead yield to another green thread on the same posix thread, and somehow arrange to be woken up when the action would no longer block.
<mark_weaver>well, the main thing is the synchronization is a *lot* cheaper for green threads. on the flip side, green threads don't take advantage of multiple cores.
<mark_weaver>synchronization is cheaper (and easier) because green threads can only be interrupted (by another green thread on the same posix thread) at certain well-defined points.
<mark_weaver>and you don't have to worry about weirdnesses like writes being observed by another posix thread in a different order than they were performed.
<wingo>ethreads did something like that
<civodul>davexunit: guile-2d essentially does cooperative threading, right?
<wingo>on that wip branch
<mark_weaver>so, if a critical section is guaranteed to run quickly, it suffices to make sure that no green-thread-switches can occur within that section. no need to lock anything in that case.
<wingo>but they need to be integrated with guile ports, really
<civodul>wingo: new I/O procedures were exposed, right?
<davexunit>civodul: yeah I used delimited continuations and a primitive scheduler.
<civodul>ok
<wingo>civodul: yes, some nonblocking reads and writes
<civodul>yeah
<wingo>but they weren't exposed outside the module, iirc
<mark_weaver>wingo: so, closing the underlying fds almost works perfectly, except for one snag: calling close(2) on a server socket that's currently blocked in 'accept' doesn't really close it. it can still accept one more connection after its closed.
<mark_weaver>so, I'm thinking of generalizing *open-sockets* to instead be *socket-shutdown-actions*, a list of procedures to call from 'stop-server-and-clients!'
<mark_weaver>and in the case of the server socket, it would also set a flag that would be checked after 'accept' returns.
<mark_weaver>WDYT?
<mark_weaver>of course, we'd have to keep track of these procedure identities, so that we could remove them upon normal shutdown also. what a pain :-(
<mark_weaver>my first thought was that server sockets could have their ports closed, but no. there's still a race condition in there, because 'accept' needs to get the fdes from the port descriptor, which will be nulled out by 'close'.
<mark_weaver>hmm, is there a way to check if a fdes is closed?
<mark_weaver>I guess there's no way to interrupt the 'accept'.
<mark_weaver>well, I guess we could make the server socket non-blocking and use 'select'.
<mark_weaver>mmm, yes, I think that's the right solution here.
<madsy>Or epoll? :)
<mark_weaver>well, I'm trying to keep the code as portable as possible. in this case, this thread will only be polling a single server socket anyway.
<madsy>If you use select you set an upper bound on the number of sockets per thread
<mark_weaver>again, this thread will only be waiting for a single server socket.
<madsy>Ah, only one socket.
<madsy>Sorry my reading comprehension sucks this evening
<mark_weaver>np :)
<bu^>do you also consider poll as portable ?
<mark_weaver>no.
<mark_weaver>select is much older than poll.
<madsy>Poll was introduced in the 2.1 kernel. Is portability older than 2.1 really an issue? ;)
<mark_weaver>we support systems other than linux
<bu^>old doesn't mean that it's not portable; poll remains posix compliant; but I understand that in your case select is the best
<bu^>even hurd has poll ^^
<mark_weaver>the other thing is, this is Scheme code that I'm writing, and Guile already exposes 'select' to Scheme, but doesn't expose 'poll'.
<bu^>oh ok
<bu^>btw in your use case select fits well
<mark_weaver>*nod*
<mark_weaver>fwiw, I think we probably should switch to using 'poll' at some point, but that's a bigger job than I'd like to take on before 2.0.10.
<mark_weaver>mainly just because of the possibility that 'poll' might not be available on some system, like say mingw or something.
<mark_weaver>it would take some research.
<madsy>mark_weaver: Oh, so guile 2.0.10 will build on MinGW? :)
<mark_weaver>I hope so.
*madsy is suddenly excited
<mark_weaver>I don't see why not, anyway.
<madsy>When do you expect a release? Before April?
<mark_weaver>we need someone to work on this. last I heard, it was possible to build a mostly-working Guile 2.0.x on mingw.
<mark_weaver>I'm hoping for next week.
<madsy>I'm using guile 2.0.9 for my 3D game/demo engine. I hope I can port it to Windows soon
<madsy>Windows support would be wonderful
<mark_weaver>madsy: would you be willing to try compiling http://hydra.nixos.org/job/gnu/guile-2-0/tarball/latest on mingw? it would be very helpful.
<madsy>mark_weaver: Does it matter whether I use MSYS or a Linux shell?
<mark_weaver>I don't know. I'm almost entirely ignorant of this stuff.
<mark_weaver>also, I don't know what a "Linux shell" is.
<madsy>mark_weaver: Well, a terminal inside a linux distro. Cross-compiling with MinGW
<mark_weaver>Linux is a kernel. Shells mostly don't care what kernel they're running on.
<madsy>When I say "linux" here, I of course mean a linux distro
<madsy>If you want to be pedantic, GNU/Linux is a kernel :)
<mark_weaver>Cross-compiling from a GNU/Linux system is probably best, but it would of course also be helpful to test the thing on a real Windows system.
<mark_weaver>madsy: that's the first time I've ever heard that before, and I think it's mistaken.
<madsy>I can cross-compile your guile-2.0 tarball later this evening, and test MSYS tomorrow. I'm too lazy to reboot or setup a Windows VM now.
<mark_weaver>great, thanks!
<mark_weaver>it would be good to make sure that 2.0.10 works on Windows using Mingw.
<madsy>I didn't try compiling 2.0.8 for Windows yet, because you people said it would probably not work
<mark_weaver>yes, as I recall there are several problems fixed in 2.0.9, and some more fixes since 2.0.9 was released.
<mark_weaver>with your help, perhaps we can make 2.0.10 Just Work (TM) :)
<bu^>btw mark_weaver I looked a bit at AMD instruction documentation and agree with you for the divide stuff; even if it would be counter intuitive for mathematicians remainder could be chosen to be with -|b|/2 < r < |b|/2 such that when discarded the quotient would be rounded to nearest. But you need good friends (at Intel/AMD) and to exhibit a strong use case if you want this to ever happen ;)
<madsy>mark_weaver: Where can I find a list of dependencies?
<madsy>I'll have to build all the dependencies myself
<mark_weaver>bu^: the main thing that mathematicians need from a modulo operator is that is maps the integers to a finite ring, for a given fixed positive divisor. 'floor-remainder', 'ceiling-remainder', 'euclidean-remainder', and 'centered-remainder' accomplish that.
<mark_weaver>s/is maps/it maps/
<mark_weaver>'truncate-remainder' and 'round-remainder' do not accomplish that.
<mark_weaver>madsy: the README lists them.
<bu^>yes but if you only have positive integers having a negative remainder would drive them crazy :)
<mark_weaver>unfortunately, processors implement 'truncate-remainder'.
<bu^>yes :s
<bu^>my guess is that it is mainly for historical reasons
<mark_weaver>with a positive divisor, 'euclidean-remainder' and 'floor-remainder' are the same thing.
<mark_weaver>either one would be good.
<madsy>mark_weaver: Any idea if I can build guile against static libraries only?
<mark_weaver>madsy: it can certainly be done. it's possible that some features might be missing though. I don't remember off hand.
<mark_weaver>madsy: Guix bootstraps from a completely static guile, for example.
<madsy>Ok, if it's possible, I'll give it a shot
<mark_weaver>cool.
<bu^>if you divide 11 by 3 you could have 4 as a quotient and -1 as remainder which would give you the rounded value directly by discarding the -1 but you would "quit" the N+ set (natural positive integers)
<bu^>and that's a good enough reason not to do so (from a mathematics perspective)
<mark_weaver>right, it's usually more convenient if the remainder is always non-negative.
<bu^>do you have some examples where this would be a great benefit (being directly rounded to nearest) ?
<mark_weaver>although there's no inherent problem with non-positive remainders (as returned by ceiling-remainder for positive divisors), or a ring that's half positive and half negative (as returned by centered-remainder).
<mark_weaver>it's really just a matter of convention.
<bu^>of course
<bu^>but you know... mathematics purity, minimalism etc... :)
<mark_weaver>but unlike the other three variants mentioned above, 'truncate-remainder' and 'round-remainder' do not cycle through a set of N values (where N is the divisor), which is bad.
<mark_weaver>'round-remainder' fails to do so if N is even, and 'truncate-remainder' fails to do so unless the dividend is constrained to a single sign.
<bu^>now for negative values its even funnier, the results of the remainder depends on which (diviend or divisor) the negative sign is
<bu^>but it could be used to do floor or ceiling rounding I guess (but might be wrong)
<bu^>yes, it's very hard to handle in a consistent "scheme" :s
<bu^>so that I understand your desperation
<bu^>maybe you could ask the openrisc guys to think about it for their next processor
<mark_weaver>it would probably be a hard sell, because people who need to write fast code specifically use the 'truncate' divide and remainder, simply because that's what virtually all existing processors implement directly.
<mark_weaver>it's a self-reinforcing thing.
<mark_weaver>so any new processor is going to want to be optimized for the existing body of C code.
<mark_weaver>and any new C code is going to want to be optimized for the existing processors.
<mark_weaver>so we're kind of stuck, unfortunately.
<bu^>not if you find some very common case use, where you can show that nearest rounding makes things easier/faster
<bu^>then the compiler can easily "adapt"
<bu^>if you want truncation or the rounding use case
<bu^>well maybe not easily
<bu^>but can be done
<bu^>and for example openrisc is about "simplicity", "cleaness" and practical use
<bu^>I think they (for example) have less constraints than big companies
<bu^>It could be possible to implement your use case to performs some tests (even through fgpa first)
<bu^>like adding a idiv2 instruction
<mark_weaver>well, either 'floor' or 'euclidean' would be the really useful ones to have. 'floor' is probably more likely to be accepted.
<bu^>but I recognise that this would be a lot of work
<mark_weaver>yeah, I think that the only realistic way for it to happen would be for processors to support both 'truncate' and 'floor', and for compilers to be modified to support the new instructions.
<madsy>mark_weaver: Does guile for windows also depend on libintl and libiconv?
<mark_weaver>but it would take many years. such systems would have to because almost ubiquitous before programmers would change their habits and start using the new floor operators.
<mark_weaver>madsy: I guess probably so.
<madsy>Just asking because those libraries are very Unix'y
<mark_weaver>s/because/become/
<bu^>it would take time yes, but you know, new revision of C standard would make the thing faster ^^
<mark_weaver>right, I'd be surprised if mingw included those interfaces by default, given that mingw is supposed to be a very thin wrapper.
<bu^>you just need to convince one big player that it's a win to in this direction then inertia will play in your favor (you know, the "it's new and smart" thing, "let use it")
<mark_weaver>it's probably not enough of a win to convince the big players to do it.
<bu^>+go*
<mark_weaver>unfortunately, I think that this is a wart that we'll have to live with indefinitely, like many other warts.
<mark_weaver>the only hope is that some radically new computer architecture will become popular, something that's not really based on C at all. such a thing might be feasible for other reasons, such as much improved parallelism or security.
<bu^>but for this there should be some critical / common algorithm that could take benefit from this
<bu^>just kill all humans and all computers, and start again; from scratch :)
<mark_weaver>heh
<bu^>I think that C adapted to the idiv instruction; but then as you said you have this resonnant loop enhancement
<mark_weaver>right
<bu^>well, easy solution !
<mark_weaver>although C predates the intel architecture, I believe.. but yeah, whatever processor C was designed for probably did truncate division.
<bu^>just make you guile compiler
<bu^>and make lisp processors back in action
<mark_weaver>heh :)
<mark_weaver>I see only two things that could dethrone the current basic architecture: poor support for parallelism, and security. both of these can't really be properly fixed without a radical new architecture.
<mark_weaver>and both of those are compelling enough to make such a redesign potentially feasible.
<bu^>I would also see a very common and ubiquitous algorithm and the change would improve performances
<bu^>that's used in billion $$$ industries and fields that need performances
<mark_weaver>right, the poor parallelism of current architectures might potentially fall into that category.
<bu^>isn't the parallelism issue mainly related to out of order reordering on the chip ?
<mark_weaver>no, I mean that it would probably be a huge win to switch to something closer to a human brain. 3 dimensional, massively interconnected, without need for any synchronization at all.
<mark_weaver>or perhaps something more like Sussman/Radul's propagators, in hardware.
<mark_weaver>or maybe some kind of 3D cellular automata.
<mark_weaver>I think something like that could vastly outperform the best systems of today. but of course it would require a massive change in the way we write programs.
<bu^>but it would be slower for number crunching even if you use other technologies
<mark_weaver>I don't see why it needs to be slower for number crunching.
<bu^>you gain flexibility and parallelism at the expense of speed
<bu^>how to state, speed per millimeter square if you want
<bu^>at the moment you wont beat semiconductors
<bu^>not only because of 50 years of research on it and billions invested
<mark_weaver>I didn't say anything about moving away from semiconductors.
<zacts>heat
<mark_weaver>I'm talking about the topology of the thing, that's all.
<zacts>they could make the chips smaller but then they would melt
<zacts>like a grilled cheese sandwich that I'm hungry for
<bu^>ok, than you have to be "really" "really" smart to make your architecture without make 1) a 1 meter square CPU; 2) burning the room and having a nuclear plant in your garden
<bu^>mark_weaver, the issue today is more like theoretical from the technology used, you cannot minimise more (with semi conductors) without spending 70% of your energy just in heat dissipation and even if you go bigger, you can't go much faster because you would have synchronisation issues (relativity etc...)
<mark_weaver>if you read up on the physics of computation, it turns out that there's no lower limit on the amount of energy it takes to do computation. there's a lower limit on the amount of energy associated with _losing-information_, and that lower limit is proportional to the temperature, for which there is also no lower limit. and it's possible to do computation where information is not lost. c.f. reversible-computing.
<mark_weaver>so there's plenty of room for improvement here. what we've got now is very far from the physical limits.
<bu^>yes but would you agree to use 99% of your energy just to heat your room ?
<bu^>without performing anything information related
<mark_weaver>we're probably quite close to the physical limits of how fast a single-threaded processor can be made to run, but that's just dumb.
<bu^>there are limits with semi conductor technologies
<bu^>other wise if you're rich you rebuild your cpu with superconduction transistors
<mark_weaver>and now we're being forced to parallelize, but the architecture is fundamentally geared for single-threaded computation. the parallelism we have now is bolted on, and really doesn't work very well at all.
<bu^>buy a nuclear plan and a poll of liquid helium :)
<bu^>for the parallelisation there is a thing that hinders the easy to add more cores in current architecture but I don't remember what it is
<bu^>read about this last week :/
<mark_weaver>anyway, this is an interesting conversation, but I really should work on getting 2.0.10 out the door soon :)
<bu^>sorry to disturb you :)
<mark_weaver>bu^: no worries :)
<madsy>mark_weaver: This could take a while. I have 8 libraries to build :)
<madsy>I'm probably back in 3 hours
<mark_weaver>madsy: great, thanks!
<madsy>mark_weaver: If guile depends on some certain features I have to enable in ./configure of some libraries, I'd like to know
<madsy>I just built libgc with gc debugging enabled
<mark_weaver>madsy: you might need to pass --disable-posix. I don't remember if that's still needed or not.
<mark_weaver>I don't know about threading.
<madsy>When compiling guile? Yeah, probably
<madsy>I was more thinking about guile's dependencies, not guile itself
<madsy>Some libraries can be built with certain features disabled
<zacts>as soon as I get to an electric outlet I'm going to try building guile again
<mark_weaver>madsy: last April, someone sent me a build script to build Guile and its dependencies on mingw32.
<mark_weaver>maybe it would be helpful. would you like me to send it to you?
<madsy>mark_weaver: Yes please. I have some trouble building libintl, so maybe that would help
<mark_weaver>madsy: okay. I'll need your email address.
<madsy>mads at mechcore dot net
<mark_weaver>madsy: sent!
<madsy>I think the libintl build scripts work fine in MSYS, but aren't made for cross-compiling
<madsy>So I'm a bit stuck
<mark_weaver>maybe the build script that LRN wrote has some clues.
<mark_weaver>he used to hang out on #gnunet, as I recall.
<madsy>Trying to cross-build the whole gettext library instead of building the mingw-port of libintl
<madsy>Seems to be successful so far
<mark_weaver>cool!
<madsy>Yay, seems to have worked. Need a smoke, before building guile
<mark_weaver>yay!
<madsy>mark_weaver: I have all the libraries install under ~/mingw/lib and ~/mingw/include/. I also have pkg-config files for libffi and libgc there. Can I easily tell autoconf about that in a simple way?
<madsy>Without setting the path for each library manually
<madsy>CFLAGS=-I/home/madsy/mingw/include and CXXFLAGS=-L=/home/madsy/mingw/lib maybe? Is autoconf that clever?
<madsy>mark_weaver: Okay, first error. ./configure worked fine. Building guile made gcc complain about missing pthread.h
<madsy>I built with:
<madsy>./configure --host=i686-w64-mingw32 --prefix=/home/madsy/mingw --enable-static=yes --enable-shared=no --enable-threads=windows --disable-rpath --enable-debug-malloc --enable-guile-debug --disable-posix --disable-deprecated --with-sysroot=/home/madsy/mingw --without-libpth-prefix
<madsy>--enable-threads=windows and it still looks for pthreads?