IRC channel logs

<civodul>Hello Guilers!

<civodul>wingo: my Tweeter/Peeple competitor in Hop: http://sed.bordeaux.inria.fr/seminars/hop_20131015.pdf and http://sed.bordeaux.inria.fr/seminars/cuicui.hop :-)

<civodul>Twitter, even

<nalaginrut>netBSD added Lua into kernel now, could write driver with Lua, anyway it's trivial news for Hurd...

<nalaginrut>civodul: scheme2js?

<civodul>one can already write anything in any language on the Hurd

<civodul>nalaginrut: yes, Hop = Scheme2JS + libraries

<civodul>nalaginrut: any pointers to the NetBSD thing?

<civodul>i wonder how they deal with GC

<nalaginrut> http://mail-index.netbsd.org/tech-kern/2013/10/10/msg015669.html

<civodul>thanks

<nalaginrut> http://mail-index.netbsd.org/source-changes/2013/10/16/msg048283.html

<nalaginrut>the second is commit

<wingo>civodul: neat :)

<nalaginrut>civodul: nice! seems it's what I always wanted !

<civodul>nalaginrut: don't repeat it, but if i were you, i'd rip Scheme2JS out of Hop and use that in Artenis

<nalaginrut>I'd like to, but how about the lib?

<wingo>cui cui!

<civodul>neat, no? :-)

<wingo>indeed :)

<civodul>during the talk i had my laptop in the DMZ, so the audience could connect to the server and post cuicuis

<civodul>that was quite fun

<wingo>a remarkably concise implementation

<civodul>yeah

<civodul>the only thing is there's no Geiser or anything, so you have to restart the server every time you change something

<nalaginrut>civodul: where is the repo?

<civodul>nalaginrut: there's hop.inria.fr, and from there there's in theory an hg repo, but somehow it's unavailable most of the time

<civodul>...

<nalaginrut>oops, why not github?

<civodul>hg!

<wingo>hg is slooooooow

<wingo>and their patch queue stuff is wonky

<nalaginrut>can I run with Guile? or have to use bigloo?

<wingo>port it to guile! :)

<nalaginrut>oops ;-)

<ZombieChicken>Is there anything resembling SLIME for guile?

<wingo>geiser

<wingo>ZombieChicken: ^

<ZombieChicken>wingo: ty

<civodul>Geiser is the nicest thing since sliced bread

*wingo doesn't buy sliced bread ;-)

<add^_>lol

<civodul>:-)

<civodul>"[gnu-prog-discuss] Guile integration with GNU Cobol"

<civodul>Make, COBOL, GDB...

<civodul>world domination is getting closer

<add^_>heh

<add^_>Don't forget Guix ;-)

<add^_>I mean, you if anyone shouldn't forget it :-)

<civodul>right ;-)

<civodul> http://www.opencobol.org/modules/newbb/viewtopic.php?topic_id=1167&forum=1&post_id=6988

<civodul>craziness all around

<civodul>embedding brainfuck code via Guile in a COBOL program

<add^_>heh

<add^_>:-D

<dsmith>wingo, Does disassemble still work with your latest changes?

<dsmith>wingo, That would be v2.1.0-1198-g5bd4b65

<dsmith>wingo, Hmm. My mistake

<mark_weaver>did I hear somewhat say "why not github?" ? http://bytbox.net/blog/2012/08/leaving-github.html

<mark_weaver>s/somewhat/someone/

<davexunit>mark_weaver: great article. reading it now.

<davexunit>I've been meaning to get off of github, but haven't made the switch yet.

<dsmith>mark_weaver, Thanks for the link. Never knew that, as I don't use github.

<davexunit>"When asked what VCS they’re using, people reply more often with “github” than simply “git”."

<davexunit>I've definitely noticed this.

<davexunit>or people telling me that "git" is down when github is getting ddos'd (again)

<mark_weaver>it's been bugging me for a while. I'm glad to have a good article to cite now :)

<davexunit>me too!

<davexunit>this article has expressed things that I've been trying to find the right words for.

<davexunit>I must admit that I'm a sucker for getting "stars" on my repos.

<davexunit>I guess I need to setup cgit and some bug tracker on a server of my own, I guess.

<mark_weaver>yeah, that gaming aspect is a powerful draw for sure. I was very nearly sucked into spending a bunch of time on stackoverflow for a similar reason.

<davexunit>yeah, it's enticing.

<davexunit>perhaps I should just move everything to gitorious.

<davexunit>though I think gitorious is a far bit more difficult to use and quite slower.

<mark_weaver>I confess I haven't used any of these services much, so I can't offer much guidance there. but gitorious at least has the benefit of running all free software, such that you can run the same software on your own server if you like. I'm not sure how gitorious stacks up on the issues raised in that article though.

<davexunit>yeah, gitorious is free software which is the big appeal.

<davexunit>I guess I could just get another $5 vps and use cgit + mailman

*stis got an idea how to fix threading and vhashes.

<stis>wo much loss of efficiency

<mark_weaver>what's your idea?

<stis>essentially make sure that there is a hook that get's evaluated in each thread at trhe creation of a new thread

<stis>Then keep the vhash in a fluid, but this is just a side point

<stis>the main point is to essentially freeze the current block in the vhash and creat new ones that points to it

<stis>simple ya!

<stis>One can also generate a random number to xor the hash and sue that to add hash entries non mutatingly to the current block in the main thread

<mark_weaver>I don't understand.

<stis>Well if we just freeze the block all would be well right!

<stis>But if it can be a waste of resources, and we would like to reuse the vast space no?

<stis>to assoc k,v on onto a slot that is ocupied, but with another key, then move to another slot seay x times

<stis>else make a new block

<stis>remains the case where we hash the position occupied with the same key in which case just make another block and freeze the old one.

<stis>That issue is that we do not want to in the mean fill the block to 10%, but in stead fill it to say 50%

<stis>not sure if that would be the mean case, but if the nwely generated hash from the old hash permutes randomly

<davexunit>I think I've asked this before but I can't remember the answer. I see 3 different hash table implementations in guile: srfi-69, rnrs hashtables, and guile's own hash table implementation. they are all a bit different. what do I use?

<davexunit>are they compatible in any way?

<stis>then we should fill it quite well before one needs to bail out

<mark_weaver>davexunit: unfortunately, they are all mutually incompatible :-(

<mark_weaver>SRFI-69 and R6RS hash tables have a nicer API, but are built on top of guile hash tables, so the latter have a performance advantage.

<mark_weaver>I'd like to unify the representations of SRFI-69 and R6RS hash tables at some point, and maybe arrange things so that they are just as fast as native guile hash tables.

<civodul>stis: that'd require adding an atomic test-and-set instruction i believe

<davexunit>mark_weaver: one thing that bothered me was that srfi-69 can convert an alist into a hash, but guile's native hashes have no such procedure.

<davexunit>it's trivial to write, and perhaps I could submit a patch for that?

<mark_weaver>davexunit: sounds reasonable to me :)

<mark_weaver>civodul, stis: using any kind of memory barrier will make vhashes slower in the single-thread case.

<civodul>mark_weaver: yes and no; i mean, the cost of the memory barrier is negligible compared to the other costs, such as cons

<civodul>of course we hope it will become non-negligible someday ;-)

<civodul>but we're not there yet, i think

<civodul>futhermore we could arrange so that the test-and-set instruction is a nop when there's a single thread

<mark_weaver>cons is not inherently expensive. it's just expensive right now because of the GC we're using. memory barriers are inherently expensive.

<civodul>like glibc does

<civodul>hmm yeah

<mark_weaver>we really need to make allocation much cheaper. I heard that libgc requires thread sychronization on every allocation, is that still true? why are there not per-thread allocation areas?

<civodul>there are per-thread allocation areas

<civodul>that only works for small allocations IIRC

<civodul>regardless: vlists will always be more costly than plain lists, as noted in the manual

<mark_weaver>I confess, I still wish that vlists were implemented in C. I wonder how much faster they would be.

<mark_weaver>I suspect they'd be 5-10 times faster.

<mark_weaver>I'd certainly like to see most of our C code converted into Scheme, but some hot spots are worth optimizing.

<stis>mark_weaver: synchronization is only needed at thread creation, it is at that time the freeze of the block is done

<mark_weaver>stis: in your idea, what would be the O() complexity of adding new entries (in both time and space), and what would be the O() complexity of lookup?

<stis>depends on how one creates threads

<stis>but in the best case (all threads in created at one go) it would be the same as the old vhash

<mark_weaver>I seem to always have trouble understanding your proposals.

<mark_weaver>though in this case it might be because I've forgotten the details of vhashes.

<civodul>stis: are you interested in make vlists thread-safe, or in making them faster?

<dsmith>wingo, Ya, works fine!

<mark_weaver>stis: one problem is that there could be a vast number of vhashes created in some programs, especially since every time you add a new entry, it creates a new vhash. if every vhash needs a separate fluid, that will cause problems, because every thread needs a vector with one element per fluid.

<mark_weaver>if the space complexity is O(N*M), where N is the number of vhashes and M is the number of threads, that's not good.

<stis>well I woudl like to see some actual scenarios befor trying to attack the problem of thread safeness.

<stis>For sure for some scenarious it is best to use a functional tree to handle the hash

<stis>but essentially if athe new threads are for example created only once and does not in itself create new tthreads

<stis>then space/time complexity should not be a problem.

<stis>if the main thread craetes M threads in sequence with computation between them

<stis>space shoudl again not be a problem, but in stead key lookup would be problematic

<stis>beacause you would need to pass through n newly created block structs

<stis>but thining about this we shoudl detect this scenario at one point and recreate the hash at the thread creation

<rooster_rus>how do i make my emacs work with guile the way edwin works?

<stis>so it should be fxable

<rooster_rus>C-x C-e with run-scheme is terrible

<dsmith>rooster_rus, Not sure about edwin, but try geiser

<stis>civodul, for now I'm interesting in speed, and I think that we can get a solution that works in many threaded scenarious that does not sink the speed too much.

<rooster_rus>In edwin C-x C-e evaluates code and tell the result in the same buffer

<civodul>stis: ok; speed is difficult, i think

<stis>we should also have a pure funcitonal tree in guile for scenarious outside of these use cases

<rooster_rus>emacs pipes it to run-scheme, which is unusable

<dsmith>rooster_rus, Try geiser. It's kind of like slime for guile

<dsmith>rooster_rus, http://www.nongnu.org/geiser/

<stis>civodul: I've built vlist in C to test out performance FYI.

<civodul>grrrr

<civodul>;-)

<stis>I checked the vlist implementation and I think that we can boost vlist lookups by a factor of 5-7 times.

<stis>That should be doable in current guile if we add vlist-lookuop as a VM op

<stis>(this was the case where it found the element in the first block)

<civodul>maybe we can add 'compile-file' as a VM op? ;-)

<stis>:-)

<stis>yeah we can and probably should wait for native guile.

<civodul>seriously, it'd be great to profile at the C level what goes on with vhash lookups

<civodul>that may give hints that would allow us to optimize the damn thing without completely rewriting it in C

<stis>Yeah I plan to profile the C versions to see the bottle necks. Otherwise other overhead can dim the pictture

<civodul>i meant profiling the current version, of course

<stis>then vlist-ref will be like tring to profile vector-ref, you would just see the overhead!

<mark_weaver>well, vlist-ref has a lot more work to do than vector-ref.

<stis>maybe I'm wrong!

<civodul>we won't know until it's actually been profiled :-)

<civodul>for instance, it could be that adding specialized arithmetic instructions could help, along with CSE

<stis>yeas vlist_ref does about 4 lookups in the fast path

<civodul>say, we'd write (assert-integer index), and that would cause the body of vlist-ref to use int+, int<= etc. instead of the fully generic ops

*civodul has to go

<civodul>ttyl

<stis>bye!

<mark_weaver>yeah, I guess we should wait until we have native code gen, and then see where the remaining performance problems are.

<davexunit>that's coming in a week or two, right?

<mark_weaver>hehe

<davexunit>I am eager for any performance improvements to guile. :)

<mark_weaver>indeed, that's understandable! once I get MIPS N32 going on Guix, I intend to spend more time on Guile.

<mark_weaver>anyway, going afk for a while. ttyl!

<davexunit>see ya, mark_weaver

*bubble is going to start compiling guile 2+ on cygwin so he can help besides support

<stis>Hmm, I was able to speed up vhash-assq 30x

<stis>using C and guile datastructures

<stis>in all it did 12M lookups in 1 sec.

<stis>about 10x faster if we load it into guile.

<bubble>mark_weaver : what was the X11 lib for guile again ?

<bubble>bindings or something

<mark_weaver>bubble: guile-xcb, but you'll need to use the git repo directly. the tarball has some problems.

<mark_weaver>though guile-figl and/or guile-sdl might be preferable, depending on what you're doing.

<bubble>thx, I'll manage

<bubble>I posted my repo of the server-side scripting cool idea of the guile webpage to guile-user

<bubble>I am now going to look into guile-sdl, should make a nice GUI for it

<ijp>the only nice thing about guile-sdl is that you get pretty pictures when you run make check

<bubble>lol as perl tk tests ? :-)

<bubble>ok, compiling both SDLs now, will extend the serverside scripting later on

*bubble got SDL2 compiled

<tupi>heya guilers

<tupi>i have a strange bug which onlu occur on extremelly powerfull server [32 cores]

<tupi>hereis it

<tupi>n ice-9/popen.scm:

<tupi> 106: 1 [close-pipe #<input: #{read pipe}# 69>]

<tupi>In unknown file:

<tupi> ?: 0 [scm-error misc-error #f "~A" ("close-pipe: pipe not in table") #f]

<tupi>

<wingo>zow

<wingo>can you reproduce it somehow?

<wingo>what version of guile?

<mark_weaver>tupi: does that version of guile include my preliminary patch to make (ice-9 popen) thread safe?

<mark_weaver>(I still haven't pushed it to the repo)

<tupi>civodul, it only happens if i run heavy stuff that occupies all 32cores for a while

<tupi>mark_weaver: let me check

<tupi>i think so

<mark_weaver>tupi: actually, the patch I gave you doesn't even entirely fix the problem, I don't think.

<tupi>GNU Guile 2.0.9.20-10454

<mark_weaver>when I tried a complete fix, I ran into deadlock problems with asyncs. I think I know how to fix it, and we'll get it fixed for 2.0.10.

<tupi>i think this version includes the patch, i am almost certain

<mark_weaver>sorry for the trouble :-(

<tupi>any chance to get a fix on git?

<mark_weaver>I'll try to fix it as soon as I can.

<tupi>mark_weaver: no problem, but do you need further info?

<mark_weaver>not at this point, no.

<mark_weaver>but I'll want you to test the proper fix, when I have it ready. I'll let you know.

<mark_weaver>anyway, gotta go offline for a bit. ttyl!

<tupi>ok

<tupi>tx

<tupi>sorry, I wrote civodul, but ment to reply to ... wingo :)

<civodul>np

<civodul>that sounds like a synchronization issue

<tupi>haha

<wingo>:)

<civodul>i really meant it :-)

<civodul>i mean the pipe is closed too early somehow

<tupi>civodul: yes

<tupi>for info, the 'too early' seems linked to the speed [power] of the cores

<tupi>on 12 cores machine [already really good server] it does show [didn't till now] the prblem, where on this incredible computer i am configuring now, it does almost always trigger the bug

IRC channel logs

2013-10-17.log