IRC channel logs

<unknown_lamer>sneek: later tell davexunit you might want to look at http://www.haskell.org/haskellwiki/Frag regarding FRP (the thesis paper associated is great, and notes a few pain points with normal FRP that they found solutions for)

<sneek>Okay.

<ijp>the inspection on r6rs records isn't quite as nice as it could be, I think srfi 99 had a better story there

<nalaginrut>oh, we have srfi-99

<nalaginrut>I didn't know that

*nalaginrut reading

<ijp>we *should*, since that's basically what the guile record api si

<ijp>is*

<nalaginrut>maybe it's totally the same with r6rs one?

<ijp>there are some differences

<nalaginrut>can it list fields as a list?

<nalaginrut>I can get field-names, but seems record-accessor doesn't work as I thought

<nalaginrut>hmm, neither rtd-accessor

<nalaginrut>alright it works

<ijp>it takes an index for some reason

<nalaginrut>the unary proc needs the original record-type obj, rather than its descriptor

<nalaginrut>now I can write some wrapper for record-type->list

<nalaginrut>thanks!

<ijp> http://paste.lisp.org/display/140278

<ijp>for guile and srfi-99, record-accessor takes a field name (as you'd expect)

<ijp>some of the r6rs record decisions are kinda arbitrary IMO

<unknown_lamer>nature of standards I guess

<unknown_lamer>luckily there are ERR5S and R7RS-small now, I guess r7rs-large is finally restarting?

<ijp>luckily?

<unknown_lamer>pick whatever standard implements what you want!

<ijp>err5rs means larceny only

<unknown_lamer>I think Scheme is where Lisp was around 1980-84 still

<ijp>lisp is still where lisp was around 1980-84

<unknown_lamer>portable code? ... may as well give up now

<unknown_lamer>Eh, Common Lisp has advanced quite a bit, and it's trivial to write code that works on every major implementation

*nalaginrut doesn't want involved such an argue, since he know C++ guys also argue much about their new standard now...

<nalaginrut>s/involved/involve

<unknown_lamer>Closer-MOP finally got almost everyone implementing the full mop, and everyone has basically standardized on the same sockets, threading, etc. interfaces

<ijp>anyway, as for portability in scheme, you can get the same level in r6rs, but there aren't enough people who care about r6rs left

<ijp>r5rs and r7rs are unportable by design

<ijp>hmm, something about this frp implemenation rubs me up the wrong way

<nalaginrut>although I have claimed I don't use r6rs, this time I have to, because I need the inheritance of record-type. But fortunately I know srfi-99 now, so I can remove r6rs code ;-P

<ijp>did srfi 99 cover inheritance?

<nalaginrut>I saw rtd-parent proc

<ijp>maybe it was just the protocol stuff they removed

<nalaginrut>so I think it has

<ijp>apparently so

<nalaginrut>well, it's a good news for me ;-D

<ijp>I don't see a srfi-99.scm, are you just talking about the builtin records?

<nalaginrut>but I can load srfi-99 in Guile REPL

<nalaginrut>illusion?

<ijp>no, those aren't quite compatible

<ijp>nalaginrut: ah, you may have gotten it from the srfi package on guildhall

<nalaginrut>hmm?

<ijp>those are written on *top* of r6rs records

<nalaginrut>I can find is 'srfi' packages

<nalaginrut>how to check more info

<nalaginrut>ok I check it with 'guild show srfi'

<nalaginrut>but it contains up to 98

<nalaginrut>there's no 99

<ijp>look up a bit

<ijp>it doesn't seem to be at the bottom of that list, but it is there

<nalaginrut>alright, there is

<nalaginrut>anyway, it's compatible with srfi interface

<nalaginrut>dunno, maybe I should include srfi-99 in my project

<nalaginrut>but actually, I didn't write rnrs compatible code, since the hash-table is Guile specific

<ijp>personally, I don't like the guile hashtable interface, because it exposes the type of hashing on each access

<ijp>srfi 69 or (rnrs hashtables) provide a nicer API

<nalaginrut>yeah, seems srfi-69 is better

<nalaginrut>Guile's hash-table is too wak

<nalaginrut>weak

<wingo>got guile startup down to 16.5M instructions

<wingo>that's around 8ms here

<wingo>guile 2.0 is 39M

<wingo>guile 2.2 was 21M yesterday :)

<dsmith>Whoo

<dsmith>THat's about three times faster than 2.0

<dsmith>No

*dsmith can't do the math

<wingo>well there are milliseconds and instructions

<wingo>i think guile 2.0 is around 13 or 14 ms here

<wingo>the way you get fewer ms is to execute fewer instructions, but it's not necessarily a linear thing

<wingo>mark_weaver: around?

<wingo>happy txgiving btw americans :)

<wingo>i don't think blocking asyncs is necessary when locking most mutexes

<wingo>the issue isn't which critical sections may be entered via an async -- it's which critical sections can call out to arbitrary code

<wingo>likely all instances of the latter are bugs

<wingo>mark_weaver: ^

<wingo>in the case of guardians, arbitrary code can't run inside a guardian's critical section, thanks to the newly added mutex

<wingo>so asyncs do not need to be blocked then

<mark_weaver>wingo: it's true that if asyncs can't be called within a particular critical section, then there's no sense in blocking asyncs in that case.

<mark_weaver>so yeah, maybe I didn't need to block asyncs while the guardian mutex is locked.

<wingo>the procedure properties thing was definitely a bug tho

<wingo>i fixed it differently in master tho

<wingo>having different fixes is fine tho

*wingo adds tho to all statements

<mark_weaver>I guess the one thing that worries me is that, as we change more C code into Scheme code, that we'll end up unintentionally adding async ticks where they weren't before.

<wingo>very true

<wingo>doing scheme concurrency is trickier for that reason -- i guess we just disable asyncs in those regions, as you suggest?

<mark_weaver>it seems more robust to block asyncs whenever we lock a mutex that might be locked by arbitrary code.

<wingo>also vm hooks... tricky

<wingo>humm, you sure?

<wingo>i think that is the wrong order

<mark_weaver>wrong order?

<wingo>i mean, the question isn't who locks you -- it's what your critical section can do

<wingo>at least for me

<wingo>what locks can your critical section take, not vice versa

<wingo>maybe it's moot

<mark_weaver>I suppose one thing we could do is this: add a debugging preprocessor symbol that enables checks: whenever an async tick is run, check that none of the relevant mutexes are held and print an error if so.

<mark_weaver>where "the relevant mutexes" are the ones that might be locked "behind the scenes" by scheme code without the user's explicit knowledge.

<wingo>is that an enumerable set of mutexes?

<wingo>some might be part of objects

<mark_weaver>well, it could work as follows: instead of blocking mutexes, just increment a thread-local counter whenever such a mutex is held.

<mark_weaver>s/blocking mutexes/blocking asyncs/

<mark_weaver>and then check that counter whenver we do an async tick.

<wingo>that's how blocking asyncs works, no?

<mark_weaver>and when the debugging mode is turned off, the counter inc/decs are removed.

<mark_weaver>yes, that's how blocking asyncs works.

<mark_weaver>but, if a critical section calling asyncs is probably a bug, then this would alert us about that instead of silently papering over the problem.

<wingo>dunno, i am skeptical. you could be right but we have a bad history of different debug modes

<mark_weaver>one way or another, I'd like to do what I can to ensure that we don't have rarely-triggered deadlock bugs lurking.

<wingo>and such deadlocks are quite easy to debug

<wingo>you just get a backtrace and voila

<mark_weaver>yeah, but the problem is that sometimes a deadlock bug doesn't show itself until you happen to do something a little different during an async.

<wingo>agreed wrt evil of recursive mutexes of course :)

<mark_weaver>for example, I wrote some code a couple of months ago to fix (ice-9 popen) thread safety that worked by locking a mutex within the after-gc-hook (which is called from an async), and suddenly I run into deadlocks during "make check"

<mark_weaver>if I'm the one who runs into this problem, that's okay. but I don't want users to run into this problem when they decide to do something a little different.

<wingo>but we rarely run guile in debugging modes

<mark_weaver>okay, so maybe we should just block asyncs whenever locking the relevant mutexes.

<wingo>so it's not clear that a problem like this would be found in our normal course of development

<wingo>that's silly imo -- it is an extra procedure call and tls lookup for no benefit

<mark_weaver>or, we could set up a hydra job to run "make check" with the relevant debugging flags.

<wingo>a hydra job could work, yes

<mark_weaver>yeah, the tls lookups worry me, I confess.

<mark_weaver>it's not an extra procedure call though.

<wingo>ah i guess you're right

<mark_weaver>but tls lookups are very slow on the architecture I'm currently focused on, so I'm sympathetic to that.

<wingo>hey!

<wingo>i wrote a mail a while ago

<wingo>and just now flushed my queue

<wingo>we could have thread-local freelists for allocation

<wingo>check the guile-devel, you might find it interesting

<mark_weaver>I thought libgc already did that. no?

<wingo>it does, but in the vm we already have the current thread

<wingo>so you can get inline allocation

<wingo>and reduce the malloc-induced tls lookup overhead

<mark_weaver>ah, I see. that would be good to fix!

<wingo>it seems you can do it entirely with libgc's public api

<mark_weaver>nice!

<wingo>it cries out for a new subr calling convention, as you noted -- size_t (*scm_t_vsubr) (scm_i_thread *thread, SCM *argv, size_t argc)

<wingo>returning values directly on the stack...

<mark_weaver>looks good to me!

<mark_weaver>I wonder how hard it would be to use some existing static code analysis tool to determine the set of procedures that might do async ticks, and then alert us if one of these procedures is called while a relevant mutex is held.

<mark_weaver>I've never looked closely at any of those tools.

<wingo>should be doable

<wingo>mozilla has a whole pile of things like that

<wingo>they are actually moving from conservative mark-sweep to precise moving generational gc...

<wingo>terrifying

<mark_weaver>wow!

<wingo>they have a bot that you can ask on the channel, "can FooProc gc?"

<wingo>and it will tell you

<mark_weaver>haha, that's cool :)

<wingo>:)

<mark_weaver>I don't know if it's feasible, but occasionally I have dreams of an alternative Guile runtime that uses precise moving generational GC. of course, this alternate runtime would not be able to use C libraries as freely, etc. but for some programs it could make a big difference in performance.

<wingo>yep

<wingo>'twould be lovely

<mark_weaver>unfortunately, I see that we are deprecating and removing functionality that would be needed to do this.

<wingo>racket also managed to make the switch

<wingo>if we switched to a moving gc i would advocate a switch to c++ at the same time as an implementation thing

<wingo>raii is too hard to give up when your pointers can move...

<wingo>but both are very distant speculative things...

<mark_weaver>true, but it would be good to keep it in the back of our minds, so that we don't make the job harder later.

<mark_weaver>yeah, as much as I strongly dislike C++, I can definitely see benefits to us using some of the functionality, and perhaps even making it easier for C++ code to use Guile in an idiomatic way.

<mark_weaver>wingo: I see one problem with your proposed 'scm_t_vsubr' signature: how does the procedure know whether there's enough stack space to hold the return values?

<mark_weaver>wingo: also, regarding your commit that removes the async blocking in guardians.c, I think that at the very least we should have a machine-readable way of marking that we are _assuming_ that asyncs will never be called within a particular section.

<mark_weaver>otherwise the resulting system will be brittle, I fear.

<mark_weaver>one possibility is to have another set of lock/unlock-mutex macros that asserts that asyncs must never be called within that section.

<mark_weaver>then we could have that macro increment a counter when a debug flag is set, and the counter would be checked during async ticks, and we could set up a job on hydra to build with that debug flag set.

<mark_weaver>what do you think?

<mark_weaver>regarding 'scm_t_vsubr', one possible solution is to guarantee that there will be enough space for at least N return values, where N is some arbitrary chosen constant, and then provide a procedure that will expand the stack if necessary.

<wingo>mark_weaver: yes, agreed re return stack size

<tupi>hello guilers

<tupi>can't we use a list to feed receive ?

<tupi>like (receive (a b c) '(1 2 3) (+ a b c))

<tupi>[as an example, my need is more complex then that...]

<mark_weaver>tupi: no. a list is a single value.

<mark_weaver>if you have a list of values that you'd like to return, do (apply values <list>)

<mark_weaver>although it's better to just do (values 1 2 3) if you can.

<tupi>mark_weaver: no: for the sake of others reading the code and because it invloves some math, i'd like to associate list values to a variable name

<mark_weaver>tupi: have you looked at (ice-9 match) ?

<tupi>i can write (let ((.. (list-ref ...) i know

<mark_weaver>you can do something like (match <list> ((a b c) (+ a b c)))

<tupi>mark_weaver: no, i wanted to use receive :)

<tupi>oh i see

<tupi>(receive (a b c ...) (apply values list...))

<mark_weaver>match is really nice. civodul, wingo, and I have been increasingly using it in our code, and to good effect.

<tupi>a bit uggly but that will do for now

<tupi>example?

<tupi>how would write the above simple construct then?

<mark_weaver>(match <list> ((a b c) (+ a b c)))

<tupi>perfect

<tupi>tx

<mark_weaver>welcome!

<mark_weaver>see http://synthcode.com/scheme/match.scm for more

<mark_weaver>it can also be used to destructure records, and of course it can have multiple clauses like 'cond'.

<mark_weaver>going offline for a bit. biab

IRC channel logs

2013-11-28.log