IRC channel logs

2013-11-09.log

back to list of logs

<madsy>Quiet here tonight
<madsy>mark_weaver: This is what I ended up with for C++ gc allocation: https://gist.github.com/Madsy/7370895
<madsy>:)
<dsmith-work>madsy: Everyone is out partying because wingo merged the rtl branch into master.
<madsy>aha ;)
<gzg>Will Guilemacs likely be ready to act-as a drop-in replacement for Emacs' core, by ~2015?
<mark_weaver>we can't make predictions. it depends on how much work is done on it.
<gzg>mark_weaver: Maybe I'm being a bit too loose with the parameters. I'm operating on the assumption that if it gets the same amount of work/attention that it did during this year's GSOC, for the next two -- just trying to get a general gauge of how much work is in-fact left to do.
<mark_weaver>I'm sorry, I haven't looked closely enough at guile emacs to know how much is left to do.
<mark_weaver>bipt is the one to ask, and he's here on channel, but rarely speaks :)
<gzg>mark_weaver: That's fine, I probably need to look into it more myself anyways. I was just looking into the "Elisp Info" a bit, and the query arose. :^)
<gzg>I'll probably look into compiling from the repo and see what works. IIRC; one can eval guile s-exps, via Emacs' core now?
<bipt>gzg, it should be substantially ready by the end of the year; not necessarily ready for merging, but completely functional and with the old interpreter completely removed
*gzg is currently looking into lilypod's and similar syntax and is trying to figure out how a plain-text midi-sequencer would work.
<bipt>and yes, you can call scheme from elisp using the "eval-scheme" function
<gzg>bipt: I am very, very excited then. Thanks for working so hard on this! :^)
<bipt>gzg, np, it's been a fun project (-:
<bipt>if you want to test it, the wip-guile branch from http://git.hcoop.net/?p=bpt/emacs.git;a=summary is probably the most reliable at the moment
*gzg is now taking a more serious look at the prospect of refactoring his introductory text, to be similar in-scope to earlier stated "Elisp Intro".
<gzg>bipt: Cool, I'll clone it and hopefully have time next-weekend tio play with it a-bit. :^)
<gzg>~150mb and only ~22% cloned ... gotta love Emacs. :^)
<madsy>Is it a big deal if I have some C code that allocates memory the GC doesn't know about, when that memory isn't associated with any scheme object?
<madsy>It would make life a bit easier if it isn't.
<TaylanUB>bipt: Hi! Was wondering where you've been. :)
<TaylanUB>madsy: You can tell the GC about the memory you allocated manually, I think that makes things better.
<TaylanUB>Oh wait, you mean you don't want the GC to go through the memory you allocated, right ?..
<madsy>TaylanUB: Yeah. I mean, I have to set the limit somewhere, right? The GC doesn't need to know about the memory used by other libraries, etc
<TaylanUB>madsy: Do you know of scm_gc_register_allocation ? I'm not sure if it's exactly what you want ...
<madsy>I know of it, and I use it to some extent
<TaylanUB>madsy: Those "other libraries" should ideally be made to be Guile extensions. :)
<madsy>I'm embedding guile, I'm not extending guile itself
<TaylanUB>OK, you can still have them be extensions of the Guile you embed, so to say ...
<madsy>But say my application uses 50 MiB of memory that the garbage collector doesn't know about. What's the downside to that?
<madsy>And that used memory is fixed, doesn't change, and does not depend on any scheme smobs
<TaylanUB>The docs for scm_gc_register_allocation say "In general, Scheme will decide to collect garbage only after some amount of memory has been allocated. Calling this function will make the Scheme garbage collector know about more allocation, and thus run more often (as appropriate)."
<madsy>Okay, so it doesn't matter at all then
<madsy>The major memory use will be my smobs anyway
<TaylanUB>madsy: I don't know of any inherent down-sides, as long as you let the GC know about it via register_allocation, though I'm not very knowledgable on the topic; has someone stated otherwise ?
<madsy>No, just that it's generally a good idea to register allocated memory with the garbage collector
*TaylanUB reads IRC logs .. this had popped up before.
<madsy>But like I said, there are cases where you can't even use scm_gc_register_allocation
<madsy>Memory that libraries use internally for example
<madsy>So I'm not even using scm_gc_register_allocation. I use scm_gc_malloc whenever I allocate my own memory
<madsy>But libraries are left to their own devices
<TaylanUB>2013-10-22 23:07:48.006047192 +0200 mark_weaver yeah, scm_gc_register_allocation doesn't solve all the messy problems of finalizers etc, but it at least lets the GC know that memory is being allocated so that it will do a GC.
<TaylanUB>I don't know the details of said problems ...
<TaylanUB>mark_weaver: Is there a page or so where we can read more about this ?
<madsy>With scm_gc_malloc, mark said I didn't even need finalizers, so :)
<TaylanUB>madsy: the gc_malloc one arranges for the GC to manage the memory, so that should be the best way to go I guess, in general
<TaylanUB>madsy: Re. other libraries' internal malloc usage, did you look into the possibility of transparently replacing malloc with scm_malloc (which does scm_gc_register_allocation on the malloced memory) ?
<madsy>TaylanUB: In theory, that works on Linux and BSD but not windows
<madsy>I say in theory, because it's very risky. By say replacing global new and new[] in C++ and malloc/callc/realloc, you would even replace those functions in the display driver
<madsy>And even if that worked, you still end up with the dilemma whether you should replace those with scm_gc_malloc, scm_gc_malloc_pointerless or scm_gc_malloc
<madsy>scm_malloc*
<TaylanUB>I see
<davexunit>happy saturday, guilers.
<madsy>Hey davexunit :-)
<madsy>Is it generally safe to call API/standard library functions in finalizers?
<madsy>I need to know that calling C++ destructors will be safe in finalizers
<mark_weaver>madsy: yes
<madsy>Also, does libguile call exit() if you send ",q" to the REPL inside scm_shell() ?
<madsy>Because it exits the whole application instead of terminating the thread
<mark_weaver>madsy: ",q" throws an exception with 'quit' as the key. if uncaught, this will call 'exit' after unwinding the stack.
<mark_weaver>however, you can catch it if you like, and do something different.
<mark_weaver>calling (quit) or (exit) also throws 'quit', btw.
<mark_weaver>I should mention that there are some problems with finalizers that have been fixed properly in the master branch of guile (which will become 2.2), but are not quite adequately resolved in 2.0.
<mark_weaver>basically, finalizers really need to be run in a separate thread. anything less is not adequate in the general case.
<mark_weaver>but in 2.0, they are run in system asyncs, which guarantees that they are called at well-defined safe points within the C code, but could run at arbitrary times within Scheme code.
<mark_weaver>for example, as I recall, asyncs are run whenever the Scheme VM jumps backwards, and at various other times.
<mark_weaver>the issue is that some Scheme code could be in the middle of manipulating some data structure, so some data might be in an inconsistent state when the finalizer is run.
<mark_weaver>also, if the scheme code locks a mutex, that mutex could be locked when the async is run.
<mark_weaver>where needed, you can work around these issues by blocking asyncs during various critical sections.
<mark_weaver>anyway, like I said, in 2.2 finalizers are run in a separate thread, so all these problems go away.
<mark_weaver>(well 2.2 doesn't exist yet; I mean in 'master')
<mark_weaver>see http://comments.gmane.org/gmane.lisp.guile.devel/13835
<dsmith>mark_weaver, You saw wingo rm'ed the stack vm?
<mark_weaver>yes, very exciting news! :)
<dsmith>Indeed
<mark_weaver>I've been so busy with other things I haven't even had a chance to pull it yet, but I'm looking forward to playing with it and working on it some more.
<mark_weaver>(I need to add more VM instructions for fast fixnum primitives, and the fixing-letrec-reloaded stuff)
<mark_weaver>madsy: btw, I probably haven't adequatedly summarized the issues with finalizers in 2.0, so I definitely recommend reading that thread I linked to above.
<mark_weaver>I guess I should also mention that before Guile 2.0.6, finalizers were run in such a way that is much less safe.
<mark_weaver>(they weren't run within asyncs, but rather at arbitrary points where any allocation can happen)
<mark_weaver>s/allocation/gc allocation/
<mark_weaver>good morning, wingo! congrats on the removal of the stack VM from master. that's definitely worth a trip to a bar or two :)
<wingo>heya mark_weaver :)
<wingo>tx :)
***micro is now known as Guest17314
***Guest17314 is now known as micro`
***sneek_ is now known as sneek