<please_help>To export constants from some C library to be usable in guile, is the only option either to redefine the constants in guile or to create an ad-hoc lib with static variables having the constants' values? <davexunit>please_help: preprocesser defined constants? #DEFINE? <davexunit>yes, just create Scheme variables that have the values of those contstants. <davexunit>the issue is that those constants are preprocessor macros, which are simply substituted at compile time. <ijp>in other words, it's hard for guile to see things which don't exist <davexunit>in my efforts to make rendering fast, I want to batch all those little bullet sprites into one big buffer to send to the GPU. however, I must take care to avoid floating point math as much as possible. <davexunit>I'm thinking that if the coordinates composing a single sprite are stored in a bytevector, I could simply copy it into the big buffer without suffering the boxing/unboxing penalty of floating point ops. <davexunit>I need to do this batching every frame, so it needs to be as fast as possible. <ijp>I'm not sure I see the question <davexunit>the question is: is it possible for me to avoid the big penalties of boxing/unboxing floats if I stuff everything in a bytevector? <davexunit>and not using bytevector-f32-ref (or whatever it's called) <ijp>I don't see why it wouldn't avoid it <davexunit>I have some other issues to sort out, but hopefully just copying mesh data from a small buffer to a big buffer won't be a problem. <davexunit>I also need to transform all the points, but maybe that can be another job for the GNU scientific library. ***heroux_ is now known as heroux
<davexunit>please_help: I can probably do that, too, but I guess I'd need to make a new attribute buffer to hold the matrices. <davexunit>that might work out just fine, I will try that before transforming the vertices on the cpu. *nalaginrut still don't see anything fresh in Clojure, except for bigger&active community <please_help>do I have to explicitly add \\0 to a (string->utf8 ...) generated bytevector before handing its (bytevector->pointer ...) over to a C function? <please_help>Any quick way to force a string to be represented as c-terminated? <nalaginrut>the string will be added 0 in the end when it's converted to bytevector <nalaginrut>mark_weaver: I was wrong? I saw it's added 0 in latin1_to_u8 <mark_weaver>nalaginrut: some of the C functions in our API automatically add a NULL terminator when asked, but string->utf8 certainly doesn't (and shouldn't) <mark_weaver>please_help: the NUL terminator is not conceptually part of a string. it is essentially part of the C data format for strings. <please_help>nalaginrut: I tried it and it's not true apparently... <please_help>mark_weaver, I know, but I had hoped that (system foreign) did something about it. Is string-append the cleaner way to fix this up? <mark_weaver>please_help: the simplest solution is probably to append "\\0" the string before converting to a bytevector. <mark_weaver>though I agree that we should probably have a nicer way to pass strings to C functions via the dynamic FFI. <mark_weaver>I suppose we could add a keyword argument to 'string->utf8', something like (string->utf8 str #:add-nul? #t) <nalaginrut>anyway, I still confused with this line "u8_result[n] = 0;" in latin1_to_u8, I thought it's added null-end <mark_weaver>otoh, (string->utf8 (string-append str "\\0")) seems clear enough to me, so I'm not sure there's a need. <mark_weaver>nalaginrut: yes, that adds a terminator, but the terminator is not included in the bytevector that is then produced in scm_string_to_utf8 <nalaginrut>mark_weaver: hah, I should trace the whole process, thanks for explain ;-) <nalaginrut>mark_weaver: I guess it's because the allocated null-terminated string won't be used directly, but it'll be memcpy to a new allocated bytevector, and it won't count the last index <mark_weaver>if you think about it, it would be crazy and wrong for us to include a NUL in the bytevector <mark_weaver>suppose you want to write this to a socket or file. of course you don't want to include the \\0 in there. <mark_weaver>suppose you want to iterate over the bytes in the UTF-8 representation. of course you don't want to iterate over the NUL byte. <mark_weaver>the NUL byte is not a character in the string, it's as simple as that. <mark_weaver>it's conceptually outside of the string. it is only a part of the conventional representation for strings in C. <nalaginrut>I just consider if it's efficient we allocate twice in this conversion <mark_weaver>nalaginrut: right, that's a fair point. we could potentially make (string->utf8 str #:add-nul? #t) more efficient than (string->utf8 (string-append str "\\0")) <nalaginrut>mark_weaver: yes, that's good point to avoid twice allocation, but I'm not talk about this ;-D <nalaginrut>scm_to_utf8_stringn will allocate the first time, then make_bytevector will be the second, and memcpy to the latter <nalaginrut>maybe we can allocate it with gc_malloc in the first time to avoid the second <mark_weaver>we could certainly do better there, though we'd have to make a first pass over the string to find its UTF-8 length first. <mark_weaver>I think we'd have to determine the UTF-8 byte length, then allocate the bytevector, and then convert to UTF-8 directly into the bytevector. <nalaginrut>at present, if we add "\\0" manually, it's tri-allocation <nalaginrut>thanks, efficiency on string operations are important for me, you know ;-P <mark_weaver>first, I think we need something like scm_utf8_length that returns the length in bytes of the UTF-8 representation of a string. <mark_weaver>that would be a useful addition to our API, for the sake of getting strings out of scheme efficiently from C code. <mark_weaver>and then we use that to eliminate an allocation+copy in 'string->utf8' <mark_weaver>indeed, you are right. the work is already mostly done :) <nalaginrut>yeah, I think there're large room to optimize, but sometimes we need cases for hint <nalaginrut>mark_weaver: you're expertise on strings & ports, please do it for me <mark_weaver>thanks for helping me think it through and finding those existing C functions :) <please_help>I think the best option would be to add a (string->c-string str encoding) to (system foreign) directly. No impact on bytevectors themselves or the scheme-side bytevector interface. <nalaginrut>I prefer implement string->c-string based on bytevectors to reduce some orthogonality. Or we'll have too many specific operations ***Shozan is now known as SHODAN
*wingo just found a goops bug that affects 2.0! <civodul>i guess the review you're doing will allow to discover many dirty things ;-) <wingo>yeah, i was trying to get g-wrap to work with master and ran into something deeeeeep <civodul>g-wrap is probably another good test for GOOPS... <wingo>slot-ref is slower than it was in c but faster than 2.0 <wingo>and initialize-object is similar i think *wingo grumbles about statprof changes <wingo>civodul: statprof is significantly different internally, i would appreciate a look to master before making statprof changes :) <sneek>rlb was here Oct 29 at 07:26 pm UTC, saying: (for projects that still keep ChangeLogs...). <sneek>Welcome back civodul, you have 1 message. <civodul>the changes were trivial, but it's always annoying when you get conflicts i guess <wingo>well it's tricky to merge, and also adds consing to the main loop of the profiler <wingo>statprof in master does a proper outer cut of stack traces :) <civodul>returning the body's return value adds consing? <civodul>i guess i really need to look at the new statprof, because it's far from obvious how that could happen :-) <wingo>you cons the values in to a list at each loop iteration <wingo>(call-with-values (lambda () ...) (lambda result ...)) <- result is allocated <wingo>civodul: i am merging stable-2.0 to master, so i can take care of it <wingo>civodul: should we use siphash in 2.2? <civodul>wingo: no idea about siphash, it would need to be discussed on the list i guess <wingo>civodul: cool, only trick is that master uses a trampoline to get a nice stack cut <wingo>that's probably a good fix for stable-2.0 tho <wingo>ffs master is failing g-wrap tests due to that damn "aggregating" test again <wingo>which looks to me like specious reasoning <wingo>i was totally convinced then but like mark_weaver i don't understand how i was <wingo>the issue appears to have been use of scm_gc_free <wingo>at least, removing scm_gc_free calls make this wct issue go away <wingo>i imagine there are ways that you could mark an object that had been explicitly freed <wingo>anyway we don't guarantee anything about free orders <wingo>and those can happen in finalizer threads, etc <wingo>o/~ steaming piles of bugs la la la o/~ <please_help>Is there a way to set a custom destructor for records? <davexunit>please_help: you can use a 'guardian' to protect things from the GC, and then some code you right can remove things from the guardian and run some finalizer procedure. <wingo>please_help: what version of guile? <wingo>"Finalizers are tricky business..." <davexunit>I have a pretty good setup now with finalizers for OpenGL objects. <davexunit>that calls the necessary glDelete* functions, and the guardian is only flushed from the main thread with the GL context, so I think I've successfully avoided the pitfalls. <wingo>scm_gc_free considered harmful <davexunit>is there a way to call the profiler from scheme, rather than using ,profile at the REPL? <davexunit>I have some code that I want to profile that I cannot run via the REPL as it's in the core event loop of my application. <davexunit>I was searching the manual index for "profile" and only the metacommand was coming up. <please_help>So, for guardians, I have to manually (destroy-guardian! ) after doing my finalization? <davexunit>please_help: do you really want to destroy a guardian, though? <please_help>isn't it the correct way to indicate an object is ready for standard collection? <davexunit>no, that destroys the guardian, meaning that it can't be used to protect any more objects from the GC. <please_help>then, pumping the guardian of its objects manually after doing custom finalization? <davexunit>first you pump the guardian to get an object out, if there is one. <please_help>there's no way to register a finalizer function to be used by the gc without using foreign objects? <mark_weaver>please_help: the problem with finalizers is knowing when it's safe to run them. <mark_weaver>the only reasonable options I know of are (1) run finalizers in a new thread, which may require adding thread synchronization to your data structures that might not need them otherwise, or (2) let the user decide when is a safe time to finalize a particular set of objects. <mark_weaver>when it's a good time to finalize, you ask the guardian for an object that is ready to be finalized. ***wingo_ is now known as wingo
<linas>fyi: a guile-2.2 maybe-bug report follows ... <linas>I hit guile-2.2 with maybe 6 or 12 threads, not sure, doing misc stuff. <linas>I see this printed "madvise failed: Cannot allocate memory" <linas>but it seems to work fine, otherwise. <linas>gc-stats says: heap-size=138M heap-free-size=140M <linas>but get this: heap-total-allocated = 36 gigabytes !! <linas>like I said .. it all seems to wrk fine ... <linas>top shows that virt=6GB and resident=250MB <linas>and its stable that way, from top point-of-view -- virtual and resident don't change <linas>I think I misunderstood what heap-total-allocated means <linas>It doesn't mean "total right now" <linas>err .. well .. I still get "madvise failed: Cannot allocate memory" printed every 5-10 seconds ***dje is now known as xdje
<mark_weaver>linas: I think messages are being generated from 'return_unused_stack_to_os' in vm.c <mark_weaver>it seems that attempting to call 'madvise' with MADV_DONTNEED is failing with "Cannot allocate memory" for linas <mark_weaver>(unless there is some other code that generates the same error message) <mark_weaver>seems like a strange error to get when telling the kernel "we don't need this memory anymore" <mark_weaver>linas: you might want to put a breakpoint on that 'perror' call to see the details of the failing 'madvise' call. or maybe strace would be sufficient. <mark_weaver>paroneayea: (srfi srfi-26) provides 'cut' and 'cute', which are similar <wingo>mark_weaver: yes that sounds like a good theory -- it's the only place there's an madvise and there's a perror afterwards <wingo>linas: i would be interested to know how madvise is failing :) <wingo>linas: also are you on gnu/linux, if so which distro <civodul>madvise(1) reads: ENOMEM Addresses in the specified range are not currently mapped, or are outside the address space of the process. <wingo>sneek: later tell tupi there were problems in guile and g-wrap, but now for me guile-gnome works fine with master <mark_weaver>wingo: we'll need to fix guile-gnome on stable-2.0 also. <wingo>mark_weaver: does it not work there? there was a goops bug that prevented some things from working in g-wrap, but i fixed that today <mark_weaver>oh, well, I haven't actually checked in the last day or so. <wingo>mark_weaver: that was the goops bug <mark_weaver>at my request, daviid did a git bisect to discover where it broke <mark_weaver>I suspect that's a different bug than the #:init-value thing, but I'm not sure <wingo>mark_weaver: yes it was the same bug, methinks <wingo>commit 48ad85fb56bc022ac10f42cf07b5657d75b5b696 <wingo>Author: Andy Wingo <wingo@pobox.com> <wingo>Date: Sun Apr 27 11:02:35 2014 +0200 <wingo> Fix foreign slot initialization and access <wingo> * libguile/goops.c (scm_sys_initialize_object): Refactor initialization <wingo> so that we don't ref uninitialized slots before initializing them. <wingo> This allows foreign slots, whose initial value is 0, to be initialized <wingo>i didn't understand why we were reffing slots to see if they were SCM_UNBOUND before initializing them <wingo>turns out it's because of #:class and #:each-subclass allocation <wingo>but the old code made it impossible for #:init-value or #:init-thunk to work for "raw" slots <wingo>well, yay for fixing the regression <wingo>now that i understand things i can fix the accessor bug i guess <wingo>no prob, sorry i took so long, at least we can do a 2.0.12 now :) <mark_weaver>you are certain that guile-gnome now builds against current stable-2.0? <wingo>mark_weaver: confirmation is best <wingo>currently fixing lingering master issues <wingo>sneek: later tell daviid when you check, please verify that g-wrap's test suite passes for you 