IRC channel logs

2014-10-02.log

back to list of logs

<davexunit>is it considered bad style to export type identifiers?
<davexunit>I have a <foo> type, and it's convenient to use the type identifier for pattern matching
<davexunit>but I don't usually see such things exported from modules
*davexunit is having trouble finding a way to efficient matrix math
<davexunit>to do*
<davexunit>4x4 matrices, trying to do hundreds of multiplications per second with an immutable matrix type. I guess I might have to give up on immutability.
<rlb>davexunit: double x[16] and memcpy? ;>
<rlb>though the mallocs would be somewhat expensive...
<rlb>(i.e. 4x4 double array should be pretty fast (offhand, I'd think))
<davexunit>I'm using a 4x4 f32 typed array currently
<rlb>and as long as you're only talking about hundreds/sec
<davexunit>performs horribly because each multiplication allocates a new matrix
<davexunit>my goal is to have a functional graphics API, so I don't want to have imperative matrix multiplication.
<davexunit>maybe behind the scenes it can be that way, but to the user it needs to be functional.
<rlb>hmm, well, at least here, unless the optimizer is going haywire, mallocing/freeing 200M 16-double arrays takes about 12s.
<rlb>but to get that, of course, you may have to involve C
<davexunit>I'm just surprised at just how slow my code is, heh. if I comment out the matrix multiplication that happens when rendering the scene graph, I go from 25FPS to 60.
<davexunit>rlb: oh, well that's C.
<davexunit>I'm after a pure Guile solution.
<rlb>davexunit: fair enough -- though I often like to double-check what I should be able to expect from the hardware so I know how I'm doing ;>
<davexunit>thanks for the quick benchmark, though.
<rlb>davexunit: I'm very rusty wrt scheme arrays, etc. -- but wonder if using a flat array might make any difference.
<rlb>i.e. if it's not doing that internally for 2d already
<rlb>(since array of array will probably be much more expensive)
<rlb>this also sounds like a place where "transients" might help (in clojure parlance -- though I haven't used them much yet myself)
<rlb>and the concept's not really relevant to normal guile/scheme since normal guile/scheme is already quite mutable
<davexunit>I think that concept could come in handy, though.
<davexunit>I try to write code in a purely functional style.
<davexunit>this transients thing might be worth looking into more
<davexunit>but I'm sort of doing this already, my matrix procedures mutate an array, but the user never does.
<zacts>hi guilers
<zacts>darn davexunit is gone
<mark_weaver>hi zacts
<zacts>hi
<zacts>I wonder if it would be difficult to parse POSIX sh with guile
<zacts>and then I guess the next step would be a guile shell, but not an scsh clone
<lloda>hello mark_weaver
<lloda>I tried using c&e etc. to run some of arrays.test under the compiler, but although the tests produce errors, the test suite doesn't see a failure.
<lloda> http://lists.gnu.org/archive/html/guile-devel/2014-09/msg00103.html
<lloda>and the patch here
<lloda> http://git.savannah.gnu.org/gitweb/?p=guile.git;a=commitdiff;h=ea342aa6f7fd4a03dc0cc4bde8e6746c1daf083e
<lloda>would be great if you could check this.
<mark_weaver>lloda: I'm looking into it now
<mark_weaver>lloda: well, it seems to work properly for me. when applying just the first of your three patches, I see 26 errors in arrays.test, and the final tally also shows 26 errors. the "make check" fails. what happens on your system?
<mark_weaver>this is using v2.1.0-150-gea342aa
<mark_weaver>(your lloda-array-support branch minus its last two commits)
<mark_weaver>lloda: note that "failures" and "errors" are counted separately.
<mark_weaver>so I end up seeing 2 failures and 26 errors
<mark_weaver>sneek: later tell lloda: regarding the test suite problem you reported: it seems to work properly for me on ea342aa (the first commit from lloda-array-support). I see 26 errors reported during the tests, and the final tally also shows 26 errors. note that "failures" and "errors" are counted separately.
<sneek>Got it.
<mark_weaver>sneek: botsnack
<sneek>:)
<mark_weaver>sneek: later tell lloda: regarding that first commit: I'd prefer to improve c&e to handle internal definitions rather than evacuating the internal definitions in the actual tests. I'll look into it.
<sneek>Okay.
<civodul>Hello Guilers!
<lloda>mark_weaver: you're right that the errors are reported. But then I don't understand what failures are. The two failures are in the same coverage.test... Anyway, thanks for looking into it.
<sneek>Welcome back lloda, you have 2 messages.
<sneek>lloda, mark_weaver says: regarding the test suite problem you reported: it seems to work properly for me on ea342aa (the first commit from lloda-array-support). I see 26 errors reported during the tests, and the final tally also shows 26 errors. note that "failures" and "errors" are counted separately.
<sneek>lloda, mark_weaver says: regarding that first commit: I'd prefer to improve c&e to handle internal definitions rather than evacuating the internal definitions in the actual tests. I'll look into it.
<lloda>thanks sneek
<lloda>wrt to the internal definitions, what if that's precisely what one wants to test?
<lloda>wrt to/wrt
<dsmith-work>Morning Greetings, Guilers
<paroneayea>davexunit: finally had a chance to read ice-9 match's docs this morning
<paroneayea>wowee
<davexunit>paroneayea: yeah it's awesome.
<davexunit>match-lambda and match-lambda* are not documented, but they are awesome, too.
<madsy>Agh.. whoever designed iconv without implementing a way to query the output buffer size deserves be shot :/
<mark_weaver>madsy: I'm not sure I understand. are you talking about the C interface? it takes pointers to size_t's, one for the input buffer and one for the output buffer. what's missing?
<mark_weaver>each of those size_t* has the size of the buffers upon entry, and the remaining size after return.
<mark_weaver>lloda: in our test suite, a failure is when the test didn't give the expected result. an error is when an unexpected exception was thrown.
<mark_weaver>lloda: I'm not sure I understand your comment about internal definitions. your first patch moves a bunch of currently-internal definitions to the top-level in the actual tests, to work around limitations in c&e. I'd prefer to enhance c&e to handle this properly, possibly by making 'pass-if', 'pass-if-equal', and 'pass-if-exception' into syntax parameters, which is anyway far more robust than the current strategy.
<mark_weaver>testing internal definitions is more fundamental, and handled elsewhere.
<mark_weaver>lloda: but you can always test internal definitions some more by putting them within a (let () ...) within the 'pass-if*'
<madsy>mark_weaver: There's no way to query iconv for *how big* the output buffer must be in order to fit all the characters generated by the conversion.
<madsy>mark_weaver: The output buffer size you pass in is just the size of the buffer you allocated. You are not at all guaranteed that it will be large enough.
<mark_weaver>madsy: ah, I see. well, if you think about how that would be implemented, you basically need to just do the conversion anyway.
<madsy>Well? They could have done the conversion without writing out the result. So you could have passed NULL and 0 to the out pointer and out size arguments.
<madsy>Then returned the number of bytes generated
<mark_weaver>well, it would be far less efficient to do it that way than using some large (per-thread?) temporary buffer and then copying it to a target buffer of the right size.
<mark_weaver>the way you're proposing would still require doing most of the logic of the conversion twice, even if it's written only once.
<mark_weaver>and furthermore it would require either (1) essentially doubling the amount of object code in iconv, one copy of code that writes the output and another that just counts bytes, or (2) slow down the single copy of code by having it frequently check whether it should be counting or writing.
<madsy>I find it questionable that it would so much slower that it would be a problem. A proper implementation is already pretty slow because you have to take E2BIG errors into account and then resize the buffer.
<madsy>Any output buffer length other than being equal to the input buffer length, is entirely arbitrary unless you specifically know about the charsets used for the conversion.
<madsy>And gambling on that "no one will ever call this functions with an array larger than N bytes" will surely bite you in the rear eventually.
<madsy>-s
<mark_weaver>well, of course you have to dynamically expand this pre-thread temporary buffer if iconv reports that the result didn't fit. and then you can just keep it at that new size and soon it shouldn't happen much anymore. no guessing required.
<madsy>Even then iconv doesn't tell you how much longer the array must be. Is twice the size enough? Four times the size? Who knows.
<madsy>So it can fail multiple times.
<mark_weaver>I can sympathize that it would be convenient to have the API you're asking for, but it would be fundamentally inefficient.
<mark_weaver>if you multiply the buffer size by a constant factor each time, the amortized time is linear in the size of the output.
<mark_weaver>a factor of 2 is the usual choice, of course.
<mark_weaver>if you tried to implement iconv, I think you'd appreciate the other side of this argument.
<davexunit>I encountered a similar thing when writing a wrapper around zlib. there's no way to know what the uncompressed output size will be, so I make an initial guess and double the buffer size and try again if it doesn't fit
<mark_weaver>you could make the same complaint about sprintf. it would be nice to know ahead of time how big the output would be. but that would require doing the formatting twice. same deal.
<mark_weaver>in theory, you could try to duplicate all the code in sprintf, but optimized to compute only the output size, but just think about what a fragile mess that would be, and a likely source of security holes. it's just not workable.
<madsy>My point of view is that designing functions which doesn't let you request the necessery buffer size required, promotes lazy/sloppy use of buffers.
<madsy>And I agree, sprintf and snprintf are guilty of this too.
<mark_weaver>okay, I've said my piece. I really have nothing new to add here.
<madsy>Yeah, same
<dsmith-work>mark_weaver: ISTR some sprintf-like thing that did exactly that (just return the size on an output buffer of NULL). Probably some windows api.
<lloda`>mark_weaver: I don't like that the -c&e versions of pass-if etc. act differently from the defaults. But I don't think much about this stuff beyond making it produce the error, so I appreciate that you take care of it.
<mark_weaver>lloda`: well, isn't the whole idea of c&e making those things act differently than the defaults? (which is to run only in the interpreter)
<mark_weaver>I agree that we should aim to make them act the same way in all other respects though :)
<lloda`>mark_weaver: well that's something I don't understand either. Why is the default in the tests to run only in the interpreter, when the default in the REPL or when running scripts or what have you is to run in the compiler?
***dje__ is now known as xdje
<rlb>anyone know anything about the status of guile-gnome-platform? https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=761211
<mark_weaver>daviid is the best person to ask about that at present.
<mark_weaver>it definitely works with 2.0 though
<mark_weaver>I've built it myself in the past
<mark_weaver>it hasn't yet been updated to gtk+3 or gnome-3 though. still on version 2 of those.
<mark_weaver>rlb: ^^
<mark_weaver>daviid is on channel here quite a bit.
<rlb>mark_weaver: ok, thanks
<rlb>(time is getting short wrt the freeze)
<stis__>heay guilers, got tabling to work with disjunctions e.g. when there is multiple solutions.
<rlb>and I'm likely to ask for some of the laggards (if there are only a few, and they don't appear to be maintained) to just be removed from debian rather than keep guile-1.8 in
<stis__>check out for http://paste.lisp.org/display/143919 to see the cool stuff.
<rlb>Though I think it's also possible we'll have one or two packages that we don't want to lose, and end up keeping guile-1.8 instead.
<rlb>(but no one should *assume* that)
<mark_weaver>sneek: later tell daviid: rlb asked about the status of guile-gnome, for purposes of getting it updated in debian to use guile 2.0. time is short wrt the freeze. can you get in touch with him about it?
<sneek>Will do.
<mark_weaver>rlb: lilypond and texmacs are the most prominent laggards. I know that lilypond is being worked on, but I'm not sure it will stabilize in time for the freeze. I have no clue about texmacs.
<rlb>mark_weaver: texmacs may have to be booted for other reasons?
<rlb> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=711383
<rlb>but I know nothing other than the headline there
*rlb scrolls down now...
<civodul>at the GHM 2013 a TeXmacs developer expressed interest in porting to 2.0
<civodul>and had already started working on it
<civodul>not sure what happened, then
<civodul>Fran├žois Poulain, that is
<tadni>rlb: I suspect that guile-gnome will get more love, due to things like the GNU Distro and the probable GUI install written for it eventually.
***ozzloy_ is now known as ozzloy