IRC channel logs

2015-02-19.log

back to list of logs

<please_help>lloda`, I'm not using array-index-map! because that writes to an array, and even writing to a fake array causes an actual write to happen
***fridim is now known as fridim_
<lloda`>please_help: why does that matter??
<jmd>I have (define foo "xxx")
<jmd>and then (define ml '("aa" "bb" foo))
<jmd>I expected (write ml) to show something like ("aa" "bb" "xxx")
<jmd>But that is not the case.
<lloda`>' doesn't mean list, it means quote
<lloda`>you're quoting foo
<lloda`>(define ml (list "aa" "bb" foo)) or (define ml `("aa" "bb" ,foo)) would produce what you expected
***sigmundv_ is now known as sigmundv
<please_help>lloda`: because it slows down everything.
<lloda`>compared to what?
<please_help>compared to not having to do a write
<please_help>hopefully that meant compared to (increment), but that's probably not true at all
<lloda`>you realize that just creating the index list is way more expensive than that write?
<please_help>I didn't know that when I wrote (increment)
<please_help>now I do ;0
<please_help>)
<lloda`>I don't mean what you do in (increment). I mean the index list created in array-index-map! internally for each iteration
<please_help>that said, I also don't quite see how to use array-index-map! to describe array-ec, and if I don't have an srfi-42 compatible construct, I have to forego the other srfi-42 things.
<please_help>ah
<lloda`>if I gave you an opaque function that did what array-index-map! does but without the write, you'd never be able to measure the write, not even for a trivial body
<lloda`>what do you need for array-ec? current-element / next-element, something like that?
<please_help>current index
<please_help>current element is provided as the last item in the parameters of array-ec
<lloda`>do you use the index for anything else than indexing?
<please_help>I don't
<lloda`>you don't need the indices, then
<please_help>how so?
<lloda`>suppose you'd only need to iterate in the first dimension
<please_help>also, I support an extension that will copy an entire array onto the array to be returned, if that's the current element provided to arracy-ec
<lloda`>if you can be more specific (is that a function, what parameters would it take, behavior) then you can write to the list re: extension
<lloda`>ok, suppose you iterate only in the first dimension
<lloda`>then you see how to obtain the first subarray and the rest of the subarrays, right?
<please_help>(array-ec) always receives a type, a shape, unspecified "things" to be sent to do-ec, and finally the result from the ec construct. If that result is an array, then array-ec copies it via array-copy! unto the array to be return, starting on the Kth dimension from the end, where K matches (array-rank result)
<please_help>to get the subarrays I can just use make-shared-array since it's going to be contiguous by construction, right?
<lloda`>you can use make-shared-array in any case, it doesn't matter if it's contiguous
<lloda`>e.g.
<lloda`>(define (array-first a)
<lloda`> (apply make-shared-array a
<lloda`> (lambda x (cons 0 x))
<lloda`> (cdr ($ a))))
<lloda`>(define (array-rest a)
<lloda`> (apply make-shared-array a
<lloda`> (lambda x (cons (+ (car x) 1) (cdr x)))
<lloda`> (cons (- (tally a) 1) (cdr ($ a)))))
<lloda`>
<lloda`>sorry, $ = array-dimensions
<please_help>would it become costly to (array-first (array-rest (array-rest ... (array-rest arr)))) ?
<lloda`>make-shared-array is costly all by itself, unfortunately
<lloda`>but you pay this cost per iteration. Current Guile has no other way
<lloda`>to create subarrays.
<lloda`>if you mean compared to second/third/etc directly, since you carry the rest along, it's only 2 make-shared-array calls per iteration instead of 1
<lloda`>anyway, this is just to show that you can loop over an array in this way.
<lloda`>so you need to loop over > 1 dimensions
<lloda`>do what mark_weaver suggested: flatten the dimensions where you loop, but not the rest
<please_help>I mean the chaining of array-first and array-rest
<lloda`>not sure if I understand, each iteration would pass (array-rest) to the next, you don't carry the array-rest operations along, just the result
<please_help>but array-rest is actually a wrapper around an array that translates indices. (array-rest (array-rest arr)) would then translate indices for the index translation of arr?
<lloda`>nonono that's not how it works
<lloda`>each array carries a set of strides and dimensions, what some people call a 'dope vector'
<lloda`>these are just numbers, and are used to map the indices to a linear address
<lloda`>when you call make-shared-array, what you do is create a different set of strides and dimensions, and that's it
<please_help>and the (lambda (my in dices) (list new in dices)) is not used to translate indices?
<lloda`>there's no continuous translation of indices or translator function hanging around nor anything crazy like that
<lloda`>yes, but only to compute those strides, once make-shared-array returns, that lambda isn't called ever again
<please_help>ah
<please_help>then can we manually specify stride and dims instead?
<lloda`>no :-( Guile doesn't have an interface to do that.
<please_help>yeh, arrays are kinda lacking it seems.
<lloda`>the only three functions to create shared arrays are, make-shared-array, transpose-array and array-contents, there's nothing else.
<lloda`>you can do absolutely any shared array with make-shared-array, so it that sense it's an elegant interface
<lloda`>it's just heavy
<please_help>yes
<lloda`>wrap it in something simpler and then you can just complain that it's slow, but first show the profile :-)
<please_help>;)
<lloda`>anyway, my array-ec friendly suggesting is to flatten just the looping dimensions. Then you can loop with a single index.
<lloda`>to do that without copying you do need the looping dimensions to be contiguous, but just those.
<please_help>but with my array-return extension, I think I can't do that
<lloda`>you mean that thing where the array grows?
<please_help>if I did, I'd have to array-set! per element rather than array-copy! the whole array
<lloda`>I don't see that, the dimensions where you copy don't change
<please_help>no, I mean that if the current value of an array-ec is an array, this array is copied verbatim unto the complete return array of the full ec
<lloda`>and you erase what you had there before?
<please_help>no, there is nothing before
<lloda`>and the iteration ends then?
<please_help>the complete ec iteration ends when the entire array is filled, be it via the array-copy! branch (which jumps K elements forward) or via the array-set! branch (which jumps 1 element forward)
<please_help>so a single call - that which I'm assuming you're calling iteration - would simply do array-copy! and then end, if an array is returned, or array-set! and then end, in any other case
<please_help>I have to go now
<please_help>thanks for the tips
<lloda`>you're welcome, I still don't get how array-ec works I'm afraid!
<please_help>a continuation is passed to the iterator part. The iterator part communicates the current-result to the continuation, which does some setting (e.g. cons the result to a list, gather it in a sum, set the current array element to that result...) and resolves to do-ec. The thing loops back and forth until the iteration end criterion is called, at which point array-ec returns the completed result.
<please_help>the continuation involves the array-ec macro proper
<dsmith-w`>Morning Greetings, Guilers
***dsmith-w` is now known as dsmith-work
<mark_weaver>please_help: what is your 'array-return' extension? How does that conflict with iterating over a 1-dimensional view of the array and then returning the original (multidimensional) array?
<mark_weaver>the idea is that even though 'make-shared-array' is heavy, you should hopefully only need to call it once.
<stis>evning folks!
<wingo>greets :)
*stis is staring to get nontrivial codes from swi prolog working ontop of guile!
<stis>err starting!
<stis>a boolean sat solver actually!
<wingo>:-))
*wingo going to implement add an interface to gdb so that you can unwind frames from guile
<wingo>instead of loading an .so for the jit reader interface
<stis>great :-), looks really really promissing!
<wingo>it is really fun to grovel things from gdb in guile
<wingo>i can traverse all of the v8 heap, find all of its functions, know the source location for any pc, all without calling into the inferior
<stis>coool!
<wingo>now i just need to be able to programmatically tell gdb how to unwind frames :) i can unwind them myself but i need to hook into gdb's backtrace so that it unwinds correctly
<stis>A question about 2.2, is there any suspicion that vhashes or vlists are performance bottlenecks in any part of the 2.2 compiler?
<mark_weaver>of course they are
<mark_weaver>wingo: sounds great!
<mark_weaver>stis: I proposed implementing vlists/vhashes in C a long time ago, but civodul objected, and he's probably right.
<stis>mark_weaver: as I said in the potluck description, It can be good to know how much better things can be, the C version is quite optimized
<stis>kind of knowing the goal.
<mark_weaver>stis: I was really surprised that you code doesn't even work on 32-bit.
<mark_weaver>*your
<stis>it's a toy example, with a little effort I can make it work on 32 bits
<stis>I just have other more burning tasks ahead of me.
<mark_weaver>well, it's very unlikely that we can use any code you write anyway, for other reasons which we have discussed before.
<stis>yep
<stis>I'm not implying that you use the code base in guile, just play with it to see what can be gained
<stis>that's my contribution to the potluck
*davexunit is interested in how an rrb-tree would perform in relation to vlists
<davexunit>I have no potluck dish as of yet, bummer.
<davexunit>stis: it would be interesting if you had some performance stats to share with the c vlist implementation.
<stis>davexunit: I took a loop that brought back the first 1000 numbers when installing a vhash of 10000 consequtive ones, 8sec on guile 2.2 for 10000 such sequences, 2.0 with the C code version.
<davexunit>stis: thanks
<stis>This simulates moderately extra work needed to lookup values.
<davexunit>our persistent vectors and hashes are different than Clojure's and I wonder how the compare.
<stis>If th elayout of the hash is optimal you may find that the two approaches is equal due to the call overhead of the C functions
<davexunit>they compare*
<davexunit>vlists aren't thread-safe, which is a bummer.
<davexunit>Clojure's, of course, are thread-safe.
<stis>you can get a simple variant of thread safeness if you stop reusing the vlist at the creation of a new thread
<stis>If that is something that doesn't happen often you can still get speedy datastructures.
<stis>It's the simplest variant to get some thread safe properties of the vhash/vlist
<stis>I do have it inside Civilist, a bit untested though.
<stis>for some workflows some kind of functional tree might be better, mark_weaver has some ideas here.
<davexunit>stis: is vlist not implemented as a tree?
<davexunit>I need to re-read the manual about this
<stis>no, it's based on vectors and special pointer treatment into those, can be a boon, but also a bummer.
<davexunit>ah, okay.
<stis>a very bad characteristic is backtracking because when you restart further down the vlist, you will create a new vector pointing to the first and so you can get a mess of small vectors pointing to each other and loose the nice lookup properties
<stis>I use vhashes as a basement for associating logical variables with values in guile-log, there I needed to reset the vhashes
<davexunit>maybe we should bring some of ijp's pfds into core guile
<stis>I love functional datastructures, that is a great idea!
<davexunit>I believe there are more-or-less ports of Clojure's popular data structures there.
<davexunit>Clojure's persistent vectors are implemented as 32-way branching trees.
<davexunit>the Elm language uses a variant of it called the "relaxed radix balanced tree"
<davexunit>if I could figure it out enough to implement it, it would make a good potluck dish. :)
<wingo>davexunit: they are like intsets
<wingo>or intmaps
<wingo>in master
<wingo>clojure maps are more interesting to my mind tho
<wingo>more interesting than vectors, i mean
*wingo -> z tho :)
<davexunit>wingo: thanks