<daviid>though thinking a bit more about this, for praticle purposes, maybe I should provide an unref optional #:recur kw, that would, if #f, performed a 'deep unref' ... <daviid>at users own risks - instances that have been unreferenced are unusable, and meant to be gc'ed - any attempt to run any gobject method on thse will raise an exception, as expected <daviid>*an unref optional #:recur kw, that would, if #t, performed a 'deep unref' ... <apteryx_>one could simply write (if e1 e1 (or e2 e3 ...) it seems <malaclyps>is Guildhall not an active project any more? Do people tend to build guix packages instead these days? ***apteryx_ is now known as apteryx
<apteryx>actually, nope, still don't understand <daviid>otherwise, yes, guix - from which one of our guiler maintainer suggested, a while ago, to derive a guile package manager, called potluck, based on guix <apteryx>can let* be implemented with just syntax-rules instead of syntax-case/ <nly>janneke: str1ngs: new-mru should work with Nomad, and kill-buffer should be fixed. *civodul builds an upgraded guile-next <lloda>wingo: got it, thx for all the work <lloda>iiuc there's no need to do anything to take advantage of declarative-modules? <lloda>re: the f64 unboxed ops, I have to say that it looks strange to have all that for f64 alone and not f32 <wingo>lloda: guile has always used f64 internally for inexact reals <wingo>so, this is really just about avoiding the flonum box <wingo>the result of an f32vector-ref gets expanded to a double <wingo>my understanding is that this is acceptable numerically <wingo>obviously there are perf/space considerations, but just from a correctness POV <lloda>I am unsure whether you'd get the same exact results as if you didn't extend, bitwise <lloda>for any reasonable program, I don't see a problem <lloda>probably mark_weaver is the person to ask <lloda>I just rebuilt in debian and I had the test for srfi-18 hang <wingo>i think there's one more optimization knob before we do great on the standalone benchmarks like ecraven's <wingo>which is to allow letrectification to avoid boxes for functions that aren't exported by their module, and just bind via letrec. it would have to be an optional optimization <wingo>because it's just too much against what people expect :) <wingo>but as a first thing to add to -O3, it could make sense <wingo>otherwise the compiler can't see that it can unbox things in e.g. the mbrot test, and it takes 50x what it should <lloda>which honestly doesn't require C perf, just good enough <lloda>declarative modules didn't do anything for my current benchmark :-/ <wingo>i guess you already manually inlined a bunch of stuff <wingo>yeah. declarative modules mostly allows people to write things more "normall" <lloda>maybe I should undo some of the macro mountain <wingo>well, manual inlining is often more effective at this point <wingo>given that declarative modules don't allow for contification yet <wingo>i think there's more juice to squeeze in f64 stuff. e.g. the compiler doesn't know how to go directly from u64 to f64 and vice versa, whereas sometimes you want that <wingo>welp, back to the work i should be doing :) <lloda>cy :-) and thx again for everything *chrislck happy about (define) being allowed in (let)... don't understand why it's not considered 'correct'... got caught out many times in my early days <jcowan>ATM only Racket and Kawa have internal support for float32, and it's going away in Racket. <kdtsh>Hi all! Can I ask a simple question about using guile here? <lloda>kdtsh: yes, just ask the question <lloda>if no one replies you can try mailing guile-user <g_bor>I have a bit of a problem with guile web server. <g_bor>I am trying to do a multipart/form-data submission. <g_bor>All is working well, until I repost the data. <apteryx>and one of the examples is implementing let* in terms of let, using syntax-rules (syntax-case hasn't been introduced yet in this book). <g_bor>On the repoest I got a backtrace, saying Bad Request-Line: <the first line of the uploaded file>> <g_bor>any ide what is going on here? <g_bor>I did the repost by refreshing the site in a browser, and clicking yes to resend post data. <kdtsh>Thanks a lot lloda. I'm trying to use the (web client) module, and have been having trouble figuring out how to use the #:header key when making an HTTP POST. I've tried various combinations of pairs, e.g. "('content-type "application/json")" and "'('content-type ((header-parser 'content-type) "application/json"))", but have gotten errors like "Bad <kdtsh> value for header quote: (content-type)", or "Header name not a symbol (quote content-type)". If I want to use a "Content-Type: application/json" header, what should I use for this argument? <wingo>apteryx: hint: (define-syntax let* (syntax-rules () ((let* () body ...) (let () body ...)) ((let* ((x y) bindings ...) <fill in here>))) <wingo>apteryx: hint: (define-syntax let* (syntax-rules () ((let* () body ...) (let () body ...)) ((let* ((x y) bindings ...) body ...) <fill in here>))) <g_bor>kdtsh: I believe you need something like '((content-type . (application/json))) <apteryx>wingo: thanks! I'm looking at your hints :-) <wingo>we can tell you more of course but it's nice to figure it out :) <apteryx>indeed! I'll be back with the solution... hopefully ;-) <apteryx>you need body1 body2 ... though right? because let requires a non-empty body <wingo>well, there are two approaches <wingo>one is, do the minimal amount of checking and rely on your expansion to "let" to catch any errors <wingo>i.e. in that case you could do: <wingo> (define-syntax let* (syntax-rules () ((let* () . body) (let () . body)) ((let* (binding . bindings) . body) <fill in here>))) <wingo>or, you try to make the pattern complete, so that the user gets an error like "bad let*" (which they presumably had in their source) and not "bad let" (which is the result of the macro expansion, having maybe propagated bad code from the input to the output) <wingo>sometimes it's actually important to do less checking that you would otherwise do, to prevent n^2 expansion times! <wingo>usually not a consideration tho <apteryx>I see! thanks again for the detailed explanation :-) <wingo>g_bor: pretty sure the current output port is overridden during "handler", dunno tho <g_bor>wingo: do you think this is a guile bug, or is it something else? <g_bor>I will also test later if this is still afiling on more recent versions. <wingo>g_bor: i don't know tbh. i think nalaginrut would know more <wingo>i would check the source code to artanis to see what they do <g_bor>ok, I will have a look at that later. <apteryx>wingo: I could figure out one solution, thanks to your hint! (define-syntax my-let* (syntax-rules () ((_ () b1 b2 ...) (let () b1 b2 ...)) ((_ ((x1 v1) (x2 v2) ...) b1 b2 ...) (let ((x1 v1)) (my-let* ((x2 v2) ...) b1 b2 ...))))) <apteryx>it recurses down until there's no more bindings to process (base case) <wingo>but it doesn't make a huge difference <wingo>it also prevents some n^2 expansion time <kdtsh>@g_bor that does it - thanks heaps :) <apteryx>Why do we need to introduce a temporary variable in the definition of 'or'? <apteryx>wouldn't the template be just as good with (if e1 e1 (or e2 e3 ...)) vs (let ((t e1)) (if t t (or e2 e3 ...)))? <wingo>apteryx: consider, (or (pk 1) (pk 2)) <wingo>it should evaluate (pk 1) exactly once <wingo>this sort of thing comes up all the time in macros <wingo>ie. (if (pk 1) (pk 1) (pk 2)) is not a correct transformation <davexunit>been extremely offline this weekend so I haven't taken it for a spin yet but I'm excited to try it <davexunit>and I can see that ludo beat me to updating the guix package <civodul>i'd really like to have the whole stack 3.0-ready when 3.0 comes out <civodul>we're not far from that, but there's some "administrative" burden <civodul>essentially adding "3.0" in all the GUILE_PKG Autoconf macros :-) <davexunit>civodul: I'm curious how much of performance boost guix could expect <civodul>i can't tell yet, but i'm curious too! <civodul>we should define a "guile3.0-guix" package... <davexunit>jit has completely changed my life on the graphics rendering side of things <civodul>Guix is a very different workload, there's lots of i/o for instance, so maybe it's less sensitive <davexunit>and I'm curious if the jit can just help with throughput <civodul>also i wonder how vlists/vhashes perform in 3.0 compared to 2.2 <civodul>Guix uses a lot of that, but that's also stressing the GC quite a bit <davexunit>if the GC is stressed than jit will have limited benefit <davexunit>it will help generate the garbage faster, I guess :) <davexunit>I have to make sure all my core number crunching code generates nearly 0 garbage in order to get acceptable results. <davexunit>and to allow room for the user program to generate some garbage. <davexunit>but with all these flonum optimizations I have unboxed ops in every critical path, I think. <manumanumanu>civodul: Do you think implementing transients for vlists would be a good idea? I mean, they are quite a bit simpler than fectors (I have a "packaged" implementation of andy's fectors if you are interested: https://hg.sr.ht/~bjoli/guile-fector), but are slower to build than transient fectors (which are amortized o(1) with pretty low constant factors). That shouldn't really be the case if I read the source <wingo>davexunit: were the recent sin/cos things any help? <wingo>regarding guix it is tricky. a lot of the guix user experience is guile compilation time <wingo>and the output of 3.0 bytecode is more voluminous, which could cancel out any speed gains and maybe more <wingo>so i too am interested to see. it may need some tweaking before we get it right <davexunit>wingo: I haven't yet tried it out. to benchmark it I guess I will use a sprite batch in which every sprite is rotated. that will cause thousands of sin/cos calls per frame and I can test against 2.9.3 and 2.9.4 and see what the max sprites at 60fps are for both. *wingo found a nice optimizer bug <wingo>(office is too hot, can't think about the problems i should be thinking about :P) <civodul>manumanumanu: re transients for vlists, i don't know! <civodul>i feel like i could use hash tables for instance in cases where vhashes are too slow <civodul>but i'd rather have faster vhashes :-) <daviid>wingo: lloda: so, guile 3.0 won't offer unboxed f32 ops then? <jcowan>In general, random float32s running around doesn't help anyone and makes type dispatch that much more complicated. What would be a Good Thing is to provide a version of @vector-map (where @ = u8, s8, ..., f32, f64) that notices when its operand is +, say, and substitutes a version that adds the corresponding elements unboxed and puts them into the output vector. <daviid>jcowan: not sure what you are talking about when yu say random float32s running around - but i was asking because in the image processing world, most lib works on f32 vectors, like vigra, which guile-cv uses - mainly for perfs/result quality ratio reasons, so they say - and vector-map will get you absoluyely nowhere, we a re talking about hundreds, actually thousands of math algo, based on but not only, linear algebra <daviid>in a typical research lab, doing img proc, the average image size is 20 million pixels, and we manipulate stack (3d) of thoushands of those ... <daviid>so, until scheme offers C perfs, i am sure it could and hope it will, if we can declare, 'à la CL', (declare (img f32vector) ...) there is no way we can 'fight' other libs 'out there' <davexunit>I use f32vectors all the time and do unboxed arithmetic with values from them <davexunit>if you look at the bytecode you'll see that it's making f64s out of them, but everything is unboxed and fast. <daviid>davexunit: last time i tried, lloda explained these are converted to a scheme float before 'anything' on them ... <daviid>so, a typical op in guile-cv is between 50x to 100x faster in C <davexunit>all I know is that I use f32vectors all the time and I can do nearly everything without boxing the float values <daviid>davexunit: i'm not a compiler expert, what i can say, is that the exact same algo, for example sum of two matrix (represented by f32vectors) is betwen 50x to 100x faster in C then in guile scheme <daviid>sure, so i hope we'll get there ... <davexunit>and my point was that values from f32vectors can absolutely be unboxed <davexunit>so your code should be a lot faster than it otherwise would have been if you've written it in such a way to take advantage of unboxing <daviid>davexunit: last time i spoke to lloda about this, that was a relatively long time ago, 2y or so, he explained to me that f32 were converted to scheme - and maybe i'm wrong, but by unboxed i meant no transformation and ready 'as is' for cpu/gpu processing <davexunit>but I'm not sure what "converted to scheme" means <davexunit>unboxing means that the value is not wrapped in a new, dynamically allocated scheme object. <davexunit>so you can call f32vector-ref and the vm may just stash the float in a register and skip the part where it allocates a scheme object to store the float <daviid>iiuc, lloda explained that guile was not doing this, back then, it was indeed allocating a scheme object ... (which is what i meant by 'trnsformation') <daviid>davexunit: i kept all algo in scheme, those i wrote in scheme in the first place, so i can try again and see ... but as you said, it is only part of problem <daviid>and i have no option but be as fast as imagj and/or opencv <davexunit>unboxing + jit should close the gap quite a bit. <davexunit>but you'll need to inspect bytecode to make sure it's generating the good stuff <daviid>i'll try again if i can find some time, then will ask for your/lloda/wingo/mark help, but now, i'm on another huge yak, and wrt guile-cv another part of the yak :) <daviid>davexunit: thanks! i'm not very good at 'low level' stuff, so when time comes, i'll always appreciate others help to get to the very details that needs to be worked on ... <daviid>on a diff but related subject, i once did think about looking at and possibly partially bind GSL <jcowan>The point of treating @vector-map specially is that you can avoid the f32->f64->f32 conversions. <daviid>i heard numpy is written in C, if that is correct and if it has a clean, stable, well maintained api, it might be worth a binding as well <daviid>*a clean, stable, well maintained C api, numpy is clean ... <jcowan>Note that numpy's UFuncs are precisely the sort of polymorphic array operations I've been talking about. <jcowan>And yes, numpy has a C API, but it's meant for use in a CPython context; it needs definitions of things like PyObject. <lloda>isn't numpy mostly glue? I don't see the point of wrapping the glue when you can wrap the underlying library directly <daviid>jcowan: ok. worth mentioning that lloda is working on newra, which you might be interested to look at, if you haven't already, maybe you know this but ... <jcowan>I don't know. I just asked on #nmpy, we'll see. <lloda>I still haven't looked at jcowan array srfi :-/ <jcowan>I don't have one personally. There are a few out there, SRFI 122 and 164. <lloda>newra is just a replacement of Guile's C by Scheme <lloda>well it does more, but not a lot <lloda>I think if we wanted to have a cool-modern-powerful array system for Guile and not just something to bind C libraries I'd look at remora or whatever is in racket <jcowan>they aren't hugely different from Guile arrays except in detail <lloda>yes, the objects to move things around are basically the same everywhere <jcowan>The only thing I have personally (in very early draft) is some higher-level procedures like inner and outer product. <jcowan>they should be easy to layer over any array package <lloda>I had a library called guile-ploy that did outer product by using the rank conjunction <lloda>but then I realized that was the same as broadcasting <lloda>it makes things like outer product trivial <lloda>in fact argument rank etc. from J translates directly to broadcasting and costs nothing <lloda>I think the J doc don't make the point well enough, because they abstract the implementation <jcowan>Not sure I follow that (I am not an array maven in any language) <lloda>broadcasting is just having some axes where the step is 0 <lloda>you use the same loop functions, but on some arguments the indices don't move *daviid thought broadcasting, i the array field, was about casting from one type to another <lloda>there's an example in newra's readme <lloda>for example in numpy you can do an outer product of 1-arrays like a(newaxis, :) * b(:, newaxis) <lloda>I read something of yours on a trac page, in case it's the same <lloda>there are somethings I didn't like that other libraries do e.g. in srfi-43 where the indices are passed to vector-map <lloda>anyway, I didn't get any feedback on newra, and there are a couple things I'm not sure about <lloda>like, I have infinite axes :p ***ngz`` is now known as ngz
<lloda>the one thing that worries me the most now is the basic temporary-array optimization <lloda>like if you do (+ C (+ A B)), don't compute the whole of (+ A B) before adding C <lloda>I'm not sure how to do that in Scheme in a way that's clean <lloda>still with native compilation in Guile the basic array-map! is now faster in Scheme than it was in C, because you save all the transitions C-Scheme that happened every element <lloda>so just for that, it's worth doing it <daviid>lloda: but scherme wousl always call (+ a b c) no? so i mean it is a user responsibility, not a compiler (even though it could be) <lloda>daviid: well yes + was a bad example <lloda>you want that to be (loop (lambda (c a b) (f c (g a b))) C A B) <lloda>where c a b are scalars, so you loop over the arays only once <lloda>and don't need the space to store (g A B) <lloda>that's an obvious optimization, many C++ libraries do it for example <lloda>and there's a lot of literature about it <lloda>but I don't know now to do it as a library in Scheme <lloda>haven't really thought about it really *civodul can build quite a few things on Guile 3: "guix build guile-gcrypt guile-git guile-json guile-sqlite3 --with-input=guile@2.2.4=guile-next --with-branch=guile-gcrypt=master --with-branch=guile-git=master --with-branch=guile-sqlite3=master" <lloda>daviid: reading above I think davexunit is correct, floats aren't converted to scm and back anymore. Still f32s will be converted to f64s because Guile's floats are f64s <lloda>even still the compiler needs to know that something is a float <lloda>which happens if you use f32vector-ref or f64vector-ref for example <lloda>that's what I understand anyway. <davexunit>yes, it's a pretty safe bet that you're going to get a float when you use either of those <daviid>lloda: right, and that conversion makes it difficult, if not ipossible, to reach similar perfs then i have using C, very unfortunately <daviid>it seems that guile will never 'get rid' of those conversion? for f32 i mean <lloda>daviid: I don't know about never, but maybe it isn't worth it until the compiler gets even better <daviid>ok, i asked because i'm stuck in the f32 world because of vigra <lloda>I think most of the time even C isn't good enough because something like BLAS or the gpu would be faster than you can code in C <lloda>daviid: I also use f32, that isn't going to change <daviid>lloda: yes, though blas and friends will only offer a very short portion of the algorithm we use in the img-proc domain, and it 'never stops', up to the point that it is difficult to follow <daviid>lloda: it seems, but i didn't look at yet, just did read, that gsl offer a cblas 'interface' <lloda>I wrote an ffi interface for cblas <lloda>I think the ffi is cleaner, you don't need to install anything <lloda>but I sunk in the macro swamp and I didn't complete all the bindings <lloda>I mean if you use the gsl bindings from Guile, you'd need to wrap them <lloda>so why not go directly to the library you want <daviid>lloda: i did look at your ffi-cblas, but didn't feel i could make guile-cv epend on it, with all due respect, then 'soon later' you switch to other projects and it feels unmaintained <civodul>it'd be interesting to implement some of the BLAS routines in Guile 3 just to see <daviid>as said above, it would only 'give me' a few of the many more algo i need <civodul>no i mean just to see what performance we get <daviid>civodul: yes, i was answering lloda <lloda>daviid: it is maintained, just barely ;p <lloda>I wanted to automate more stuff instead of cp'ing code <lloda>and my macro game is limited, so I got stuck there <lloda>still I'd like to complete them eventually <lloda>thing is, I don't really like cblas as a library <lloda>but it's so standard, there's no way around <daviid>lloda: don't worry, i took another route and will stick to it for now <lloda>civodul: I think you'd need to write very dirty scheme to compete <jcowan>One thing I have on the deep back burner is a simple-minded translator from minimal Fortran to Scheme, along the lines of the various Fortran to CL translators. <jcowan>For libraries, it wouldn't have to do the whole Fortran I/O thing. <lloda>f77 is very raw compared to modern fortran <lloda>it's true there's a lot of software written in f77 <lloda>I had the chance to use modern fortran recently and it's actually nice <lloda>still low level, like you make a mistake and it segfaults :p <jcowan>oh yes, I realize that. But these libraries that everyone uses are often f77 or even f66. <jcowan>netlib, I think, is almost all f66. <lloda>modern fortran has a nice C interface, you can use it almost the same way if you mean to write glue <lloda>wish some of that stuff got updated :-/ <lloda>I had to wrap pieces of SLATEC a while ago and it was a nasty thing <lloda>like it took a complex array by separate real & imag, and they had to be compact arrays <lloda>and it was all static and global stuff inside, no thread safe at all <lloda>but if you want this or that function, that's where they are