IRC channel logs

2019-08-26.log

back to list of logs

<daviid>though thinking a bit more about this, for praticle purposes, maybe I should provide an unref optional #:recur kw, that would, if #f, performed a 'deep unref' ...
<daviid>at users own risks - instances that have been unreferenced are unusable, and meant to be gc'ed - any attempt to run any gobject method on thse will raise an exception, as expected
<daviid>*an unref optional #:recur kw, that would, if #t, performed a 'deep unref' ...
<apteryx_>isn't the 't' bound variable useless in this example? https://paste.debian.net/1097377/
<apteryx_>one could simply write (if e1 e1 (or e2 e3 ...) it seems
<apteryx_>(the example (and explanation) comes from this book: https://scheme.com/tspl4/further.html#./further:h0)
<apteryx_>ah, I understand now
<malaclyps>is Guildhall not an active project any more? Do people tend to build guix packages instead these days?
***apteryx_ is now known as apteryx
<apteryx>actually, nope, still don't understand
<daviid>malaclyps: for pure scheme code, look at https://akkuscm.org/
<daviid>otherwise, yes, guix - from which one of our guiler maintainer suggested, a while ago, to derive a guile package manager, called potluck, based on guix
<daviid>malaclyps: here is a potluck status related email - https://lists.gnu.org/archive/html/guix-devel/2017-04/msg00649.html
<apteryx>can let* be implemented with just syntax-rules instead of syntax-case/
<apteryx>?
<apteryx>I'm struggling to find the solution
<rain1>apteryx: it can be
<rain1>what are you trying?
<nly>janneke: str1ngs: new-mru should work with Nomad, and kill-buffer should be fixed.
<wingo>o/
<rain1>o/
*civodul builds an upgraded guile-next
<lloda>\o
<lloda>wingo: got it, thx for all the work
<lloda>iiuc there's no need to do anything to take advantage of declarative-modules?
<lloda>default -O being -O2
<wingo>lloda: right
<lloda>greay
<lloda>y/t
<lloda>re: the f64 unboxed ops, I have to say that it looks strange to have all that for f64 alone and not f32
<wingo>lloda: guile has always used f64 internally for inexact reals
<wingo>i.e. flonum has a double
<wingo>so, this is really just about avoiding the flonum box
<lloda>oh
<wingo>the result of an f32vector-ref gets expanded to a double
<lloda>I see
<wingo>my understanding is that this is acceptable numerically
<wingo>do you agree?
<wingo>obviously there are perf/space considerations, but just from a correctness POV
<lloda>I am unsure whether you'd get the same exact results as if you didn't extend, bitwise
<wingo>right
<lloda>for any reasonable program, I don't see a problem
<wingo>cool
<lloda>probably mark_weaver is the person to ask
<wingo>yeah
<lloda>I just rebuilt in debian and I had the test for srfi-18 hang
<lloda>it passed on retry
<wingo>irritating :P
<wingo>same thing tho
<wingo>i think there's one more optimization knob before we do great on the standalone benchmarks like ecraven's
<wingo>which is to allow letrectification to avoid boxes for functions that aren't exported by their module, and just bind via letrec. it would have to be an optional optimization
<wingo>because it's just too much against what people expect :)
<wingo>but as a first thing to add to -O3, it could make sense
<wingo>otherwise the compiler can't see that it can unbox things in e.g. the mbrot test, and it takes 50x what it should
<lloda>that sounds excellent
<lloda>not having to go down to C
<lloda>which honestly doesn't require C perf, just good enough
<lloda>declarative modules didn't do anything for my current benchmark :-/
<wingo>aw bummer
<lloda>yeah ;p
<wingo>i guess you already manually inlined a bunch of stuff
<lloda>all I could
<lloda>it takes minutes to compile
<wingo>yeah. declarative modules mostly allows people to write things more "normall"
<wingo>*"normally"
<lloda>maybe I should undo some of the macro mountain
<lloda>or labyrinth
<wingo>well, manual inlining is often more effective at this point
<wingo>given that declarative modules don't allow for contification yet
<lloda>ok
<wingo>i think there's more juice to squeeze in f64 stuff. e.g. the compiler doesn't know how to go directly from u64 to f64 and vice versa, whereas sometimes you want that
<wingo>u64/i64
<wingo>welp, back to the work i should be doing :)
<lloda>cy :-) and thx again for everything
*chrislck happy about (define) being allowed in (let)... don't understand why it's not considered 'correct'... got caught out many times in my early days
<jcowan>ATM only Racket and Kawa have internal support for float32, and it's going away in Racket.
<kdtsh>Hi all! Can I ask a simple question about using guile here?
<lloda>kdtsh: yes, just ask the question
<g_bor>Hello gentle people!
<lloda>if no one replies you can try mailing guile-user
<g_bor>I have a bit of a problem with guile web server.
<apteryx>rain1: i'm just practicing my syntax-foo with the exercises here: https://scheme.com/tspl4/further.html#./further:h0
<g_bor>I am trying to do a multipart/form-data submission.
<g_bor>All is working well, until I repost the data.
<apteryx>and one of the examples is implementing let* in terms of let, using syntax-rules (syntax-case hasn't been introduced yet in this book).
<g_bor>On the repoest I got a backtrace, saying Bad Request-Line: <the first line of the uploaded file>>
<g_bor>any ide what is going on here?
<g_bor>s/ide/idea
<g_bor>I did the repost by refreshing the site in a browser, and clicking yes to resend post data.
<g_bor>this is on guile 2.2.4
<kdtsh>Thanks a lot lloda. I'm trying to use the (web client) module, and have been having trouble figuring out how to use the #:header key when making an HTTP POST. I've tried various combinations of pairs, e.g. "('content-type "application/json")" and "'('content-type ((header-parser 'content-type) "application/json"))", but have gotten errors like "Bad
<kdtsh> value for header quote: (content-type)", or "Header name not a symbol (quote content-type)". If I want to use a "Content-Type: application/json" header, what should I use for this argument?
<wingo>apteryx: hint: (define-syntax let* (syntax-rules () ((let* () body ...) (let () body ...)) ((let* ((x y) bindings ...) <fill in here>)))
<wingo>er
<wingo>apteryx: hint: (define-syntax let* (syntax-rules () ((let* () body ...) (let () body ...)) ((let* ((x y) bindings ...) body ...) <fill in here>)))
<g_bor>kdtsh: I believe you need something like '((content-type . (application/json)))
<apteryx>wingo: thanks! I'm looking at your hints :-)
<wingo>we can tell you more of course but it's nice to figure it out :)
<apteryx>indeed! I'll be back with the solution... hopefully ;-)
<apteryx>you need body1 body2 ... though right? because let requires a non-empty body
<wingo>well, there are two approaches
<wingo>one is, do the minimal amount of checking and rely on your expansion to "let" to catch any errors
<wingo>i.e. in that case you could do:
<wingo> (define-syntax let* (syntax-rules () ((let* () . body) (let () . body)) ((let* (binding . bindings) . body) <fill in here>)))
<dsmith-work>Morning Greetings, Guilers
<wingo>or, you try to make the pattern complete, so that the user gets an error like "bad let*" (which they presumably had in their source) and not "bad let" (which is the result of the macro expansion, having maybe propagated bad code from the input to the output)
<wingo>sometimes it's actually important to do less checking that you would otherwise do, to prevent n^2 expansion times!
<wingo>usually not a consideration tho
<apteryx>I see! thanks again for the detailed explanation :-)
<g_bor>I have posted some details about this repost failure to this stackoverflow question: https://stackoverflow.com/questions/57658043/how-to-fix-guile-web-server-failure-when-data-is-reposted
<wingo>g_bor: pretty sure the current output port is overridden during "handler", dunno tho
<wingo>maybe not
<g_bor>wingo: do you think this is a guile bug, or is it something else?
<g_bor>I will also test later if this is still afiling on more recent versions.
<wingo>g_bor: i don't know tbh. i think nalaginrut would know more
<wingo>i would check the source code to artanis to see what they do
<g_bor>ok, I will have a look at that later.
<g_bor>Thanks anyway.
<apteryx>wingo: I could figure out one solution, thanks to your hint! (define-syntax my-let* (syntax-rules () ((_ () b1 b2 ...) (let () b1 b2 ...)) ((_ ((x1 v1) (x2 v2) ...) b1 b2 ...) (let ((x1 v1)) (my-let* ((x2 v2) ...) b1 b2 ...)))))
<apteryx>it recurses down until there's no more bindings to process (base case)
<wingo>yep :)
<wingo>guile's version uses syntax-case and explicit recursion, in theory in order to give better errors: https://git.savannah.gnu.org/cgit/guile.git/tree/module/ice-9/psyntax.scm#n3117
<wingo>but it doesn't make a huge difference
<wingo>it also prevents some n^2 expansion time
<kdtsh>@g_bor that does it - thanks heaps :)
<g_bor>kdtsh: yw!
<apteryx>Why do we need to introduce a temporary variable in the definition of 'or'?
<apteryx>wouldn't the template be just as good with (if e1 e1 (or e2 e3 ...)) vs (let ((t e1)) (if t t (or e2 e3 ...)))?
<wingo>apteryx: consider, (or (pk 1) (pk 2))
<wingo>it should evaluate (pk 1) exactly once
<wingo>this sort of thing comes up all the time in macros
<wingo>ie. (if (pk 1) (pk 1) (pk 2)) is not a correct transformation
<davexunit>wingo: thanks for the 2.9.4 release.
<apteryx>ah, that makes sense!
<apteryx>wingo: thanks again
<davexunit>been extremely offline this weekend so I haven't taken it for a spin yet but I'm excited to try it
<davexunit>and I can see that ludo beat me to updating the guix package
<wingo>:)
<civodul>i'd really like to have the whole stack 3.0-ready when 3.0 comes out
<davexunit>that would be neat.
<civodul>we're not far from that, but there's some "administrative" burden
<civodul>essentially adding "3.0" in all the GUILE_PKG Autoconf macros :-)
<davexunit>civodul: I'm curious how much of performance boost guix could expect
<civodul>i can't tell yet, but i'm curious too!
<civodul>we should define a "guile3.0-guix" package...
<davexunit>jit has completely changed my life on the graphics rendering side of things
<civodul>i can imagine
<davexunit>the things that are possible now...
<civodul>Guix is a very different workload, there's lots of i/o for instance, so maybe it's less sensitive
<davexunit>yeah, that's what I'm wondering about
<davexunit>but there's also a lot of graph traversal
<davexunit>and I'm curious if the jit can just help with throughput
<davexunit>and make that stuff a lot faster
<civodul>yeah
<civodul>also i wonder how vlists/vhashes perform in 3.0 compared to 2.2
<civodul>Guix uses a lot of that, but that's also stressing the GC quite a bit
<davexunit>if the GC is stressed than jit will have limited benefit
<davexunit>it will help generate the garbage faster, I guess :)
<davexunit>I have to make sure all my core number crunching code generates nearly 0 garbage in order to get acceptable results.
<davexunit>and to allow room for the user program to generate some garbage.
<davexunit>but with all these flonum optimizations I have unboxed ops in every critical path, I think.
<manumanumanu>civodul: Do you think implementing transients for vlists would be a good idea? I mean, they are quite a bit simpler than fectors (I have a "packaged" implementation of andy's fectors if you are interested: https://hg.sr.ht/~bjoli/guile-fector), but are slower to build than transient fectors (which are amortized o(1) with pretty low constant factors). That shouldn't really be the case if I read the source
<manumanumanu>correctly...
<wingo>davexunit: were the recent sin/cos things any help?
<wingo>regarding guix it is tricky. a lot of the guix user experience is guile compilation time
<wingo>and the output of 3.0 bytecode is more voluminous, which could cancel out any speed gains and maybe more
<wingo>so i too am interested to see. it may need some tweaking before we get it right
<dsmith-work>sneek: botsnack
<sneek>:)
<davexunit>wingo: I haven't yet tried it out. to benchmark it I guess I will use a sprite batch in which every sprite is rotated. that will cause thousands of sin/cos calls per frame and I can test against 2.9.3 and 2.9.4 and see what the max sprites at 60fps are for both.
*wingo found a nice optimizer bug
<wingo>(office is too hot, can't think about the problems i should be thinking about :P)
<civodul>manumanumanu: re transients for vlists, i don't know!
<civodul>i'm not a big fan of that approach
<civodul>i feel like i could use hash tables for instance in cases where vhashes are too slow
<civodul>but i'd rather have faster vhashes :-)
<daviid>wingo: lloda: so, guile 3.0 won't offer unboxed f32 ops then?
<jcowan>In general, random float32s running around doesn't help anyone and makes type dispatch that much more complicated. What would be a Good Thing is to provide a version of @vector-map (where @ = u8, s8, ..., f32, f64) that notices when its operand is +, say, and substitutes a version that adds the corresponding elements unboxed and puts them into the output vector.
<daviid>jcowan: not sure what you are talking about when yu say random float32s running around - but i was asking because in the image processing world, most lib works on f32 vectors, like vigra, which guile-cv uses - mainly for perfs/result quality ratio reasons, so they say - and vector-map will get you absoluyely nowhere, we a re talking about hundreds, actually thousands of math algo, based on but not only, linear algebra
<daviid>in a typical research lab, doing img proc, the average image size is 20 million pixels, and we manipulate stack (3d) of thoushands of those ...
<daviid>so, until scheme offers C perfs, i am sure it could and hope it will, if we can declare, 'à la CL', (declare (img f32vector) ...) there is no way we can 'fight' other libs 'out there'
<davexunit>I use f32vectors all the time and do unboxed arithmetic with values from them
<davexunit>if you look at the bytecode you'll see that it's making f64s out of them, but everything is unboxed and fast.
<daviid>davexunit: last time i tried, lloda explained these are converted to a scheme float before 'anything' on them ...
<daviid>so, a typical op in guile-cv is between 50x to 100x faster in C
<davexunit>I don't really follow, sorry
<davexunit>all I know is that I use f32vectors all the time and I can do nearly everything without boxing the float values
<daviid>davexunit: i'm not a compiler expert, what i can say, is that the exact same algo, for example sum of two matrix (represented by f32vectors) is betwen 50x to 100x faster in C then in guile scheme
<davexunit>that doesn't surprise me
<davexunit>bytecode vm vs. native compiler
<davexunit>among other things
<daviid>sure, so i hope we'll get there ...
<davexunit>but the original point was about unboxing
<davexunit>and my point was that values from f32vectors can absolutely be unboxed
<davexunit>so your code should be a lot faster than it otherwise would have been if you've written it in such a way to take advantage of unboxing
<daviid>davexunit: last time i spoke to lloda about this, that was a relatively long time ago, 2y or so, he explained to me that f32 were converted to scheme - and maybe i'm wrong, but by unboxed i meant no transformation and ready 'as is' for cpu/gpu processing
<davexunit>2y is a long time ago in guile terms
<davexunit>but I'm not sure what "converted to scheme" means
<daviid>neither do i
<davexunit>unboxing means that the value is not wrapped in a new, dynamically allocated scheme object.
<davexunit>so you can call f32vector-ref and the vm may just stash the float in a register and skip the part where it allocates a scheme object to store the float
<daviid>iiuc, lloda explained that guile was not doing this, back then, it was indeed allocating a scheme object ... (which is what i meant by 'trnsformation')
<daviid>davexunit: i kept all algo in scheme, those i wrote in scheme in the first place, so i can try again and see ... but as you said, it is only part of problem
<daviid>and i have no option but be as fast as imagj and/or opencv
<daviid>*imagej
<davexunit>unboxing + jit should close the gap quite a bit.
<davexunit>but you'll need to inspect bytecode to make sure it's generating the good stuff
<daviid>i'll try again if i can find some time, then will ask for your/lloda/wingo/mark help, but now, i'm on another huge yak, and wrt guile-cv another part of the yak :)
<daviid>davexunit: thanks! i'm not very good at 'low level' stuff, so when time comes, i'll always appreciate others help to get to the very details that needs to be worked on ...
<daviid>on a diff but related subject, i once did think about looking at and possibly partially bind GSL
<jcowan>The point of treating @vector-map specially is that you can avoid the f32->f64->f32 conversions.
<daviid>i heard numpy is written in C, if that is correct and if it has a clean, stable, well maintained api, it might be worth a binding as well
<daviid>*a clean, stable, well maintained C api, numpy is clean ...
<jcowan>Note that numpy's UFuncs are precisely the sort of polymorphic array operations I've been talking about.
<jcowan>And yes, numpy has a C API, but it's meant for use in a CPython context; it needs definitions of things like PyObject.
<lloda>isn't numpy mostly glue? I don't see the point of wrapping the glue when you can wrap the underlying library directly
<daviid>jcowan: ok. worth mentioning that lloda is working on newra, which you might be interested to look at, if you haven't already, maybe you know this but ...
<jcowan>I don't know. I just asked on #nmpy, we'll see.
<lloda>I still haven't looked at jcowan array srfi :-/
<jcowan>I don't have one personally. There are a few out there, SRFI 122 and 164.
<lloda>oh
<lloda>still
<lloda>newra is just a replacement of Guile's C by Scheme
<lloda>well it does more, but not a lot
<lloda>I think if we wanted to have a cool-modern-powerful array system for Guile and not just something to bind C libraries I'd look at remora or whatever is in racket
<jcowan>they aren't hugely different from Guile arrays except in detail
<lloda>yes, the objects to move things around are basically the same everywhere
<jcowan>The only thing I have personally (in very early draft) is some higher-level procedures like inner and outer product.
<jcowan>they should be easy to layer over any array package
<lloda>I had a library called guile-ploy that did outer product by using the rank conjunction
<lloda>from J
<lloda>but then I realized that was the same as broadcasting
<lloda>in a way
<lloda>so newra has broadcasting
<lloda>it makes things like outer product trivial
<lloda>and it's essentially free
<lloda>in fact argument rank etc. from J translates directly to broadcasting and costs nothing
<lloda>I think the J doc don't make the point well enough, because they abstract the implementation
<jcowan>Not sure I follow that (I am not an array maven in any language)
<lloda>oh
<lloda>broadcasting is just having some axes where the step is 0
<lloda>so they repeat
<lloda>you use the same loop functions, but on some arguments the indices don't move
<malaclyps>has anyone looked at Coalton? Is this something that would map easily to Scheme/Guile, or does it depend on deep Common Lisp magic, do you think? https://github.com/stylewarning/coalton
*daviid thought broadcasting, i the array field, was about casting from one type to another
<lloda>there's an example in newra's readme
<lloda>daviid: not really
<daviid>lloda: tx
<lloda>for example in numpy you can do an outer product of 1-arrays like a(newaxis, :) * b(:, newaxis)
<jcowan> https://bitbucket.org/cowan/r7rs-wg1-infra/src/default/ArraysCowan.md is actually an old proposal of mine that I am not following up on
<jcowan>written a loooong time ago
<lloda>I read something of yours on a trac page, in case it's the same
<lloda>there are somethings I didn't like that other libraries do e.g. in srfi-43 where the indices are passed to vector-map
<lloda>that isn't free
<lloda>anyway, I didn't get any feedback on newra, and there are a couple things I'm not sure about
<lloda>like, I have infinite axes :p
<lloda>I don't know
***ngz`` is now known as ngz
<lloda>the one thing that worries me the most now is the basic temporary-array optimization
<lloda>like if you do (+ C (+ A B)), don't compute the whole of (+ A B) before adding C
<lloda>I'm not sure how to do that in Scheme in a way that's clean
<lloda>still with native compilation in Guile the basic array-map! is now faster in Scheme than it was in C, because you save all the transitions C-Scheme that happened every element
<lloda>so just for that, it's worth doing it
<daviid>lloda: but scherme wousl always call (+ a b c) no? so i mean it is a user responsibility, not a compiler (even though it could be)
<daviid>*schemers would
<lloda>daviid: well yes + was a bad example
<lloda>take (f C (g (A B)))
<daviid>ok, sorry :)
<lloda>you want that to be (loop (lambda (c a b) (f c (g a b))) C A B)
<lloda>no no
<lloda>I mean don't be
<lloda>where c a b are scalars, so you loop over the arays only once
<lloda>and don't need the space to store (g A B)
<daviid>right
<lloda>that's an obvious optimization, many C++ libraries do it for example
<lloda>and there's a lot of literature about it
<lloda>also some in lisp
<lloda>but with lists ofc
<lloda>but I don't know now to do it as a library in Scheme
<lloda>haven't really thought about it really
*civodul can build quite a few things on Guile 3: "guix build guile-gcrypt guile-git guile-json guile-sqlite3 --with-input=guile@2.2.4=guile-next --with-branch=guile-gcrypt=master --with-branch=guile-git=master --with-branch=guile-sqlite3=master"
<lloda>daviid: reading above I think davexunit is correct, floats aren't converted to scm and back anymore. Still f32s will be converted to f64s because Guile's floats are f64s
<lloda>even still the compiler needs to know that something is a float
<lloda>which happens if you use f32vector-ref or f64vector-ref for example
<lloda>that's what I understand anyway.
<davexunit>yes, it's a pretty safe bet that you're going to get a float when you use either of those
<daviid>lloda: right, and that conversion makes it difficult, if not ipossible, to reach similar perfs then i have using C, very unfortunately
<daviid>it seems that guile will never 'get rid' of those conversion? for f32 i mean
<lloda>daviid: I don't know about never, but maybe it isn't worth it until the compiler gets even better
<daviid>ok, i asked because i'm stuck in the f32 world because of vigra
<lloda>I think most of the time even C isn't good enough because something like BLAS or the gpu would be faster than you can code in C
<lloda>daviid: I also use f32, that isn't going to change
<daviid>lloda: yes, though blas and friends will only offer a very short portion of the algorithm we use in the img-proc domain, and it 'never stops', up to the point that it is difficult to follow
<lloda>daviid: agreed, yes
<daviid>lloda: it seems, but i didn't look at yet, just did read, that gsl offer a cblas 'interface'
<lloda>I wrote an ffi interface for cblas
<lloda>I think the ffi is cleaner, you don't need to install anything
<daviid>ok
<lloda>but I sunk in the macro swamp and I didn't complete all the bindings
<lloda>its https://notabug.org/lloda/guile-ffi-cblas if someone would like to help
<lloda>I mean if you use the gsl bindings from Guile, you'd need to wrap them
<lloda>wrap the wrapper
<lloda>so why not go directly to the library you want
<daviid>lloda: i did look at your ffi-cblas, but didn't feel i could make guile-cv epend on it, with all due respect, then 'soon later' you switch to other projects and it feels unmaintained
<daviid>also
<civodul>it'd be interesting to implement some of the BLAS routines in Guile 3 just to see
<daviid>as said above, it would only 'give me' a few of the many more algo i need
<civodul>no i mean just to see what performance we get
<daviid>civodul: yes, i was answering lloda
<daviid>:)
<civodul>ah :-)
<lloda>daviid: it is maintained, just barely ;p
<lloda>I wanted to automate more stuff instead of cp'ing code
<lloda>and my macro game is limited, so I got stuck there
<lloda>still I'd like to complete them eventually
<lloda>thing is, I don't really like cblas as a library
<lloda>it's a poor interface
<lloda>but it's so standard, there's no way around
<lloda>what can we do
<daviid>lloda: don't worry, i took another route and will stick to it for now
<lloda>ok, np
<lloda>civodul: I think you'd need to write very dirty scheme to compete
<lloda>I mean try to compete
<civodul>probably :-)
<jcowan>One thing I have on the deep back burner is a simple-minded translator from minimal Fortran to Scheme, along the lines of the various Fortran to CL translators.
<jcowan>For libraries, it wouldn't have to do the whole Fortran I/O thing.
<jcowan>and f77 would probably suffice.
<lloda>f77 is very raw compared to modern fortran
<lloda>it's true there's a lot of software written in f77
<lloda>still
<lloda>I had the chance to use modern fortran recently and it's actually nice
<lloda>still low level, like you make a mistake and it segfaults :p
<lloda>but acceptable looking
<jcowan>oh yes, I realize that. But these libraries that everyone uses are often f77 or even f66.
<lloda>true
<jcowan>netlib, I think, is almost all f66.
<lloda>yeah
<lloda>modern fortran has a nice C interface, you can use it almost the same way if you mean to write glue
<lloda>wish some of that stuff got updated :-/
<lloda>I had to wrap pieces of SLATEC a while ago and it was a nasty thing
<lloda>like it took a complex array by separate real & imag, and they had to be compact arrays
<lloda>and it was all static and global stuff inside, no thread safe at all
<lloda>but if you want this or that function, that's where they are