IRC channel logs
2025-01-06.log
back to list of logs
<ttz>Hi there, I am wondering why Guile does not constant-fold (λ (n) (1+ (1- n))). Upon running ,optimize (λ (n) (1+ (1- n))) I get (lambda (n) (+ (- n 1) 1)) and upon running ,disassemble (λ (n) (1+ (1- n))) I get: <ttz> 0 (instrument-entry 53) at (unknown file):19:13 <ttz> 2 (assert-nargs-ee/locals 2 0) ;; 2 slots (1 arg) <ttz> 3 (call-scm<-scm-uimm 1 0 1 3) ;; sub/immediate at (unknown file):19:29 <ttz> 5 (call-scm<-scm-uimm 1 1 1 1) ;; add/immediate at (unknown file):19:25 <ttz> 7 (reset-frame 1) ;; 1 slot <ttz> 8 (handle-interrupts) <ttz>Where arithmetic operations are still performed. <lilyp>ttz: unfortunately, adding numbers is not side-effect free – you could get allocations or even exceptions <lilyp>with floating points, you might even get one-directional rounding errors <ttz>I understand for the float issue. If I can guarantee that my number is an integer, however, can't I reduce this expression ? <ttz>I don't understand how allocation can be a side-effect <dthompson>ttz: if you want more optimal code for that procedure you have to constrain the inputs <dthompson>constraining to an integer is one step, but I don't think it will change the bytecode just with that. <dthompson>because an integer may be a fixnum or bignum <ttz>do you have any hint? <dthompson>do you have a value range that you know the input will be in? <dthompson>guile can perform unboxed arithmetic when it knows the inputs are 64-bit doubles or integers <ttz>typically 64-bit integers <ttz>fixnums then if is that it? <dthompson>the fixnum range is smaller, 63 bits on 64-bit hardware <dthompson>here's an example guard: (unless (and (integer? x) (<= 0 x (ash 1 64))) (error "not a u64" x)) <dthompson>I didn't test it, but the intention is to ensure that x is a u64. adjust as necessary. <Codeko>Can i just shadow define with define-with-documentation in my C Guile program ? <lilyp>probably with the right use-modules clause <ttz>How come there is no fixnum? predicate? <ttz>ah my bad, so I just didn't see it in the manual of the core API <ttz>Is there a reason why this is not in a base predicate? <dthompson>without that predicate, you can refer to most-negative-fixnum and most-positive-fixnum <ttz>anyway it doesn't seem to halp the compiler to produce better optimized code <dthompson>well, not the fixnum? predicate but the others <ttz>when reusing my earlier ,optimize (lambda (n) (unless (fixnum? n) (error)) (1+ (1- n))) example <ttz>I do not see any folding <dthompson>the reason that probably doesn't work is because the fixnum? check isn't inlined <dthompson>so the compiler still doesn't know anything specific about the type of n <ttz>is this linked with the fact that it is not a base procedure? <ttz>you mean that the fact it is not a primitive is the reason why it is not inlined? <dthompson>well, I mean that primitives are open coded, they may be inlined or may not be. whether other procedures are inlined depend on other things. <dthompson>more inlining happens when procedures belong to the same compilation unit i.e. the same module <dthompson>cross-module inlining can happen, too, but doesn't seem to be here. <dthompson>which reflects my experience with the rnrs arithmetic stuff <ttz>It seems to me that inlining predicates is typically the kind of stuffs we would like to get more optimized code. That way type checks at the start of a primitive can be used as a sort of dependent typing? <dthompson>yeah you need all the information in the procedure so guile's type inference pass can do its thing <dthompson>in this case it doesn't seem to improve the generated code <dthompson>this level of nitty gritty bytecode analysis is usually unnecessary, though <ttz>well small inefficiencies tend to accumulate fast <ttz>I was trying to write a transduce procedure the other day, and was surprised to see that there were a real cost of composing the transformations rather than writing them in a more imperative way <ttz>In that case at least, missed optimizations influence programming style. <dthompson>just giving the usual advice against premature optimization <dthompson>but if you know what you're doing, go for it. I spend plenty of time reading bytecode disassembly. <ttz>Sure, the root of all evils etc. I am currently trying to implement functional data-structures and comparing them with the Guile hash-map. It's my first time trying to optimize Guile code. <ttz>I am surprised no functional map exists in the default libraries. <dsmith>ttz, What do you mean by "functional" map? <dthompson>I think the reason there's nothing built-in is because you can get surprisingly far with just lists <dthompson>so the requests mostly come from people who are used to clojure and can't imagine not having them <ieure>Clojure really knocked hash-maps outta the park. :) <dsmith>Ah, map as in data structure, not map as in higher-order-function <ttz>well I agree that you can go surprisingly far with just lists <ttz>but sometimes you really do need an efficient mapping data-structure (e.g. memoization even though for cases that preserve temporal locality an alist would be near optimal) <ttz>Plus, having a hash-table as only alternative to alists is kinda shameful for a "functional-oriented language" <dthompson>yeah, definitely. just explaining why I think there hasn't been much movement to add something to the standard library <dthompson>scheme is less functional than people say it is, though <ttz>and I thank you for sharing your view on it, and for the link: I am going to check it right away <dthompson>I think it's only a matter of time before something is included in guile but probably there should be some third-party lib that does it right that guile can later adopt <dthompson>when I'm writing async programs with fibers I want a good functional map <ttz>I got pretty far just with a splay tree <dthompson>guile has vlists and vhashes which are functional but they're not great, imo. <ttz>100LoC or so and about 1us per search on 1e7 elements <ttz>for comparison a quick bench on an Rust HashMap got me ~70ns in an equivalent setting <ttz>which is only 20x different <dsmith>ttz, Is that with the default Rust hasher? <ttz>I didn't spend much time setting up the Rut bench <ttz>I do not guarantee these numbers: I am planning to spend some time later for a proper comparison <ttz>Splay trees also have the nice feature to adapt to the query distribution, which means allocation... but I still got down to .5us per search in a sequence of contiguous keys <ttz>I still need to implement a HAMT since it is all the rage in the closure world. But my heart definitely goes with the splay tree <dthompson>would be interesting to see a comparison between the two <dthompson>you may be interested in reading the code for intmaps and intsets, which are special purpose functional data structures used by the compiler. (language cps intmap) and (language cps intset) <ttz>Are these Patricia tries ? I read the Okasaki paper describing them as the ones used in the ghc back at the time <ttz>I will have a look at these impls though thanks for mentioning them <ttz>Are they in Guile or C ? <rlb>(jvm clojure's at least) <rlb>dthompson: I'd be more than happy to help get fash or something else in to guile if we knew what we wanted. I suppose settling on an api would be a key thing. <dthompson>rlb: this is why I think maybe we need to "hash it out" with some third party libraries first <dthompson>afaik there isn't an existing one that is widely used <rlb>...sounds plausible, but "it's been a good while"? i.e. I agree, but I have a suspicion this could be a case where some flavor of "we" might have to do that, and maybe it gets picked up elsewhere, or not, but we end up with something we're willing to go with (if we do start externally). <ttz>I would be glad if I could help too <ttz>Thanks for those links rlb, I will have a look at it. Which data-structure is implemented under the name of fash? Is it a HAMT? From a quick glace it seems you are using bitmaps <rlb>*I* I'm not doing much of any of that -- that's all wingo's code afaik (other than the minor changes you see in the history on top of the original). <rlb>And I haven't yet read it all carefully either. <rlb>Basically got it working for what lokke needed and that was all for now. Though I really do still need some flavor of dissoc. <rlb>(right now I cheat on that front) <ttz>okay, then I'll have to guess :) but my guess is that it is a HAMT <ttz>I read an interesting article from the Racket people who have added the stencil vector to the compiler, which is the building block of a HAMT. Their benchmarks show that they perform pretty well when compared to languages based on the JVM. <pukkamustard>there's srfi-146 providing a functional mapping api. the reference implementation uses HAMT and red-black trees. its usable from Guile via the package guile-srfi-146. <ttz>Yes, but I got really poor performances. Have you some good experience of it? <ttz>From what I saw, there are layers and layers of functions and lambdas that I am not confident Guile can optimize away. Maybe my benches were off, I will add them at my final comparison, you are right! <pukkamustard>not much experience, unfortunately. I thought the api is nice and it worked for what I needed. interesting to hear that performance is not so good. <dthompson>layers of functions are not necessarily indicative of an issue. it all depends. <ttz>well I remember being surprised of how slow insertions were. <ttz>lookup were okay if I remember <ttz>will bench it more thoroughtly <ArneBab>dthompson: regarding the functional hashes and such (and also fibers): I think this is a failure mode in which Guile currently hangs: ice-9 is intended to be "a seed crystal to crystallize the mass of software." — https://www.gnu.org/software/guile/manual/html_node/Status.html — but we’ve become extremely slow in adding to it. Fibers have become a paradigm of threading in Guile already, but they are external, and it seems that not <ArneBab>many larger changes have happened in ice-9 since 2011 or so; that the rate of change slowed. <ArneBab>As if we were afraid to move forward within Guile itself. <ArneBab>(or blocked by a review bottleneck?) <dthompson>a lot of the stuff in ice-9 is not very good so slowing down feels okay to me <dthompson>I think fibers has proven itself to be a good enough api that it should become part of guile. it's part of the hoot standard library since async is critical to client-side web programming. <dsmith>There has been a problem with too much change in the past (ask the lilypond folks). wingo is brilliant, but he admits he not much good as a maintainer, would rather write new stuff. <dsmith>There is a huge backlog of bugs and patches. <dthompson>an additional maintainer would be great to handle that stuff. <dsmith>Yes, I think so. I know mdj has been active again. <dthompson>I also think there's the practical issue that debbugs is extremely bad and makes it hard to do the work <rlb>(utf8 branch is current again, fwiw) <rlb>We guarantee .go compatibility across all z releases, but maybe not y releases, right? <ttz>dthompson: why are you seaying debugging is extremely bad? <rlb>(I imagine people know, but there are some tools to work with it too, including some emacs mode(s), which might help...) <ttz>ArneBab: it would be sad for Guile to stop evolving indeed. But from a realively new comer point of view, it is not clear to me what is Guile's path: I would love it to make a functional turn (functional mappings, match and more combinators by default for a start) but it does not seem to be any of its claim. <ieure>I just want errors that tell me what file and line the problem occurred on. <ttz>I cannot say I exhaustively read the manual and docs, but the only claim I could come around was to be GNU's extension language, which is not much. <dsmith>Remember, guile started as a library for C progs for configuration and extensions in Scheme. That focus has been changing over time. <dthompson>ttz: "debbugs" is the bug tracking system. I wasn't talking about debugging. <ieure>dthompson, They sometimes do, but end at some macro in (ice-9 whatever). <ttz>dthompson: not when reloading code with C-c C-b in Geiser, then you get some cryptic input:... <dthompson>ttz: guile encourages functional programming a lot already. that's kind of the preferred default way to do things <ieure>Has happened to me a lot, very frustrating. <dthompson>ttz: because you're not submitting data from a file port <dthompson>it's over a pipe, which has no file name nor line number <ttz>Maybe, but it seems to be the default behavior in Geiser <ieure>Explanations are much less satisfying than improvements. <dthompson>when reading scheme expressions, the reader gets source locations from the port it is reading from <ttz>I understand it (maybe not as good as you explained it) <dthompson>which is what geiser is using when you eval something <dthompson>so the engineering problem is: can someone come up with a special port for geiser that provides context that would make sense? what would that be? how would it do it? <dsmith>Hmm. I wonder if there could be a way to "force" a file/line, like #line in C. <dsmith>ISTR there are some hooks for that <ieure>CIDER and SLIME both manage this, might be worth looking to see what those do. <ttz>Guile encourages functional programming, that is true... maybe I am fixating on the data-structure issue, but I think it is telling. <dsmith>Buecause lilypond or something (autogen?) needed them <Kolev>I'm excited that Lilypond is Scheme-powered. <ieure>In any case, I have also seen Guile fail to to report the file/line when not using Geiser. Will have to share next time I see it happen. <Kolev>"Nothing's written in Scheme." - "Lilypond, the top-notch music typesetter, is." <ieure>I suspect it's due to not knowing where the code came from when evaluating a macro expansion, or something close to that. <ttz>compiler optimizations can also mess with the stack-trace (e.g. inlining prevents inlined calls to be displayed, and there is also the annoying disappearance of function parameter values) <ttz>maybe there is a way to set the level of optimization performed in the REPL <rlb>ttz: perhaps (system base compile) default-optimization-level (haven't delved) <ttz>dthompson: apart from the functional mappings, not having match (for easy destructuring) or transducers (for performance) prevents easily writing in functional style. Also, I would love to have algebraic data-types à la Haskell/ML. The define-datatype from Friedman's book Essentials of Programming Languages is nice for example. <ttz>btw, the repo you linked dthompson looks very interesting. How does it fare performance-wise? I tried using the guile-redis (I think) library but IO was hopelessly inefficient. I did not spend much time profiling by my guess was the serialization procedures. <dsmith>Ok, Re: geiser and filenames/line numbers: There *are* functions to set those on a port: set-port-filename! set-port-line!