<daviid>goglosh: (with-input-from-string ... read) <davexunit>paroneayea: substituting the build right now :) <paroneayea>that "don't rely on something something multiple values and the continuation" thing mark_weaver said the other day is one thing that prompted me to submit the guile-next package to guix <paroneayea>but it's a lot of datastructure traversal, so who knows where the speed tradeoffs went <paroneayea>5 seconds vs 4 seconds to process a medium sized complexity json-ld document 1000 times <paroneayea>also notably, whether I use vhashes or sjson my implementation of the json-ld expansion algorithms is much slower than the python version... I guess there are still plenty of optimizations to be done <paroneayea>I suspect python is also using faster datastructures tho <paroneayea> 0 (scm-error misc-error #f "~A ~S" ("no such language" objcode) #f) <mark_weaver>paroneayea: what were you doing with 'compose' that you think may have been relying on a bug? <mark_weaver>our 'compose' procedure is actually meant to handle multiple return values; they all get passed as separate arguments to the next procedure <mark_weaver>vhashes are slow, unfortunately. they are written in pure scheme, and not with efficiency in mind. <mark_weaver>a few people have proposed rewriting them in C, but civodul objected. <mark_weaver>which is not an unreasonable position. when we have native code compilation it should be possible to make our scheme implementation as fast as the C version, or maybe even faster (due to avoiding the transitions between C and Scheme) <paroneayea>to just take two out of 3 of the returned values <paroneayea>mark_weaver: anyway, both alists and vhashes are kind of slow <mark_weaver>yeah, that's the other thing. HAMTs are better for many use cases, because it allows fast *replacement* of an entry, where the old one is removed. <paroneayea>I notice there's many ._cache things inside the python implementation <mark_weaver>paroneayea: you can do this instead: (receive (a b . rest) (values 1 2 3) (+ a b)) <paroneayea>mark_weaver: well the good news is everything else worked, and actually I *should* have been catching the third one anyway, it was useful <paroneayea>so in a couple of places I did a sort over a delete-duplicates because I needed a sorted, unique list before processing <paroneayea>and at least in the case of HAMTs or hashmaps, the de-duping would not be needed <mark_weaver>if you need a sorted, unique set of keys, then maybe the data structure should be some kind of balanced tree structure instead. alas, we don't have those built in to guile. <paroneayea>well, the good news is that I think 200-1000 expanded json-ld documents per second should be more okay than it sounds <paroneayea>because theoretically the documents are only ever expanded once <paroneayea>so it's only on receipt and processing of new messages in the api <paroneayea>and for local messages being passed around, that can be skipped entirely because you "know" what the context of your application's structure is already <davexunit>ACTION thinks about performance monitoring for guile web applications using statprof <paroneayea>scheme@(activitystuff json-ld)> ,profile (expand json-test-2) <davexunit>you need to either increase the sample rate or run the code many times so samples can be taken <mark_weaver>paroneayea: profile a loop that expands it many times <paroneayea>well by switching the absolute-uri? default of #:sloppy to #t it kills the worst offender <paroneayea>which is to say, switch from checking if guile can parse into a full uri (which uses regexes) over to just checking if ther's a #: in it <paroneayea>worst next offenders are equal? (not sure how to get around that) then vhash-assoc (used by preventing cycles in the graph, even when using alists, to see if something has or is mid-being defined) which is passed from procedure to procedure, so I'm not wild about switching to a normal hash-table (but that would be likely quite the speedup) <paroneayea>embarassingly I've never used a language profiler before, don't tell anyone! <davexunit>I haven't used them a ton, but Guile makes it very accessible <davexunit>and that was a pretty nice way to visualize performance issues. <paroneayea>I'd really like to see how hamts affect performance of stuff like this <mark_weaver>I wouldn't expect great performance from pfds. it seems to me that ijp prioritizes portability, correctness and elegance in his code (which is a great thing), and not so much performance. <mark_weaver>I vaguely recall hearing that wingo wrote something like HAMTs. if so, it would be worth trying. wingo optimizes for performance. <mark_weaver>there's an old scheme library for weight-balanced trees that might be a good fit here. <mark_weaver>ah, I was probably thinking of fectors, so maybe he didn't yet do HAMTs. <mark_weaver>Slib and MIT/GNU Scheme contain essentially the same classic implementation of weight-balanced trees in scheme. they are probably reasonably well-optimized for scheme code. <davexunit>mark_weaver: he implemented those too, called them 'fashes' <paroneayea>I look up fashes and fectors and get ads about hot flashes. <paroneayea>not my favorite medium but seems an efficient way to get a ping to him <mark_weaver>yeah, in recent times it's not easy to get in touch with wingo <mark_weaver>almost certainly, wingo wrote that code based on what he knew the compiler in 'master' could do, to make sure the generated code would be good. <paroneayea>I'm so glad I wrote abstract aliases around the json associative array structure in json-ld <paroneayea>it means I can just swithc the aliases over really easily to point to alists, or vhashes, or fashes <paroneayea>makes testing this with different structures reallly, really easy <lloda>Guile source has no braces there <civodul>ah right, for a single line, no need for braces <amz3>paroneayea: just wanted to tell that, in the message on the ML there is mention of guix as an the emacs of OS, is that on purpose? <paroneayea>amz3: I mean, civodul has had a talk nearly called that <paroneayea>switching the implementation over to fashes, that port is also faster on guile 2.0 rather than 2.2 <paroneayea>so I guess some things are probably much faster in 2.0, but maybe not things that do this much consing <paroneayea>a lot of that time is spent on system/vm/elf.scm:675:0:parse-elf64-section-header <paroneayea>the number one slowdown of 2.2 according to my profiler is the bytevector-u64-ref calls from parse-elf64-section-header procedure <mark_weaver>paroneayea: interesting. I would expect that to only be used when .go files are loaded, which should only be at startup. <mark_weaver>I would think wingo would be interested in this, but it's hard to get his attention lately <sneek>wingo was here Sep 04 at 04:41 pm UTC, saying: ok, /me -> bar. ttyl :). <paroneayea>so I suddenly wonder if there's a big difference in guile 2.2 between the time of running things at the interpreter vs compiled scheme code <paroneayea>my emacs is filled with parse-elf things right now since that trace <paroneayea>it looks like each section of the code is accessing an elf-section type thing <paroneayea>I wonder if it will do this if I don't use statprof <mark_weaver>hmm, looks like statprof needs to read the debug information from the elf file to determine where the program counter is at each clock interrupt. <paroneayea>okay, time to see if this gets the same kind of output *without* statprof <paroneayea>no statprof, and my screen is being filled with elf table and bytevector references <paroneayea>looks like it starts after the first make-promise <paroneayea>so I suspect statprof has upped the elf / bytevector access a bit, especially at the beginning, but a lot of it seems to be happening here anyway <mark_weaver>well, this is wingo's area, and I don't have time to look into it right now <mark_weaver>(my free time has been completely obliterated by the two kids in the house; anyone who wants to do anything else significant with their lives would best avoid having children) ***holomorph is now known as ubybzbecu
***ubybzbecu is now known as holomorph
<ArneBab>mark_weaver: I agree: my free creative time went down a lot. <ArneBab>with one child it’s much easier than with two <ArneBab>mark_weaver: though I do still manage to get some free time — it requires much more coordination than before, though <paroneayea>make-promise needs to call program-minimum-arity, which calls find-program-minimum-arity, which inspects elf stuff <davexunit>paroneayea: sounds like something for a regression report? <paroneayea>not sure where to report it... the list? the bug tracker? <mark_weaver>the guile maintainers are all stretched thin right now, so I'm not sure anything is well reviewed nowadays, but at least in the bug tracker it won't be forgotten. <paroneayea>do I just send an email to bug-guile@gnu.org? or should I use the debian bug sending tools <mark_weaver>nah, just a normal email with the info that you believe is relevant <mark_weaver>in this case, the details of your system are not relevant. <davexunit>I'm currently trying to figure out how to use monads in Ruby to wrangle some nasty things <paroneayea>I really want monads in guile proper with some of the more common monads <paroneayea>I should focus on getting https into guile well before that though ;) <davexunit>I want to explain monads to my coworkers, but it's hard. I need to internalize more about them. <please_help>It's a monoid of the class of endofunctors. I don't see the issue here. <davexunit>"endofunctor" is one of the most techno-babbley words I've ever read. <paroneayea>it's a highly awkward title, enough to make me not watch it for a while <paroneayea>but the "Monads and Gonads" talk is really interesting <davexunit>that's the first word everyone thinks of when they hear "monads" <dsmith-work>Hm.. make check is (still) failing in control.test on master. <davexunit>and my damn phone keeps changing it to "nomads" <amz3>paroneayea: what I meant, is that a lot of people seem interested by the emacs-like nature of guix, and maybe more information about that aspect of guix might be intersting <paroneayea>ArneBab: just bumped into your shakespeare example again <paroneayea>ArneBab: wisp does seem nice in that it might provide for those places where you want a DSL but don't want it to have too many parens <paroneayea>ArneBab: I think the main thing that may allow/prevent me from playing with it is, did you mention if you were working on an emacs mode at any point? <paroneayea>I guess it couldn't get geiser integration but it could still be nice for playing with <ArneBab>paroneayea: there already is an emacs mode, though not as nice as geiser (by far): M-x package-install wisp-mode <paroneayea>ArneBab: I may give it a run the next time I'm procrastinating <ArneBab>it was one of the first things I did after I could execute some wisp :) <paroneayea>ArneBab: have you thought about seeing to get it included as a guile core language? <ArneBab>paroneayea: having wisp included in guile core would be great! <ArneBab>My current rough roadmap was: (1) package wisp for guildhall (2) get guildhall included in guile <ArneBab>and somewhere between setting up my own guildhall repository and adding a `guild hall` command which allows to quickly shuffle around remote repositories <paroneayea>ArneBab: well, maybe I should add another guix package in the meanwhile... <paroneayea>I've gotten pretty fast at packaging little things for guix :) <paroneayea>ArneBab: it might be interesting in my Guix talk to show a wisp file version of a package instead <paroneayea>to be like, "see, these parentheses just mean this thing, even if it looks foreign to you" <ArneBab>“let me run that” ← after guile autocompiling the file. <ArneBab>^ proves that it’s an equivalent representation <paroneayea>ArneBab: I wonder if a smallll amount of geiser support could be added: <paroneayea>ArneBab: it simply compiles the file and reloads the module <paroneayea>so no need for fancy detect beginning/end of a wisp expression things <paroneayea>so it could still allow for quite a bit of live hacking, because you can get some functions written, then recompile the file <ArneBab>(until 8. October I’ll have to say “feel free to add it” — I have to get my PhD thesis into shape to show it to my Professors on 28. September, 1.10. and 8.10.… <ArneBab>it needs to run guile with -L <wisp-dir> --language=wisp <ArneBab>paroneayea: if you get to adding compiling to wisp-mode, please tell me! (thanks to marmalade, updating the mode is pretty straightforward) <paroneayea>ArneBab: well I think I might have just packaged it for Guix <ArneBab>can you put wisp-scheme at the top-level? If not, you might have to adjust its define-module line and the use-module line in language/wisp/spec.scm <ArneBab>(that’s something I wanted to do for some time: move wisp-scheme into language/wisp/ because that feels cleaner) <paroneayea>ArneBab: /home/cwebber/programs/wisp-0.8.6/bootstrap.sh: 25: /home/cwebber/programs/wisp-0.8.6/bootstrap.sh: [[: not found <ArneBab>so I’ll have to fix that unportable part… <paroneayea>ArneBab: hm, I'll have to make another go at it later <paroneayea>ArneBab: but the problem it has in the guix package: <ArneBab>did you post anything after no luck still:? <ArneBab>paroneayea: (the next line is I’ll have to make… <paroneayea> 35: 3 [#<procedure 1076450 at language/objcode/spec.scm:33:9 ()>] <paroneayea> 616: 2 [for-each #<procedure substitute-one-file (file-name)> #] <paroneayea> 486: 1 [with-atomic-file-replacement "json/Makefile.in" ...] <paroneayea>ERROR: In procedure mkstemp!: No such file or directory <paroneayea>not sure why it would be looking for json/Makefile.in <paroneayea>ArneBab: btw there's no quote/backquote equiv is there? <ArneBab>maybe I’m using some obscure autotools feature? <ArneBab>you can use ` at the beginning of a line and then ,atom , : code ... or simply ,(code ...) <ArneBab>paroneayea: does guix pre-compile the scheme files? <paroneayea>ArneBab: I think it's just doing ./configure && make <paroneayea>ah well, at least I got it running outside of guix :) <ArneBab>I wonder what keeps it from working in guix… <ArneBab>it it trying to mkstemp! for a directory which does not exist? <ArneBab>do you have json somewhere in the guix recipy? <paroneayea>ArneBab: I think it shouldn't be including the json stuff... <paroneayea>I had copied in some code from the guile-json packaging <paroneayea>configure: error: cannot find GNU Guile 2.0 or later. <ArneBab>is there a guile executable named guile-2.0? <paroneayea>ArneBab: nah that refers to the guile-2.0 package <ArneBab>I’m using this: AC_CHECK_TARGET_TOOL([guile], [guile-2.0], [no]) <mark_weaver>paroneayea: look in config.log to see what went wrong <paroneayea>that part is right, but I must be doing something wrong where it can't find guile with ./configure <ArneBab>(which I hope isn’t totally wrong — autotools best practices which work with make distcheck aren’t that easy to find) ***karswell` is now known as karswell
<ArneBab>paroneayea: the configure explicitly checks whether the executable guile-2.0 is in path <ArneBab>it cannot accept guile < 2.0, and Gentoo still has that <ArneBab>(actually it should check for guile >= 2.0.11) <ArneBab>(but I couldn’t find out how to do that) <mark_weaver>ArneBab: guile.m4 from the guile distribution contains autoconf macros to find guile. it's probably better to use those. <mark_weaver>(in the guile source, it's in meta/guile.m4, and it gets installed to PREFIX/share/aclocal/) <paroneayea>mark_weaver: do you remember if /bin/sh is bound to bash in the compilation environment of guix? <mark_weaver>paroneayea: it's not available in the build environment, no <paroneayea>guess I need to change that then too for ./bootstrap.sh <mark_weaver>although we have some default phases in gnu-build-system that take care of the shebangs at the top of scripts <mark_weaver>ArneBab: btw, you don't need to (and should not) copy those macros. they should be automatically available on any distribution that has the guile development package installed <mark_weaver>and of course, they are only needed when you run autoreconf, which most users shouldn't need to do <ArneBab>mark_weaver: ok — you caught me in time :) <ArneBab>can I use GUILE_PROGS to search for >=2.0.11? <paroneayea>ArneBab: I got things to build, but the tests fail <paroneayea>./test-driver: ./syntaxtests.sh: /bin/bash: bad interpreter: No such file or directory <paroneayea>ArneBab: np, more fun than doing the work I'm *supposed* to be doing <mark_weaver>ha, that's a great word, "procrastivity". I may use that myself :) <paroneayea>ERROR: failed to create path for auto-compiled file "/tmp/nix-build-guile-wisp-0.8.6.drv-2/wisp-0.8.6/testrunner.w" <paroneayea>test /tmp/nix-build-guile-wisp-0.8.6.drv-2/wisp-0.8.6/tests/syntax-underscore.w failed. Diff: <paroneayea>mark_weaver: any idea why it can't create the path? I would have thought that would be no problem in the build dir <mark_weaver>normally we disable auto-compilation while building code for guile <mark_weaver>but the easy workaround would be to set HOME to "/tmp" <paroneayea>builder for `/gnu/store/k5z2d7sy7y1737ldlmm1l7w44f8zbx04-guile-wisp-0.8.6.drv' failed to produce output path `/gnu/store/asqf4pqkcrqxqhw7afjaiyhzwh95in4i-guile-wisp-0.8.6' <paroneayea>ArneBab: mark_weaver: looks like it simply didn't do anything upon "make install"? <ArneBab>paroneayea: it doesn’t have an install step yet… <ArneBab>it creates the files to run guile with wisp in the wisp folder <ArneBab>(because I do not yet know how to do make install right) <ArneBab>need to find the guile site path and put guile-scheme.scm and language/wisp/spec.scm there. <paroneayea>ArneBab: yeah it either needs that to be installed with guix, or I need instructions on where to put the files