IRC channel logs
2021-09-24.log
back to list of logs
<ArneBab>sts-q: I mean to contribute the C-versions to Guile so this becoms the regular speed of Guile. <ArneBab>stis: I mean to contribute the C-versions to Guile so this becoms the regular speed of Guile. <ArneBab>stis: do you by chance have a blog post where you describe how we can copy your approach? :-) <stis>We can work togehter with this, for sure, but II am wondering what you want to port <stis>let me work some weeks more on this and I will produce documentations. A srfi might also be interesting <lispmacs[work]>Hi, I notice for binary output, that calls to put-u8 are really expensive in program run time, and my application is doing lots of binary output to stdout. I'm wondering how to dramatically speed that up. Do I have to come up with my own buffering system, and dump bytevectors? Or is there some other way to do this which is a lot faster? <stis>I use bytevectors a lot to have good performance in streaming applications, but check if the port is buffered or not if not buffered you can get speedup by having it <lispmacs[work]>stis: I set the output port to block mode and that made it about 20x faster <lispmacs[work]>In my application, I have some number objects, two digit numbers, which have to be outputted to stdout as ascii characters. Is there a faster way to do this than (display N)? The `display` procedure seems to be really slow compared to procedures like put-u8. <lispmacs[work]>is there maybe a lower-level procedure that is faster but less safe (assumes conversion from an integer, e.g.) <lispmacs[work]>I tried doing number->string and then put-string, but it was slower than just (display <number-object>) <RhodiumToad>if you know it's two digits, it's likely faster to do it via table lookup <RhodiumToad>that trick is common even in C code now for decimal output <RhodiumToad>(i.e. don't do division by 10 to get digits, divide by 100 to get pairs of digits and use a lookup table) <lispmacs[work]>I tried this approach, but didn't get a performance improvement. It appears that I saved time not using display, but then got a large performance hit in the garbage collection area <lispmacs[work]>maybe this is something I would have to implement as a C function <fnstudio>hey, suppose i wanted to use hash tables for implementing sets and for speeding up some set operations; is the idea to use keys for set items and something like #f for values? <fnstudio>so basically a hack around using some dummy value across all keys <tohoyn>fnstudio: I think that is how it is done. It is better to use #t as the value so that hash-ref returns #t when a key is in the set. <fnstudio>tohoyn: hey thanks for getting back to me, oh right, yeah #t makes much more sense, ty!! <maximed>lispmacs[work]: If not, maybe use (setvbuf 'block ...) <maximed>lispmacs[work]: to avoid a syscall for every put-u8 <maximed>stdout is non-buffered by default I think <lloda>lispmacs[work]: git main has a patch that extends bytevector-fill! to (bytevector-fill! bv fill start end) <rlb>civodul: (don't know if you saw this days ago) I'm confused again with respect to the gmp settings we should have in debian. It looks like mini-gmp is disabled by default and that disables scm_install_gmp_memory_functions, so things should be safe by default? In which case we don't have to have --enable-mini-gmp in debian, and I'm not sure we could for 3.0, even if we wanted to, since we've been shipping 3.0 for a while without it, <rlb>and it breaks the abi? <civodul>rlb: hi! you're safe if you --enable-mini-gmp; otherwise, it depends <civodul>if you use GMP instead of mini-GMP, you have to set scm_install_gmp_memory_functions = 0 to be safe <civodul>i think passing --enable-mini-gmp changes the ABI <civodul>would be good double-checking with libabigail <rlb>Right - and the default is scm_install_gmp_memory_functions = 0, and we have warnings about changing it, so perhaps the current arrangement in debian is fine? i.e. you'll only be in trouble if you do the things we warn about? <rlb>fwiw, since 3.0's has been in testing for a while and has now shipped in stable (bullseye), I suspect we might have to wait until a soname change to change that in debian. <rlb>If it does change the ABI. <rlb>s/it/--enable-mini-gmp/ which is why I was double-checking. <civodul>yes, scm_install_gmp_memory_functions = 0 is fine too, just slows down some applications like the compiler <civodul>i think i wrote the options & tradeoffs for Debian somewhere long ago :-) <rlb>OK, right -- I assume we can consider --enable-mini-gmp for 4 (whenever that is) depending on when we change the soname to indicate backward incompatibility. <fnstudio>anything you think can be simplified e.g. getting rid of the lambda and using 'equal?' straightaway? <fnstudio>mwette: oh brilliant, thanks, i think i need to medidate on it a bit <fnstudio>very useful, an approach i need to familiarise with <fnstudio>(out of curiosity, did you spot anything very noticeably wrong in my version?) <civodul>rlb: (sorry for the delay) yes, i think you can just "scm_install_gmp_memory_functions = 0" for now and switch to --enable-mini-gmp at a later point in time; there's no rush <civodul>apologies for the rushed answers earlier! <mwette>fnstudio: im not very familiar with map-or etc <mwette>lampilelo: some issue with that: or does nothing <mwette>but I see where you are going: if performance matters I think there is a difference <lampilelo>right, i rearranged some stuff before posting it and didn't think it through afterwards, there should be (and ...) around (or ...) (any ...) <atuin>is there a way to find the path of loaded module? <atuin>mmm i tried that but I don't get the full path, just the module name <atuin>mmm yeah my problem here is that seems i have guix modules in 2 paths, and i'm getting the wrong version of sudo in this case :D <dthompson>follow the load path in order and the first match is the right one <atuin>so it's followed in order always, ok <dthompson>if you really want to be sure, then I'd run strace on the process and hunt for the open <atuin>I thought that maybe the struct returned in guile had that info <dthompson>guile ought to print out the absolute file name of the module, though. <dthompson>I wonder if there's a good reason why it doesn't. <atuin>mmm no idea, but it would useful in this case :D <atuin>I'm basically adding the %load-path returned by guix repl to be able to get definions from my current guix profile instead of from the system <dsmith-work>scheme@(guile-user)> (search-path %load-path (module-filename (resolve-module '(srfi srfi-1))))$1 = "/usr/local/share/guile/3.0/srfi/srfi-1.scm" <atuin>now i'm confused that returns the right path, a file where sudo is defined to 1.9.8p1 <atuin>but sudo symbol in the repl is 1.9.7p1 :D <atuin>ok, maybe it's time to use strace <dsmith-work>Yeah, I think it would be nice to save the absolute path of a module once/if it's been loaded. <dsmith-work>Searching through the %load-path could give different results at different times... <atuin>dsmith-work so it's not deterministic then, or just it does not follow the order <dthompson>I think they mean if the load path is modified between the module being loaded and you searching it <dsmith-work>I mean if you search once, then the load-path changes, and then search again, you might find a different file. <dthompson>but it's an unlikely scenario in a controlled setting <dthompson>guile isn't going to change things on you unless you tell it to <dthompson>so having the module hold onto the resolved absolute file name would be handy <dthompson>I've been through this recently writing a "tree shaker" that can make a bundle of all the guile modules that your program actually uses, leaving out the rest. <atuin>mmm actually my problem is not related at all with the %load-path, maybe it's getting the module from the %load-compiled-path <dthompson>the actual bytecode is read from there. the source files are only used for debugging. <dthompson>does anyone know if it's possible to load *only* the compiled modules, without the source files present at all? I've never been able to get that to work. <dthompson>it would be nice to be able to ship guile programs with only the compiled bytecode in order to greatly reduce download size. <atuin>yeah, well adding the same compiled paths did not fix it :D <atuin>using plain guile i still get teh wrong version of the symbol <dsmith-work>dthompson: Yeah, I wanted that too. Like for an embedded system. <mwette>I've used load-compiled (in the right order) with --no-auto-compile before. <mwette>But that can be sensitive to guile-version. <dthompson>mwette: yeah bytecode incompatibilities and all that. in my case I'm controlling the guile version.