IRC channel logs

2021-09-24.log

back to list of logs

<ArneBab>sts-q: I mean to contribute the C-versions to Guile so this becoms the regular speed of Guile.
<ArneBab>stis: I mean to contribute the C-versions to Guile so this becoms the regular speed of Guile.
<ArneBab>stis: do you by chance have a blog post where you describe how we can copy your approach? :-)
<stis>We can work togehter with this, for sure, but II am wondering what you want to port
<stis>let me work some weeks more on this and I will produce documentations. A srfi might also be interesting
<stis>ArneBab: ^ ^
<lispmacs[work]>Hi, I notice for binary output, that calls to put-u8 are really expensive in program run time, and my application is doing lots of binary output to stdout. I'm wondering how to dramatically speed that up. Do I have to come up with my own buffering system, and dump bytevectors? Or is there some other way to do this which is a lot faster?
<stis>I use bytevectors a lot to have good performance in streaming applications, but check if the port is buffered or not if not buffered you can get speedup by having it
<lispmacs[work]>stis: I set the output port to block mode and that made it about 20x faster
<stis>o/
<lispmacs[work]>In my application, I have some number objects, two digit numbers, which have to be outputted to stdout as ascii characters. Is there a faster way to do this than (display N)? The `display` procedure seems to be really slow compared to procedures like put-u8.
<lispmacs[work]>is `format` faster? I'll give that a try
<lispmacs[work]>format definitely is not faster
<lispmacs[work]>is there maybe a lower-level procedure that is faster but less safe (assumes conversion from an integer, e.g.)
<RhodiumToad>number->string ?
<lispmacs[work]>I tried doing number->string and then put-string, but it was slower than just (display <number-object>)
<lispmacs[work]>not sure why
<RhodiumToad>if you know it's two digits, it's likely faster to do it via table lookup
<RhodiumToad>create a bytevector with 00010203...99
<lispmacs[work]>ah, okay, I'll try that
<RhodiumToad>that trick is common even in C code now for decimal output
<RhodiumToad>(i.e. don't do division by 10 to get digits, divide by 100 to get pairs of digits and use a lookup table)
<lispmacs[work]>I tried this approach, but didn't get a performance improvement. It appears that I saved time not using display, but then got a large performance hit in the garbage collection area
<lispmacs[work]>maybe this is something I would have to implement as a C function
<RhodiumToad>huh, what were you doing that caused a lot of GC?
<fnstudio>hey, suppose i wanted to use hash tables for implementing sets and for speeding up some set operations; is the idea to use keys for set items and something like #f for values?
<fnstudio>so basically a hack around using some dummy value across all keys
<tohoyn>fnstudio: I think that is how it is done. It is better to use #t as the value so that hash-ref returns #t when a key is in the set.
<fnstudio>tohoyn: hey thanks for getting back to me, oh right, yeah #t makes much more sense, ty!!
<maximed>lispmacs: Is the port buffered?
<maximed>lispmacs[work]: If not, maybe use (setvbuf 'block ...)
<maximed>lispmacs[work]: to avoid a syscall for every put-u8
<maximed>stdout is non-buffered by default I think
<lloda>test
<lloda>lispmacs[work]: git main has a patch that extends bytevector-fill! to (bytevector-fill! bv fill start end)
<lloda>that is just memset
<dsmith-work>Happy Friday, Guilers!!
<rlb>civodul: (don't know if you saw this days ago) I'm confused again with respect to the gmp settings we should have in debian. It looks like mini-gmp is disabled by default and that disables scm_install_gmp_memory_functions, so things should be safe by default? In which case we don't have to have --enable-mini-gmp in debian, and I'm not sure we could for 3.0, even if we wanted to, since we've been shipping 3.0 for a while without it,
<rlb>and it breaks the abi?
<rlb>
<civodul>rlb: hi! you're safe if you --enable-mini-gmp; otherwise, it depends
<civodul>if you use GMP instead of mini-GMP, you have to set scm_install_gmp_memory_functions = 0 to be safe
<civodul>i think passing --enable-mini-gmp changes the ABI
<civodul>would be good double-checking with libabigail
<rlb>Right - and the default is scm_install_gmp_memory_functions = 0, and we have warnings about changing it, so perhaps the current arrangement in debian is fine? i.e. you'll only be in trouble if you do the things we warn about?
<rlb>fwiw, since 3.0's has been in testing for a while and has now shipped in stable (bullseye), I suspect we might have to wait until a soname change to change that in debian.
<rlb>If it does change the ABI.
<rlb>s/it/--enable-mini-gmp/ which is why I was double-checking.
<civodul>yes, scm_install_gmp_memory_functions = 0 is fine too, just slows down some applications like the compiler
<civodul>i think i wrote the options & tradeoffs for Debian somewhere long ago :-)
<rlb>OK, right -- I assume we can consider --enable-mini-gmp for 4 (whenever that is) depending on when we change the soname to indicate backward incompatibility.
<rlb>(in debian)
<rlb>Thanks.
<fnstudio>hi, i need to check whether a list has at least two equal elements next to each other; i thought of this https://paste.debian.net/1213123/ but it looks too verbose to me
<fnstudio>anything you think can be simplified e.g. getting rid of the lambda and using 'equal?' straightaway?
<fnstudio>oh sorry, i spotted a typo
<fnstudio>brb
<fnstudio>ok, amended version: https://paste.debian.net/1213124/
<mwette>I like named let: https://paste.debian.net/1213131/
<mwette>^ not tested
<fnstudio>mwette: oh brilliant, thanks, i think i need to medidate on it a bit
<fnstudio>very useful, an approach i need to familiarise with
<fnstudio>(out of curiosity, did you spot anything very noticeably wrong in my version?)
<civodul>rlb: (sorry for the delay) yes, i think you can just "scm_install_gmp_memory_functions = 0" for now and switch to --enable-mini-gmp at a later point in time; there's no rush
<civodul>apologies for the rushed answers earlier!
<mwette>fnstudio: im not very familiar with map-or etc
<lampilelo>fnstudio: mwette: just for fun, another approach: https://paste.debian.net/1213138/
<fnstudio>lampilelo, mwette: awesome, thanks!
<mwette>lampilelo: some issue with that: or does nothing
<mwette>but I see where you are going: if performance matters I think there is a difference
<lampilelo>right, i rearranged some stuff before posting it and didn't think it through afterwards, there should be (and ...) around (or ...) (any ...)
<atuin>hi!
<atuin>is there a way to find the path of loaded module?
<dthompson>atuin: module-filename
<dthompson>
<atuin>mmm i tried that but I don't get the full path, just the module name
<atuin>`"srfi/srfi-26.scm"`
<atuin>something like that
<dsmith-work>atuin: Right. Need to search through the load path.
<dthompson>^ yeah
<atuin>mmm yeah my problem here is that seems i have guix modules in 2 paths, and i'm getting the wrong version of sudo in this case :D
<dthompson>follow the load path in order and the first match is the right one
<atuin>so it's followed in order always, ok
<atuin>I can do that
<dthompson>if you really want to be sure, then I'd run strace on the process and hunt for the open
<dsmith-work>Ya, strace reveals all
<atuin>yeah, that's another option
<atuin>I thought that maybe the struct returned in guile had that info
<dthompson>guile ought to print out the absolute file name of the module, though.
<dthompson>I wonder if there's a good reason why it doesn't.
<atuin>mmm no idea, but it would useful in this case :D
<atuin>I'm basically adding the %load-path returned by guix repl to be able to get definions from my current guix profile instead of from the system
<dsmith-work>scheme@(guile-user)> (search-path %load-path (module-filename (resolve-module '(srfi srfi-1))))$1 = "/usr/local/share/guile/3.0/srfi/srfi-1.scm"
<dsmith-work>
<dthompson>oh dang there's a search-path procedure!
<dthompson>did not know
<atuin>ohhhh
<atuin>nice
<atuin>yeah, it works nice
<dthompson>always learning something new
<atuin>now i'm confused that returns the right path, a file where sudo is defined to 1.9.8p1
<atuin>but sudo symbol in the repl is 1.9.7p1 :D
<atuin>ok, maybe it's time to use strace
<dsmith-work>Yeah, I think it would be nice to save the absolute path of a module once/if it's been loaded.
<dsmith-work>Searching through the %load-path could give different results at different times...
<atuin>dsmith-work so it's not deterministic then, or just it does not follow the order
<dthompson>I think they mean if the load path is modified between the module being loaded and you searching it
<dsmith-work>I mean if you search once, then the load-path changes, and then search again, you might find a different file.
<dsmith-work>What dthompson said
<atuin>ahhh yes, ofc
<dthompson>but it's an unlikely scenario in a controlled setting
<dthompson>guile isn't going to change things on you unless you tell it to
<dthompson>but in the general case it's a problem
<dthompson>so having the module hold onto the resolved absolute file name would be handy
<dthompson>would also just save us all a step :)
<dsmith-work>Right. Unlikely.
<dthompson>I've been through this recently writing a "tree shaker" that can make a bundle of all the guile modules that your program actually uses, leaving out the rest.
<atuin>mmm actually my problem is not related at all with the %load-path, maybe it's getting the module from the %load-compiled-path
<dthompson>the actual bytecode is read from there. the source files are only used for debugging.
<dthompson>does anyone know if it's possible to load *only* the compiled modules, without the source files present at all? I've never been able to get that to work.
<dthompson>it would be nice to be able to ship guile programs with only the compiled bytecode in order to greatly reduce download size.
<atuin>yeah, well adding the same compiled paths did not fix it :D
<atuin>using plain guile i still get teh wrong version of the symbol
<mwette>have %load-hook keep track?
<dsmith-work>dthompson: Yeah, I wanted that too. Like for an embedded system.
<mwette>I've used load-compiled (in the right order) with --no-auto-compile before.
<mwette>But that can be sensitive to guile-version.
<dthompson>mwette: yeah bytecode incompatibilities and all that. in my case I'm controlling the guile version.