IRC channel logs
2022-11-20.log
back to list of logs
<mwette>Hi all. I'm trying to set up fibers with one task that reads from a socket and another task that runs a repl. I tried setting socket to non-blocking and yield-current-task if socket dry. Also, using top-repl with readline and set-readline-read-hook to yield-current-task. Not working. Any ideas? I'm doing this suspendable-ports=#f and hz=0. See top proc here: https://paste.debian.net/1261266/ <mwette>The guile repl runs but the socket code never does. <ArneBab>old: guile 2.2.7: 2.06 slowdown, guile-3.0.8-main: 1.02 slowdown, guile-3.0.8 JIT=-1 (⇒ no jit): 2.25 slowdown. <old>ArneBab: Slowdown against which base? <ArneBab>old: slowdown against the fastest run in the specific test <old>which is a variant with Guile? <ArneBab>old: it’s the geometric mean of the slowdown in each test <old>So clearly JIT is helping here <ArneBab>turning off the JIT causes a 2.2x slowdown <old>What is the geometric deviations? Are they close to 1? <old>Just to know if results (speedup) are more predictable with the JIT or without <ArneBab>I can just pastebin the full results <old>Btw what benchmark are you running? <ArneBab>I should really write how to do this benchmark … <old>awesome results thanks <old>I'm sure it would be useful to other. Me first <ArneBab>old: one more result: with the JIT forced (=0), I get even higher performance for benchmarks — I don’t know how this would translate into production, though: Guile 2.2.7: 2.84, Guile 3.0.8: 1.40, Guile Jit: 1.01, Guile Nojit: 3.10 <ArneBab>old: this approach should work, but the setup-instructions (cloning and make and such) are untested. If you find missing pieces, please tell me! <count3rmeasure>does anyone know why the bytevector-u8-ref (and similar functions) aren't documented in the manual, from (rnrs bytevectors) or am I just not seeing them? <count3rmeasure>i noticed that guix uses them in the elf.scm file, but had missed them in the documentation, to be clear this is 3.0 code, and I was reading the manual for the appropriate version <cow_2001>old: i've read 10% of eintr.info by lines <cow_2001>even wrote a stupid looking command to check my current position in the text: printf "%s%%\n" "$(units --terse "`grep -n "$1" eintr.info | sed 's/:.*//'` / `wc eintr.info | col1`" "percent")" <count3rmeasure>up until I had discovered this I was unaware of any other code that uses the embedded guile interpreter in gdb, if anyone knows of other work I'd love to hear about it <old>ArneBab: Thank you. I will use that in my research on dynamic instrumentation! <old>cow_2001: no need to learn every errno :p. Typically you will see the errors you can get in the man pages of a syscall for example <old>Also quite good command for quick checking: `errno -l` <old>I think it's packaged with moreutils <old>ArneBab: By the way is there a reason for the geometric mean? <ArneBab>old: yes: the geometric mean avoids that a single test dominates the result. <cow_2001>old: it's elisp intro, not error ints O_O <cow_2001>so i am going into the emacs rabbit hole deep this time <old>cow_2001: Oh. Because the info manual of EINTR is about error number of libc lol <ArneBab>old: what exactly are you researching? <old>I do dynamic instrumentation at runtime. I change instruction of a program when it's running for instrumentation purpose <old>I use Guile as my testing tool. I can basically instrument every function entry/exit in libguile for example <old>I'm interested to see what's the overhead of diverting the computation to a probe and returning to the program <old>From my testing, an empty probe incurs around 20-30 ns of overhead per call <old>This is a micro-benchmark. I now want a macro-benchmark to see the impact on real cases <old>So I can say something along like: My method slow down the program by 10% <old>cow_2001: I should start reading elisp manual some day .. It's on my todo list .. <cow_2001>old: rabbit hole is deep and endlessly splitting <cow_2001>and merging, too. you would find it useful somehow in other fields, i am sure. <ArneBab>old: sounds interesting — is the use-case better debugging or profiling? <old>ArneBab: Both and more <old>I see runtime verification (e.g. bound checking); Hot patching for security vulnerabilities; Debbuging (very fast breakpoint); Tracepoints <old>I also have the idea of runtime fault injection to see what's the behavior in a software when a condition is met but it's a very rare condition <old>Basically, everything you can do with LD_PRELOAD, but during runtime and not just during loading! <old>cow_2001: Image how deep is the rabbit hole of the real manual, if this is just an intro ^^ <cow_2001>old: there is a game developer, Jon Blow, who calls rabbit holes "rat holes" <tohoyn>daviid: I already read your response <old>cow_2001: I know the name. He's the one developing the Jai language <cow_2001>old: we will both die before it is released ~_~ <old>Right. I haven't check it for a while now, but I remember cool features in it <old>Maybe in 10 years will see <old>I don't get why spend that much time on a language. I get that C++ is not something likeable, but there's alternative to it even for video game <old>Like D language. I bet the GC fear is still very prominent in that industry <lilyp>but video games have other bottlenecks, so... <old>GC is only triggered when an allocation is made <old>It's not a timer thing. So really you just don't do any allocation in your hotpath and you're fine <old>D has the @nogc attribute to ensure you don't make any allocation in your function for that reason <old>But I get that GC can be see as a beast that can eat your precious CPU time <old>lilyp: The 'you' are not targeted to your person :p <lilyp>Well, "don't do allocation in your hot path" is tricky enough if you don't have control over internals <lilyp>for instance, if you need to loop over a list in scheme, you have map and map! <old>True. This is why D has @nogc. you can only use functions that are also mark with @nogc <lilyp>the latter *may* alter the list so as to not allocate, but it also may not <old>And the possible runtime help you can get is diminish a little <old>D as manual and automatic allocation with the GC I think <old>So you can still do allocation but manually in your hotpath <old>There's probably some way of doing the same in Scheme <old>I'm not familiar enough with the VM of Guile though to assert that <old>Since a simple `cons` will do an allocation .. it might be tricky <lilyp>old: Scheme will optimize away allocations, but that requires digging into your compiler <lilyp>not fun, not even for c++ folk <old>True. But it's doable! <old>Not trivial that's for sure <lilyp>on the topic of c++, I do think more harm has been done to this world in order to avoid it than not <old>lilyp: You have an example? <cow_2001>i loved rustlang when i used it, but the community and the curl | sh and crates based installation is terrible. <cow_2001>better people than me say something about it having no spec <lilyp>tsk tsk tsk, the implementation is the spec ;) <old>lilyp: I agree with Go and Rust <old>I feel bad for D. It's a sane version of C++ that nobody is using :( <old>Meanwhile the cool kids Go and Rust get all the attention <lilyp>the problem is that there's a gnu compiler for D <lilyp>that means you don't get away with doing weird shit on your own hack of clang <singpolyma>D just wasn't enough of an improvement I think. It's still requires you to manage your own heap just like C, C++, zip, etc <old>D has garbage collection by default <singpolyma>old: sure, but then it's not a competitor with C or C++ at all in that mode <old>You can mark function where you don't want that feature. You can also just use what is call `betterC`. Basically all runtime of D stripped but you keep the meta-programmation <singpolyma>If I can afford a GC I'll use a high level language <old>Well betterC is a valid competitor I guess <old>It's C, but with fat pointer for array + size and meta-programming <old>Last time I check, they were adding borrowing like Rust I think <old>You also have RAII, modules, embedded unittest. Really all of that without a GC. To me that looks like a C++ competitor <old>And yet I've never used D myself .. <singpolyma>Yeah, I just think it's not enough improvement to entice away from C or C++ <old>ya that's the selling point of it IMO <old>that and memory checking I guess <old>I though that the point of Rust was to have to CVE? <old>The existante of CVEs in Rust invalid it's own existance then <lilyp>the point is that rust only really "fixes" one class of issues: those related to memory management <singpolyma>Luckily that's the most annoying issues for many of us :) <lilyp>and it doesn't even do that good of a job there, like any other language that invented the "unsafe" keyword <singpolyma>If I never have to call free again I will be very happy <old>singpolyma: Luckily we don't have this kind of problem in Scheme :-) <antipode>singpolyma,old: you can easily forget to (close-port ...) in Scheme <old>it's one thing that I don't really understand sometime in Scheme <old>You don't have memory leak, but you can have resource leaking like filedescriptor <old>Why not close the port on GC? If all reference to a port is lost, then you're good to close the underlyin filedescriptor right? Except if it's copied in a variable or in C <antipode>The file descriptor doesn't leak, eventually (ignoring conservativeness of the GC), the descriptor will be be closed once the port is dead. <antipode>What you are proposing is already implemented. <singpolyma>You usually want prompt closure though in say a server <old>There's no actual leak <old>yes make sens. You shutdown the socket when you're done with the client, not when the GC says so <old>But if you ever forget, you will not get ENFILE <antipode>It's mostly an optimisation (there is no GC-across-the-network) <antipode>old: For some fd-creating things in Guile, gc is called in case of ENFILE, but not for all, IIRC. <old>It does make sens though <old>But we have something real nice with dynamic-wind in Guile <antipode>To be clear, I was referring to ‘But if you ever forget, you will not get ENFILE’ -- to my knowledge, this is not always the case. <antipode>dynamic-wind is only nice under the assumption that you aren't doing fibers-style scheduling with continuations. <old>I've learn that the hard way by implementing my own userspace thread <antipode>Not a general solution, though it would be nice if there was a 'dynamic-wind-for-resource-cleanup-working-nice-with-scheduling' variant. <old>How about `dynamic-wind/maybe` :p <antipode>Maybe there could be some '%with-resource' parameter object, that defaults to (lambda (thunk clean) (dynamic-wind values thunk clean)) but that could be overridden by Guile Fibers. <old>What I do rn in my case is that I use a parameter that mark if the unwinding is caused by a context switch <old>That only me to not unlock async mutex for example <old>ofc, that does not work for dynamic-wind that are external to the library <antipode>... this %with-resource looks like a solution to me? (at least for close-port and Fiber stuff) <old>I don't think. The biggest problem is how do you detect if it's an exception or a context switch <old>To me the solution would be to have some kind of global state per thread that you set to #t before aborting to the prompt for context switching <old>This is what I do, but it only applies to my internal dynamic-wind <old>But if `dynamic-wind` would use that fluid everywhere in Guile, then problem solve <old>(fluid-set! execute-dynamic-wind-guards? #f) (abort-to-prompt ...) <old>And then reset it to #t <old>Then #f when you rewind to the captured continuation and then #t again just before continuing <old>dynamic-wind then just check the value of that fluid to determine if it should execute its guards <old>But I see your point with %with-resource. If dynamic-wind was a parameter, thing could be implemented that way <lilyp>singpolyma: the thing tho is that you never have to call free in C++ <lilyp>if you do, you're either doing something wrong or you're very deep in the std lib <singpolyma>lilyp: well, delete, free, remembering to pick the right smart pointer, whatever <old>I prefer malloc/free than a bunch of std::unique_ptr<MyVeryLongClass> everywhere <old>But I get the advtange of using that <old>I also get the disadvantage of 30 seconds compilation of a single unit <old>How's rust on compilation time? Is it worst than C++? <antipode>old: I think %with-resource will work, if fibers is modified to add a per-fibers variable (e.g. with a fluid) and sets it to #true/#false before suspending/restarting a task, (as seems to be part of your proposal), then Fibers can adjust %with-resource to see if its just rewinding or not. <old>Sure but what about dependencies that use dynamic-wind normally? <old>They are not aware of fibers internally so they can't use %with-resource. They are using dynamic-wind for example <old>Ofc you could be careful and cherry-pick what you call in a fibers/userspace thread <old>IMO, the model of dynamic-wind should be revisit internally in Guile to be more friendly toward context switching