IRC channel logs
2025-10-16.log
back to list of logs
<dpk>does anyone actually use (ice-9 calling)? (you’re not allowed to look up what it is before answering) <dsmith>dpk, That file is old. Looks like from 97 <dpk>the last substantive change to it was in the year two thousand <dpk>indeed, that’s how i noticed it <dpk>i was wondering how much work it would be to get rid of the remaining defmacros in Guile … <dpk>it would probably be just as well to delete this module. it might break scwm – but does that even run under modern Guile? <dsmith>Oi! I haven't touched scwm in ages. <dpk>(i’m just guessing this was an scwm thing, because the earliest commit says it was contributed by Maciej Stachowiak …) <dsmith>Greg Badros (sp?) was the scwm guy (IIRC) <dsmith>scwm was my gateway drug into scheme <dsmith>Guile was very slow to statup back then, and it was nice to have a "hot" scheme running in your WM. You could pass expressions to it over X11 <dsmith>It was accumulating a lot of bitrot, and when I sent in some patches to tune it it up, they made me the maintainer. <apteryx>is there some primitive in Guile to limit a program segment execution to a single thread? <apteryx>disbling GC, signal threads, any kind of threads Guile might normally spawn during its execution. <old>apteryx: maybe GC_NPROCS=1 <old>from libgc: Linux w/threads only. Explicitly sets the number of processors <old>that the GC should expect to use. Note that setting this to 1 <old>when multiple processors are available will preserve <old>correctness, but may lead to really horrible performance <old>since the lock implementation will immediately yield without <apteryx>that's just for gc though, right? And I'd like to set this programmatically. <old>but that's global and is applied when loading the program <old>what is your intent here? <old>you want to reach a quiscent state when forking in a multi-thread program <old>you might want to use pthread_atfork for that if it can help <apteryx>old: yes, if quiescent means single thread ;-) <old>quiscent means reaching a stable states that is safe for forking <old>for example, if you have mutexes that are held by other threads and you need them to not be locked before forking <old>in other words, you reach a point where you are still mutli-threaded, but you guaranteed that your process is like a single thread program <old>I'm not familiar with the unshare syscall unfortunatelly so I don't know what are the prerequisites for it <apteryx>no other threads running when it's called (single threaded, or "it is not sharing its address space with another process or thread"), per 'man 2 unshare'. <apteryx>thanks for the explanation. is pthread_atfork also useful for clone(2)? <old>apteryx: unforutnatelly no. only for fork <old>But clone should not be a problem with unshare in that case unless CLONE_VM is passed to it <apteryx>it'd be used via e.g. (without-threads EXP) <old>but hm look at the strace log you provided <old>80492 21:39:43.562152988 2 guix (1453.1453) > unshare flags=1048608(CLONE_NEWNS|CLONE_NEWUSER) <old>80493 21:39:43.562154088 2 guix (1453.1453) < unshare res=-22(EINVAL) <old>EINVAL CLONE_THREAD, CLONE_SIGHAND, or CLONE_VM was specified in flags, and the caller is multithreaded. <old>none of the above flags are used <apteryx>it's subttle CLONE_NEWNS implies CLONE_VM, IIRC <old>hmm I don't think so <old>101:# define CLONE_NEWNS 0x00020000 /* Set to create new namespace. */ <old>89:# define CLONE_VM 0x00000100 /* Set if VM shared between processes. */ <old>90:# define CLONE_FS 0x00000200 /* Set if fs info shared between processes. */ <old>It is not permitted to specify both CLONE_NEWNS and CLONE_FS in the same clone call. <apteryx>ah, the problem is "specifying CLONE_NEWUSER automatically implies CLONE_THREAD.", and CLONE_THREAD implies CLONE_VM, eh. <apteryx>so CLONE_NEWUSER requires single thread operation to avoid the EINVAL from unshare <old>and the unshare is done right after a clone syscall in the child process? <old>(I don't see the clone syscall in the strace log) <apteryx>someone found that stopping the GC threads with (disable-gc) was enough to fix the issue for them, but I'd rather a more bulletproof solution taking care to stop/halt all other threads also (finalizers and signals, I heard) <apteryx>old: yes, the child in clone does some mounts and then calls unshare <old>in linux-container.scm right? <apteryx>old: yeah, I couldn't capture the clone (child part) with strace, because adding -f wouldn't reproduce the issue (too slow) <apteryx>I have other captures with sysdig (bpf-based) and perf trace though <old>so I don't know the internal details of libgc, but it seems that it spawn new thread in your child process <old>that's kind of impressive that it managed to keep working after a fork <apteryx>impressive of the garbage collector? <apteryx>I guess it's expected in that there's a non-trivial procedure called, 'mount-file-systems' in the child, running Scheme code, which would cause the GC to get triggered, I guess. <old>apteryx: yes. Usually library that are multi-threaded can recover from a fork/clone <old>and setting GC_MARKERS=1 works? <apteryx>it's one of the first thing I had tried I think <apteryx>it makes sense; the guile chid is containerized, isolated from such environment variable <old>GC_set_markers_count <old>while the GC is disabled <old>GC_set_markers_count(1) <apteryx>old: this is callable from Scheme? GC_set_markers_count <old>apteryx: with FFI I suppose <apteryx>But 1 GC thread would still be a problem, right? <apteryx>We want only our execution thread, nothing else <old>I think setting to 1 will just use your thread for marking <apteryx>so (disable-gc) is more appropriate (that's actually callable, TIL) <old>yes but depending on the amount of work you need to do before calling execve <old>apteryx: I tried a little thing in the REPL <old>it seems that the GC spawn a single thead <old>even without touching GC_MARKERS <old>that's your problem for sure <old>what I would suggest. Before the clone, do a call to (gc) <apteryx>Great! The GC be disabled before clone(2) to be safe, correct? <old>that way you free the memory just before the clone call so you have more memory in the child <old>then, disable the GC before the clone and enable it back in the parent after <apteryx>ah, it's (gc-disable) and (gc-enable) :-) <old>yes I would to avoid any possibility of allocation that would trigger the gc <apteryx>that's neat, I didn't know we could control the GC so easily <old>lots of things are not documented :-p <apteryx>at least all the public API should be, in my opinion, we'll get there! <morenonatural>hey, y'all… I have a growing number of elements that I want to sort after getting them… <morenonatural>I was thinking of using vlists, but it seems they don't have sorting functions <morenonatural>should use a different "growing container" ? just manually copy elements to a vector? <identity>morenonatural: why do you need the elements to be in a vector? <morenonatural>identity: to sort them… they're paths and I wanna sort them by mtime <identity>dthompson: i think there is an echo in here :) <rlb>cwebber dthompson: congratulations <rlb>daviid: hmm, so after looking at your approach and what I did last time, could I also have just let close exist in the module (dropped #:hide), called (define-generic close) to promote it, and then continued to #:replace it? <rlb>i.e. maybe I didn't actually need to #:hide and then grab the original binding via @. <rlb>(Probably not important, just wondered.) <dthompson>I got disconnected briefly and didn't realize cwebber already dropped the link <daviid>dthompson: starling is inaccessible for me - I wanted to suggest euouae to also read your some of your code ... <dthompson>my git server has been periodically DoS'd by AI training bots so sorry about the 504s there's not much I can do <dthompson>I've blocked all the bots that identify themselves so now the really bad ones that spoof their user agent string are what I get <daviid>dthompson: ah ok, catbird it is then :-) tx - yes, AI training bots are, among other problems, a nightmare for hosting admins ... <dthompson>I need to setup something that rate limits based on IP... not a perfect solution and I hate doing iptables stuff so idk when I'll get around to it <dthompson>I don't want to run anubis or anything like that either <mra>quick questions: in guile fibers, are channels MPSC? that is to say, is it okay to have multiple fibers sending through a channel, as long as there is only a single reader? <identity>dthompson: rate limiting based on IP address is questionable as the bots have /many/ addresses <dthompson>right that's the "not a perfect solution" part <identity>what i mean is, i highly doubt it is going to have any impact at all <identity>mpsc is Multiple Producer/Single Consumer <dthompson>mra: multiple fibers can read from and write to channels <mra>dthompson: sorry, rust terminology. "multiple producer single consumer" <mra>wasn't sure if it was general terminology or rust-specific <dthompson>I just didn't recognize the acronym. I'm good now :) <dthompson>channels have 2 queues inside of them: a get queue and a put queue <dthompson>readers wait on the get queue and writers wait on the put queue <dthompson>when you read from a channel the fiber suspends until there's a message available to read. <dthompson>when to write to a channel the fiber suspends until there's a reader avaialble <dthompson>gotta go afk but hope that helps and wasn't just rehashing what you already know. apologies if so. <mra>dthompson: no, that's great! clarifies my mental model. i was thinking too much of channels as being like queues, but they aren't like that <mra>you need a simultaneous reader and writer. channels aren't like mailboxes <mra>i'll leave one more question here: is there a way to think of fibers as having return values? in my mind, if i have a function f, it seems like i can sort of call it asynchronously by spawning a channel and a fiber which runs f and then writes the output of f into the channel <mra>it seems like that channel then acts a bit like a promise, and reading from it is like awaiting <mra>just want to check if that's a reasonable understanding <kjartano`>mra: I built an abstraction like that a little while ago. <kjartano`>I am currently in the process trying to replace it with Goblins. <mra>ah, okay! good to know that I'm at least thinking about things in a reasonable way <mra>kjartano`: how are you using Goblins to replace this? my mental model of Goblins doesn't really involve concurrency, per se <mra>but then I'm not super knowledgeable about Goblins <kjartano`>I am trying to use their promises instead of my custom solution with that spawn. <kjartano`>It is turning out to be a significantly larger project than I initally expected. <kjartano`>But it is teaching me a significant amount about Goblins and Hoot. <mra>kjartano`: ah, gotcha! i think that for what i'm tinkering with fibers are probably more appropriate, but that's cool <mra>what are you working on? <kjartano`>The web server provides a UI and serves up files that are produced by the Make like system. <mra>ooh, that's interesting!