IRC channel logs
2025-09-22.log
back to list of logs
<euouae>If it's a bad idea to invent a top-level module directory like 'data' where data structures can go -- similar to what Haskell has <euouae>e.g. do data/rope.scm and (use-modules (data rope)) <euouae>like a shared directory under prefix, others can also install under data/ <ieure>My end goal is to take a list of Guix packages and define transformed versions of them. <ieure>The linked code doesn't work, complaining "uninterned symbol cannot be saved to object file" <ieure>But Guile doesn't have `intern', so. I have no idea how to make this work. Nor what the equivalent define-syntax version of this might be. <ieure>Okay, `string->symbol' makes an interned one. <ieure>Still open to advice on how to express this as define-syntax instead of defmacro. <euouae>When using SRFI-41 streams, is the entire structure of iterable values stored in memory near the end of iteration? <euouae>i.e. if I have a stream of n values, does it mean that O(n) memory is used? <euouae>I'm thinking that this might be the case because of memoization <mange>I think it depends on whether you keep a reference to the earlier values. If you're working through the stream then the earlier values might be available to collect while you're going, if there are no references to them. <euouae>the values will be alive anyway, as I'm iterating over a data structure <euouae>what I'm worried about is extra O(n) memory associated with memoization for n values <euouae>i.e. suppose I convert a vector to a stream and I iterate over the stream. have there now been n associations i -> vec[i] in memory? <euouae>because the stream remembers (stream-ref strm i) right? <mange>A stream is like a linked list of pairs with a lazy car and cdr. If you hold onto the head, then yes, it will have to keep them all in memory. If you don't hold onto the head, then the earlier stream-pairs can be collected by the garbage collector while you're going, so it doesn't all have to be in memory at once. <euouae>They can be collected but nonetheless they're created <euouae>Thus there's O(n) overhead in stream iteration <mange>Unless there's some fancy optimisation I don't know about, yes. <mange>Not O(n) memory overhead, necessarily. <mange>Because the memory can be collected. <euouae>they're heavy loops is what I'm getting at <euouae>I wonder if I can get away with eager comprehensions, but I'm a bit terrified of the SRFI-42 extension mechanism <euouae>The only reason I'm trying to optimize as much as I can is because I intend to share this library and it's a general-purpose data structure <mange>(let loop ((stream (vector->stream my-vector))) (while (stream-pair? stream) (print (stream-car stream)) (loop (stream-cdr stream)))) should be able to collect as it goes. This means that the runtime will make a memory/cpu tradeoff by deciding when to collect the garbage that it's creating. If you're memory constrained this could do a collection on each iteration and have constant memory overhead. <euouae>Right, but regardless of whether O(n) memory lives at the same time or is created and collected immediately, it's still O(n) operations too <mange>Sure, so I'm happy to say "O(n) overhead", but not "O(n) memory overhead". <mange>Either way, I wouldn't use streams if you want performance. <mange>They're useful to separate two processes so a consumer can pull values as they need them, but that carries some overhead. <euouae>all I want is to iterate over the leaves of a binary (cons) tree (forward or reverse) starting at a given index i (corresponding to i-th leaf) <euouae>I'll try to use SRFI-42 and see if I can do it <mange>If you want performance I'd write a binary-tree-each which takes a tree and a procedure and does the iteration itself. You can have very low overhead for that. <euouae>that is an option but it is somewhat more cumbersome to the niceties of SRFI-42 or even streams (stream->list etc) <euouae>but thanks for the suggestion -- I might go with it if my SRFI-42 attempt fails. <mange>I'm not familiar with srfi-42, so I can't comment on that, but it looks reasonable. <euouae>it's a bit like common lisp's loop macro <euouae>you can say "for each string in list, for each character in string and its index, do/fold/collect/etc " <euouae>hm the extension mechanism is quite simple actually <euouae>all you need is to define a generator thunk that returns a sentinel to signify end of generation <euouae>cool, I managed to write a generator for binary trees using SRFI-42 <ArneBab>SRFI 42 feels really nice - I turned to it initially when switching from Python to Guile because it feels like list comprehensions. It somehow fell to the wayside for me, though. and I’m now mostly using let loop directly. <euouae>for certain trees providing an SRFI-42 generator for their leaves is nice <euouae>because handrolled loops are messy <omentic>anyone have experience with guile hoot, by the way? need to convert js's (many) notions of lists to a scheme native form over the ffi and am not sure what a reasonable way to do so is <omentic>i have an unreasonable way to do so... (iterating and creating a scheme list element-by-element, every time) <ArneBab>omentic: not much experience myself, but dthompson has. Maybe ask in #spritely <omentic>ArneBab: i have! but thx for the suggestion. they basically side-step this, no bindings for anything involving HTMLCollections/NodeLists/etc <ArneBab>omentic: sounds like your work could be very important for others (including future me), too <wehlutyk>I'm trying to take WAYLAND_DISPLAY=wayland-1 <wehlutyk>to run it inside a second wayland inside gnome (as I use to try out GWW) <wehlutyk>is this possible, or not taken into account as an env variable? (i.e., is that a new feature, or an available feature for which I just have a problem on my side?) <kestrelwx>wehlutyk: Are you sure the second display isn't wayland-2? <wehlutyk>`env | grep WAYLAND` gives `WAYLAND_DISPLAY=wayland-0` <dthompson>chickadee uses sdl2 currently and just asks for it to open a window <wehlutyk>yeah! I'm trying to figure out how how sdl2 would take that <wehlutyk>and if it would be given to sdl2 even if only calling chickadee <kestrelwx>It seems to use XWayland by default, I've got a window on another monitor with 'SDL_VIDEODRIVER=wayland WAYLAND_DISPLAY=$my-second-display' <kestrelwx>I didn't notice it was a chickadee question at first. <kestrelwx>Well, not another monitor, another wayland session. <mwette>dthompson: I believe $WAYLAND_DISPLAY is the name of the wayland socket under $XDG_RUNDIMR_DIR, where the socket gives connection to the compositor. <mwette>So, I'm guessing SDL_VIDEODRIVER tells SDL to use the wayland protocol on that socket instead of the X11 protocol on the X11 socket. <dthompson>mwette: yeah I suppose so. not sure how sdl implements things <dthompson>all I know is that it's not something that chickadee has any control over <kestrelwx>I think recent SDL prefers Wayland over X11. <kestrelwx>I think in older versions you need to set the videodriver hint to "wayland,x11" in app. But I don't quite remember. <dthompson>okay so maybe chickadee does have some control through hints <dthompson>but you could try modifying chickadee to set the hint and see if it works <kestrelwx>(define hint ((@ (system foreign) string->pointer) "SDL_VIDEODRIVER")) <kestrelwx>(define hint-val ((@ (system foreign) string->pointer) "wayland")) <kestrelwx>((@@ (sdl2 bindings) sdl-set-hint-with-priority) hint hint-val 1) <kestrelwx>"wayland,x11" also seems to work and shouldn't crash on x11. <kestrelwx>Might have to make sure one is on Linux before setting that, though. <dsmith>I asked about the fbsd addclosefrom_np thing in #autotools <dsmith>xcvb says: guile asked for gnulib, and gnulib goes "allow me, then", so it's what you get. This is probably a change required in gnulib itself to make it recognize that fbsd now has addclose <rlb>Or, we'd have to vendor the gnulib version somehow and enable it more manually/selectively? <rlb>Not saying we should, but suppose we might be able to if it weren't "too hard". <rlb>(until they get that change, assuming it's not imminent) <dsmith>Ya. Poking at it to see if that's viable. <dsmith>ACTION doesn't *want* to know auto* tools <rlb>For something like this, might be fine to be fairly crude? <rlb>Say if we can "easily" probe with a test.c for native support, and if so, don't ask for (or disable) gnulib's support, otherwise, do what we've always done? <rlb>And I'd say "go ride a bike" -- this isn't going anywhere :) <dsmith>In m4/gnulib-comp.m4, there is a gl_INIT. In there is sever POSIX_SPAWN_* tests <rlb>My worry of course is that it's too deeply entangled somehow -- is that what the gl_INIT bit might imply? <euouae>I've written an M4 implementation in Python so at least I got that down <euouae>I actually did it of my own volition too <euouae>Thank you, the maintainer of m4 helped me once the project got traction <euouae>Autotools is mostly m4 + perl, and the knowledge of how it's all put together and what everything means in the autotools world <dsmith>I think what is going on is several posix_spawn_* functions are looked for, and if one doesn't exist, then it falls back to gnulib. The addcosefrom is NOT being looked for. <dsmith>The guile auto* stuff is actually specifially checking for that. <dsmith>With AC_CHECK_FUNCS([..., posix_spawn_file_actions_addclosefrom_np, ...]) <euouae>so what are you saying you think is happening? that it doesn't check? <euouae>Let me see what AC_CHECK_FUNCS does <dsmith>Guile looks for it, and wants to use it. gnulib independently decides to provide posix_spawn_* things for some reason (not sure why just yet) <dsmith>And as we know, the mixing of the two causes issues. <euouae>I was just checking to see if there's some codeberg concept where wiki-like issues can be discussed so that we can collaboratetively put notes on this issue <dsmith>So *why* is gnulib posix_spawn_* code chosen? <euouae>I think there might be such a feature but it's too much to figure out right now <euouae>First I'm trynig to figure out what AC_CHECK_FUNCS does and how <euouae>Okay, AC_CHECK_FUNCS sets a symbol if a function is available. It's done by autoheader. <euouae>One thing I note in "Generic Function Checks" from autoconf info manual is "If you need to check the behavior of a function as well as find out whether it is present, you have to write your own test for it" <euouae>So keep that in mind, that there is a possibility to write a more elaborate test <euouae>is that from guile/configure.ac? <dsmith>I wonder why gl_POSIX_SPAWN is there 3 times? <euouae>right, there's some bits for POSIX_SPAWN there <euouae>oh don't go down that path -- it's all state-dependent <euouae>gl_POSIX_SPAWN is probably doing some checks based on the state of m4, and they run it three times probably to say "okay, if you failed earlier, maybe now at a more elaborate check? <euouae>Anyway, autoheader will define HAVE_* symbols after processing AC_CHECK_FUNCS <euouae>so what happens on FreeBSD? Does it define HAVE_posix_spawn_file_actions_addclosefrom_np? <euouae>You can use MESSAGE(...) to see during ./configure <euouae>We're trying to debug a problem that's not present in fbsd 13 but present in fbsd 14 correct? <dsmith>I suspect that's true. Have not tried in 13 <euouae>ah it upcases the macro, so it should be HAVE_POSIX_SPAWN_FILE_ACTIONS_ADDCLOSEFROM_NP <euouae>I think Guile has correct code if the _np function is missing, but it otherwise breaks <euouae>so probably works fine in fbsd 13 since _np is added in fbsd 14 <euouae>okay, and guile is only affected in two places by the presence/absence of this macro: <dsmith>The actual problem point is #ifdef HAVE_ADDCLOSEFROM in libguile/posix.c: do_spawn <euouae>One file, libguile/posix.c and HAVE_POSIX_SPAWN_FILE_ACTIONS_ADDCLOSEFROM_NP is "renamed" to HAVE_ADDCLOSEFROM, and then <euouae>and then that macro is used in 2 places: in absence, define close_inherited_fds(), and then if defined uses `spawn_file_actions_addclosefrom_np(&actions, 3)` else `close_inherited_fds(&actions, max_fd)` <rlb>Without really understanding -- I imagine you were looking to see if we could just make the posix.c decision "fancier" too? <dsmith>That's why I was thinking just remove the AC_CHECK_FUNC for it <euouae>If you remove the check, you're essentially removing whatever differentiation the author of posix.c made there <rlb>(or maybe that doesn't make sense) <euouae>He says "This function appears in glibc 2.34. It's both free from race conditions and more efficient than the alternative." <euouae>So ideally you'd like to keep it, not remove it <euouae>Otherwise you're penalizing those who would benefit from it <dsmith>Just saying that removing that check allowed both fbsd and linux (Deb Trixie) to function <euouae>dsmith: what you need to do is change `#ifdef HAVE_ADDCLOSEFROM` to `#if defined(HAVE_ADDCLOSEFROM) && !defined(USE_GNULIB)` <euouae>right? because that function is incompatible with gnulib, am I getting it right? <euouae>especially USE_GNULIB_SPAWN is what is needed, that kind of macro <dsmith>At this time, right. Cause gnulib does not (re)implement it <euouae>in fact both uses of HAVE_ADDCLOSEFROM should be done like that <euouae>because you don't need to define close_inherited_fds() when you can use the alternative <euouae>I'd replace HAVE_ADDCLOSEFROM with USE_ADDCLOSEFROM, and I'd define it according to that logic <euouae>What's missing now is this: How do we know that we're using gnulib's spawn functions? I don't know that part, maybe you do? <dsmith>I do think that path along these lines will "fix" the issue. However, I'm still wondering why gnulib is chosed here over native. <euouae>I could install freebsd and minimize & bisect guile/configure.ac to see what the m4/gnulib logic is <euouae>dsmith: can you use nm(1) to see where the symbols come from? <euouae>you'll see which spawns are from fbsd and which from gnulib <dsmith>The gnulib ones are renamed using #defines <euouae>so all of the spawn functions are gnulib <euouae>or do some spawns survive from fbsd? I mean clearly _np does <dsmith>euouae, I did not include anything related to this in PR #17 at this time. <euouae>dsmith: there is no easy way to keep notes on an issue in guile's codeberg right now that I can tell <euouae>dsmith: do you want to open a dummy PR? or if you have your own fork, open a dummy PR there <euouae>It just helps to summarize some of the findings because *I* have a bad memory <dsmith>euouae, rlb was going to file a bug on the spawn prob.... <dsmith>I don't know. Are we still using the lists exclusively? Or does codeberg have some kind of issue tracker? <euouae>codeberg has Issues but you need to enable them from repo settings <rlb>Except for "transient" information, I'd still favor the list or the bug tracker, unless we're planning to move to codeberg issues wholesale (we may be) because without a solid intention (and the resources in the future) to export that data "somewhere else", the forges are much more likely to eventually lose it. <euouae>I do prefer mailing lists too but with one caveat, that they make editing impossible and that sucks <rlb>Where historically, eventually isn't all that long < decade for the links to eventually break, etc. <rlb>And no, hadn't gotten to filing the bug yet. <rlb>I'm tied up with some other things, so yeah, please do if you like. <euouae>I want to summarize what we've said above <rlb>I'm a bit more sanguine about codeberg, though since it seems to have a solid api, and you *could* export it, but I've also poked at the api, and it's a lot more complex than "copy a maildir" -- and I think debbugs is "just" a postgres db. <euouae>dsmith: apparently posix_spawn_file_actions_addclosefrom_np() was implemented in FreeBSD 13.1 first. I'd be surprised if Guile 3.0.10 does not crash there? <euouae>oh wait, the FreeBSD 14.3 manual says posix_spawn_file_actions_addclosefrom_np() first appeared in 13.1 but the 13.1 manual says it appeared in 14.0! <euouae>I think their docs have a bug, sigh. I need to dig into their git repo <dsmith>The ports tree for guile have patches where they work around it. <rlb>I suppose if it sounds like there's any chance gnulib's going to fix it soonish, we could also just upgrade that before the release. <dsmith>I tried to find one the other day.. It's under /usr/ports/language/guile3/ <euouae>my only other recommendation is to remove the renaming of the HAVE_POSIX_SPAWN_FILE_ACTIONS_ADDCLOSEFROM_NP macro and use it directly <euouae>HAVE_ADDCLOSEFROM looks like an autoconf macro for the addclosefrom() function which is bad <dsmith>Yeah, don't know why that was changed.. <euouae>probably for brevity someone didn't like the long symbol <euouae>you can use \ to break up lines if they get too long <euouae>it's okay, he's not going to take it personally if we revert that ;P <dpk>yeah, the lack of addclosefrom in the standard is a real mess <dpk>actually the whole file actions thing is a real mess, however good the posix_spawn idea is conceptually <dpk>ACTION wonders if the memory leak she reported in Guile’s posix_spawn usage was fixed yet <dpk>ACTION has been meaning to write a blog post about the greatness and awfulness of posix_spawn … <dsmith>dpk, I never knew about it before this fbsd issue. <euouae>Hm, probably the _np function has been introduced in 13.1, so the later manuals are correct. <dsmith>euouae, The fun thing is, there is more documentation on the _np things in freebsd than for glibc! <dpk>nope, the memory leak is still there <euouae>hehe, yeah, well I've got to go. I won't post in bug-devel yet <euouae>you might've figured it out with that patch you posted <dpk>maybe i never reported it to debbugs (probably, because sigh, debbugs) <dpk>the file actions are initialized but never freed <dpk>there should be a posix_spawn_file_actions_destroy around line 1429 <dpk>if someone else wants to claim the glory, i’m going to bed now and can’t write a patch or report this, so you have at least a few hours in which you could report and/or fix it instead of me :D <dsmith>And of course "gmake check" passes <dsmith>Shoudld I add that to PR17 or make a new one specifically for that issue? <rlb>Either's fine, as long as (I'd say) it's a separate commit. Enough things that probably need some "explaining" in either the commit messsage, or comments. <rlb>i.e. be good to have a summary of y'all's findings, etc. <rlb>...for the poor sods (likely one of us) who has to work on this next time.