IRC channel logs
2025-09-18.log
back to list of logs
<euouae>does anyone know if with srfi-64 it is possible to print the random state? *random-state* to replicate fuzzying unit tests? <euouae>oh I see, I need to edit my test driver <euouae>Maybe I can simply conjoin (random-state->datum *random-state*) to the test name <euouae>then I can do something like (set! *random-state* (datum->random-state '(multiply-with-carry 202862877 196301851))) to replicate the test <euouae>hm, no good, I don't seem able to replicate the random state from the test for some reason. <euouae>ah I'm printing it at the wrong moment, heh <euouae>ACTION will have to think about this some day <rlb>dsmith: I might have been thinking of some of the stuff in build-aux/ <kestrelwx>Is '(sxml simple)' expected to be able to work with valid HTML? I understand 'xml->sxml' currently expects uppercase 'DOCTYPE', whereas the standard allows any case. <kestrelwx>It works fine if I skip the '<!doctype>' line of course. <identity>last time i checked SG/XML require DOCTYPE to be upppercase, thought case-insensitivity may be an HTML5 thing <identity>HTML5 is not strictly SGML-based, so probably that <kestrelwx>That's my question. Makes sense if it doesn't promise that. <ArneBab>If that doesn’t load, just install guile-lib and use the info reader for Guile Library. <kestrelwx>I'll look into it if SXML won't be enough for me, thanks. <kestrelwx>Yea, way less trouble just going with 'htmlprag'. <ArneBab>ACTION reached the same conclusion two years ago. htmlprag actually also provides pretty nice html generation. <rlb>dpk: Hmm, I may take a closer look later, but at least for me, I don't have much experience with define-library or rnrs records yet, and both issues also seem likely to call for a look by Andy or Ludovic, whatever I might end up thinking. I also wondered whether the second was something we would change in a Z release -- if not, might make sense to tag it for the 4.0 Andy appears to be planning. <rlb>If nothing else I'll add the tag I started using for that. <dpk>also, is there any plan to move issues and not only patches from debbugs to Codeberg? :D <rlb>Though for new things it appears clear that pr's are acceptable. <rlb>(Wonder if anyone's working on a "fancy" emacs mode for codeberg like the one for the debian bug tracker -- though I haven't actually used that one all that much yet.) <rlb>Perhaps less likely. <ieure>rlb, There's a Guix package. <rlb>Not a big deal, but right now what I'd like is some way to migrate a few prs/issues between two repos so I can merge my pre-migration rlb/guile repo with a real fork of guile/guile -- though I'm probably just going to rename my repo (had little activity) and make a new rlb/guile as a fork. <rlb>(Not really worth much distraction.) <rlb>i.e. I suspect it *can* work fine on freebsd if the conditions are right (fs, etc.). But skipping is fine for now. <dsmith>> File systems are allowed to expose ranges of zeros with SEEK_HOLE, but not required to. <dsmith>> Applications should use fpathconf(_PC_MIN_HOLE_SIZE) or pathconf(_PC_MIN_HOLE_SIZE) to determine if a file system supports SEEK_HOLE. See pathconf(2). <rlb>Oh, thanks, not sure I'd seen that. Notable bits of what bup does was just via accumulated experience/reports. <dsmith>In my testing 14.3 on ufs under vmware, it was behaving as if there was no hole. <rlb>...now I'm curious. One moment please :) <rlb>looks like test-sparse-files does detect/test sparseness in my vm, but it's qemu/kvm and looks like it installed as zfs. <rlb>anyway, doesn't matter -- skipping is fine for now <rlb>dsmith: hmm, how about "date --version | grep -q GNU" <dsmith>Posix date doesn't have --version (or any -- long args). I figured that would be enough. <ieure>rlb, I don't think --version is POSIX. At least, the BSD /bin/date tells me "/bin/date: illegal option -- -" <rlb>Right, it'll fail for non-gnu, and that's the signal? <rlb>Just thought it'd be a bit more specific. <ieure>BSD `date --version returns a status of 1. <dsmith>What I don't know, is if the other *BSD are the same.. <ieure>I think userspace is very similar across the BSDs. <rlb>right, also wondered about that, i.e. was the non-gnu invocation posix, or possibly just another custom case. <rlb>(haven't looked yet) <rlb>In any case, I just thought for setting GNU_DATE, grepping for GNU was a bit more narrowly construed. <dsmith>Yeah, I didn't know how positively test for posix, so I went the other way around. <rlb>Do you already know if the non-gnu invocation is posix? Otherwise, I'll go check. <rlb>Not critical, but I suppose another option would be to just have some GUILE_MAKE_BUILD_DATE with ne of those two commands in it (after testing to see which one doesn't crash). <rlb>Since neither is standard -- and of course can always add more variant tests later if we ever need to. <dsmith>It looks like BUILD_DATE is for repeatable build? Or just lying about when it was built? <rlb>No idea -- hadn't even looked to see what it was actually doing. If we could count on bash (I assume we can't) there might also be printf. <dsmith>Right. The main /bin/sh is... sh <rlb>Yeah, I just assume autoconf based projects generally intend to work with posix sh in the limiting case. <rlb>(and very likely guile's included) <dsmith>What I haven't looked into at all is readline. <rlb>In any case, not worth too much time/effort -- I'd say maybe eitehr the "grep GNU" for now, or the "test each variant" to find one that works. <rlb>the latter's more flexible, but a bit more change, and either's better than what we have for now. <dsmith>Grepping for GNU is fine. Belt-and-suspenders <rlb>One other bit -- do we care that the sed -i isn't portable? <dsmith>Should have been a separate commit? <rlb>Not critical IMO - might also have been fine to have all of the changes in one "Fix build compatibility with FreeBSD (sed, date, etc.)" or something :) <rlb>Also, assuming the previous release was broken on this front, if you want to, could also add a "Guile will now build on FreeBSD" to the "Changes to the distribution" section (or wherever) ni the NEWS. <dsmith>There is another piece I have not commited yet. My change works on both fbsd 14.3 and Trixie, but I'm not sure if it's the right thing. <rlb>Oh, OK, well then perhaps could save the news entry for the commit(s) where we get everything working again. <rlb>And yeah, I didn't quite get to the point of understanding exactly what was going on there either. <rlb>Just have a note to file a bug later :) <dsmith>The system <spawn.h> and the glib version have different data structures. Havoc ensues when you mix them. <euouae>the mixing is an overlooked error then right? <euouae>I was reading the last bug report you linked to <dsmith>Yes, there is a warning about passing a pointer to the wrong type <dsmith>But executing code results in a segfault <euouae>do we know if this bug is a small one or a big one? <rlb>Causes segfaults, so "big"? <euouae>is it endemic inside the design or just some mixup <dsmith>Well, segfault you know. Pretty huge <euouae>I mean in terms of how large a patch to fix it would be <rlb>i.e. autoconf is setting things up to mix/match the components, which isn't a thing (iiuc). <euouae>I mean it happens on 3.0.10 and not 3.0.9. Was there a major rewrite of some component causing this in 3.0.10? <euouae>I know plenty of autotools and I'm OK with C and/or threads, I'll try to install FreeBSD and see how far I can get debugging it. Are we trying to fix it on 3.0.10 or HEAD? <dsmith>The workaround is to pass ac_cv_func_posix_spawn_file_actions_addclosefrom_np=no at ./configure time <euouae>"It should be investigated why FreeBSD's native posix_spawn(3) is not enough for Guile and why it insists on using rpl_posix_spawn*()." <-- this is what I want to fix <euouae>It seems that one thing that can be done is to add logic that says "if you use rpl_posix_spawn*() then do not look up posix_spawn_file_actions_addclosefrom_np()" <dsmith>So, does adding AC_CHECK_HEADERS([spawn.h]) change hoe glib is used? Maybe there needs to be some conditionals in there... <rlb>dsmith: current situation is that your fix works, but we not *sure* about it, or do you already know that the main issue is that you just have to make sure the header is included to get everything "lined up" definitions-wise, etc.? <dsmith>rlb, "works-for-me", but what about other systems than two I tested? <dsmith>I don't know at all how this glib stuff is meant to be used. I understand at a high level, but not the machinery. <euouae>are you talking about gnulib or glibc <dsmith>no. glib. The "local hand-rooled" function used when a system doesn't have it natively <rlb>Oh, is that fallback glib? I'd assumed it was gnulib or similar (or maybe you meant glib->gnulib). <rlb>oh, nvm I was mentally conflating glib as glibc :) <rlb>And yeah, it seemed like the key issue was that the fallback had to be all or nothing wrt several bindings, not "pick and choose". <rlb>I was hoping that maybe it was as simple as "if you actually include the header, you'll get the full set 'everywhere we care about'". <rlb>Also, if it always crashes in the tests when it's wrong, then we can be more adventurous? <rlb>i.e. improvement is improvement? <dsmith>YEs, I was hoping that too. But merely adding a check for the header was not enough. <dsmith>It's not just the tests that crash. it's any spawn <dsmith>It may be there needs to be some #if condidionals around the glib code? Or in a Makefile? <euouae>dsmith: I'm still confused, glib or gnulib? <rlb>So do you know why removing that posix_spawn_file_actions_addclosefrom_np check func "fixed" it? <rlb>(sorry if you already said) <euouae>dsmith: Are you talking about gnulib, the project you linked or not? <rlb>euouae: that project is what provides a bunch of fallbacks and selectively choosing some of the spawn related functions from there, but others from native bsd is what's causing the havoc, I think. <dsmith>I've seen the project I've linked to refered to both as gnulib and glib <euouae>I think you're right, I had forgotten that it's referred to as glib <rlb>i.e. configure's picking the wrong things one way or another. <euouae>I don't think you need to fix this at C level <euouae>with cpp conditionals, what you need is to fix the configure.ac <euouae>I think can do it, I'll try now as we speak <dsmith>So. Does anoyone know of a system that does *NOT* have <spawn.h> (and the implementaion) <rlb>dsmith: so we only check for that posix_spawn related function, but not posix_spawn(p) itself? <rlb>If so, vaguely wonder if testing for those first might "help"... <dsmith>I think so. Just looking for posix_spawn_file_actions_addclosefrom_np becuse it was not in early glibc? <dsmith>The comment mentions glibc >= 2.34 <rlb>Actually, wait -- I suppose the top level question is what's feasible, i.e. it could be just wrong to check for that function in isolation, because you might pick the fallback on a platform that doesn't have it, but does have posix spawn more generally, and unless that function is implemented in a way that's guaranteed to work with any local spawns (and sounds like it's not), then this is just wrong? <euouae>is it crazy to add a custom .c program that uses all of the above and if it segfaults just don't use posix_spawn_file_actions_addclosefrom_np? <rlb>If so, then I think (just speculating) that configure needs to be more sophisticated, i.e. it needs to test the platform first with the gnulib fallbacks "masked" somehow, and if that's insufficient, just switch to the fallbacks wholesale (if they'll work). <euouae>that way you can avoid tricky logic on which combinations work <rlb>(And I think that lines up with some of the comments in the fbsd bug at least.) <rlb>euouae: I think the issue might still be relevant to the test program, i.e. spawn will work fine on freebsd, but only if you don't build in a way that picks up any of the glib fallbacks. <rlb>And the question is how to avoid that. <rlb>dsmith: but yeah, at the moment, that lone posix_spawn_file_actions_addclosefrom_np seems questionable to me. <rlb>I have no idea if there's a general/easy way to "suppress" gnulib during a test, and then suppress it's implementation for a thing completely if the test result says you want only the platform. <dsmith>It looks to me like this initial code was just for a deficiency in glibc, not as a general "does this system have posix_spawn_*" <rlb>dsmith: right, seems like it's not accounting for the existence of broader gnulib fallbacks at all. <rlb>I'd really assume that this must be a more general thing in gnulib, i.e. doubt all other "subsystems" (collections of related functions) are "mix and match", so seems like there almost has to be some normal way to switch things on/off as a collection... <rlb>Maybe I'll poke around later. <rlb>e.g. probably can't use the platform regcomp with a gnulib regexec, hypothetically speaking. <dsmith>I would hazard a guess that the gnulib implementation *is* the glibc implementation? And that's why there is no issue on a glibc system? <rlb>ACTION doesn't know if gnulib has regexec :) <euouae>I know because I've written python wrappers that use gnulib <dsmith>fbsd has their own C library of course, as well as a different compiler (cc is clang not gcc) <rlb>dsmith: and presumably they do have that optional function? <dsmith>> The posix_spawn_*_np() functions are non-standard functions implemented after the similar functionality provided by glibc. <euouae>right after gl_INIT there's some code that says "we provide our own lock.h, let other modules know we have it <euouae>couldn't in theory the same be done about the posix threads module if the OS is FreeBSD? <dsmith>I would think it would be best to use the system provided functions, if they are available. <euouae>that's another option, to avoid using native functions, but that's not what I said <euouae>I can't even find posix_spawn_file_actions_addclosefrom_np in gnulib's source code, I'm a bit confused as to how it's being provided <dsmith>But I don't know. Maybe there are $REASONS <dsmith>It's called in libguile/posix.c about line 1415 <dsmith>euouae, You might have just nailed it. <euouae>yeah but they say it is in their docs? <dsmith>euouae, Note that glib is "vendored". You don't link agaisnt it, you copy the source directly into your code <euouae>I suppose Gnulib module: — means it's not provided <dsmith>Hmm. Missing in FreeBSD 13.0, provided in 14.3 <euouae>So what you really want is a BIG change <euouae>You want to change the thread handling in FreeBSD to be native, that'll solve all the problems <dsmith>I assume you mean s/thread/spawn/ <euouae>another option is to continue using gnulib's stuff and put some conditionals (build system or CPP) to avoid that dreadful function in that case <euouae>oh yeah, spawn.h sorry. I've never used it, it's related to process spawning or not? <dsmith>These functions are for dup/open/close fd'd when doing a fork+exec <euouae>hm... that is not as bad as threads <dsmith>Not required. Can do all that by hand. Just some convenience functions in a lib to make life a little easier/more consistent. <dsmith>The reason I'd prefer to use the system library versions it there might be some implementation specific knowledge they have that a general lib version doesn't <Kolev>I think I'm running into the Nix lang problem in another domain: web dev. SSG Zola uses a weird templating engine. <Kolev>Maybe I should have stuck to Haunt, not Zola. <euouae>is it Jinja? that's what it looks like <Kolev>euouae: Technically it's Tera. But yes. <dsmith>So.. The rpl_posix_spawn_* stuff is from lib/spawn.in.h A bunch of #defines that rename the posix names to rpl_* <euouae>probably REPLACE_POSIX_SPAWN should not be used in FreeBSD <dsmith>But where are the actual rpl_ funcitons? Oh. I get it now. <euouae>lib/c++defs.h: _GL_FUNCDECL_RPL_1 (rpl_##func, rettype, parameters, __VA_ARGS__) <dsmith>The local gnulib names *look* like posix, but they are actually named rpl_* <euouae>gnulib uses CPP features to concatenate symbols and make them on the fly <euouae>so you probably won't find some rpl_ definitions directly <dsmith>So there is no rpl_posix_spawn_file_actions_addclosefrom_np <dsmith>So we just need to add support for it in lib/spawn.in.h ? <dsmith>Or is it as simple as importing a newer version of gnulib? <dsmith>I don't see anything related addclosefrom_np in gnulib git <rlb>there's no manpage either? <rlb>(and it's not in "info libc", or am I overlooking it?) <dsmith>It's not listed in Trixie man 3 posix_spawn <rlb>ACTION wonders what's going on there. <dsmith>It is in /usr/include/spawn.h, behind #ifdef __USE_MISC <euouae>dsmith: it's not in gnulib in any version <euouae>I have gnulib in a repo I checked HEAD <euouae>You could contribute it to gnulib if it's simple to implement <rlb>So doea it even have a "formal" specification for what it does, error codes, etc.? <euouae>on my debian its on /usr/include/spawn.h <dsmith>Yes, the fbsd manpage mentions it's a gnu libc extension <euouae>This is the implementation, it's tiny. You can probably just vendor it in guile <euouae>it does nothing if __SPAWN_SUPPORT_CLOSEFROM is not definde <dsmith>rlb, BTW: That pathconf stuff about holes show min hole size in my setup as 32768 <dsmith>So theoretically, making the hole at least that large would allow the test to pass... <rlb>except on maybe lustre :) <rlb>If we decide to mess with it, I might be inclined to just consider doing whatever bup ended up doing if it's not too crazy, i.e. can probably afford 1mb or whatever for the test... <rlb>bup in that code just bases it on a multiple of the st_blksize of the test fs. <dsmith>Aaaannnd.. The tests pass with all the constants adjusted for 4096 -> 32768 <rlb>dsmith: plausible for me to convert that bup code to a (define (pick-sparse-block-size path) ...)? <rlb>I'll be trivial I think since we already have stat:blksize. <rlb>i.e. that's where we've settled after various reports/platforms, etc. <rlb>maybe something like <rlb>(define (get-likely-sparse-block-size path) <rlb> (* 20 (st:blksize (stat path)))) <rlb>And then as you can see, that's used to probe. <rlb>or maybe get-likely-sparse-(data|test)-size -- dunno <rlb>since it's clearly *not* the block size. <rlb>(depending on what we mean by block) <dsmith>Is that the backup thing you were working on some years (decades?) ago? <dsmith>So what does 20 * st:blksize come out to? <rlb>Unexpectedly inherited it at one point. <rlb>depends on the platform/fs <rlb>i.e. much larger number for lustre, smaller for ext4 <rlb>And this is still just s heuristic, but it's more dynamic, and does work for (as we found out) lustre where the block needs to be mb, not kb. <rlb>s/block/sparse chunk/ <rlb>On my ext4 fs, it's unsurprisingly (* 20 4096). <dsmith>I guess the sparse bit must be a multiple of some size. <rlb>And 20x is almost certainly larger than it needs to be most everywhere, but as long as it's "big enough". <rlb>right, I don't recall for sure (been a while), but that I think that 20x size is then just used as the min for all hole/data regions when trying to test behavior. <dsmith>16 * 4k is 64k. 20 seems an odd number <rlb>I forget, but it may have been chosen by fair die roll :P <rlb>I can delve if we like, i.e. may have been details in the various related commits. <rlb>But not sure it matters as long as it "works" (i.e. is big enough). <rlb>And it may still not work everywhere -- there's no reason an fs couldn't do something very unusual, since more or less anything is allowed. <rlb>same wrt "du" numbers wrt internal reflinks, deduplication, cow, compression, with fancier filesystems, e.g. zfs, btrfs, bcachefs, ... <dsmith>I guess that pathconf() call is only on fbsd (maybe other *bsd?) <dsmith>Well there is a Linux man page for it.. <rlb>Oh, I forgot you mentioned that -- if that's supposed to be the number we want, then sure, could just try that, or a small multiple of it. <rlb>As mentioned, I don't think I knew about that var. <rlb>(If that's what you were doing, then maybe ignore me...) <rlb>Wait, is pathconf per-path? <rlb>ok, yeah, if that's what those values are actually *for*, then maybe that's what you'd really want? <dsmith>Ahh, but linux doesn't have _PC_MIN_HOLE_SIZE <rlb>I'd assumed the hole sizes could vary with more exotic filesystems, i.e. "no promises" for any given region of a file. <dsmith>Yes, per path. Or to an open fd. <dsmith>Because /boot is fat and /data is zfs and / is ext4 <dsmith>Might even depend on media block/cluster size <rlb>Anyway, the st_blksize multiple "works", so I'd be fine with that or if we think the fpathconf values are applicable, that, if we want to be fancier -- either one. <rlb>Or, as you have for now, just skip it. <rlb>any of those options are better than broken :) <dsmith>Skipping is prob best. For now. Cause it's probably very system dependant what a resonable default size should be... If any. <dsmith>(some fs might not support holes...) <rlb>i.e. pick a value, then see if you can use it to create a sparse file, if not, skip <rlb>Or did you mean that that test might still be too coarse? <dsmith>Hm. But the point of the test is to see if the SEEK_HOLE and SEEK_DATA work properly by hitting the edges of the hole. <dsmith>Maybe I'm not undestanding what you are suggesting? <rlb>Oh, I wasn't even thinking about a specific test, just trying to decide whether or not the fs supports sparseness in general. With respect to portable *tests*, there's a lot you just can't test for, generally, I think. <rlb>i.e. probably not likely to be able to test (portably) with much precision. <rlb>I don't recall for sure, but I think some of the bup tests are mostly just checking that sparseness works somewhat "coarsely". <rlb>I'll plan to take a look at that actual test more closely later, if that seems useful, otherwise, fine to skip or even just try the larger size you tested, or... <rlb>Worst case, it'll work in more places, and we'll still have some that could work but don't yet?