IRC channel logs
2024-02-13.log
back to list of logs
<apteryx>I think it's a bug! I assumed having the 'guix' command installed meant its modules were available <apteryx>they are on a Guix System but that's more by accident <apteryx>cow_2001: 'hall dist -x' works but 'hall dist' doesn't. hm <cow_2001>apteryx: i ran hall dist -xf and it erased it contents <apteryx>you're right, -f triggers it (otherwise it was skipped) <apteryx>I'll try changing the autoloads into use-modules just to check <apteryx>package-name is supposed to be usable as a procedure, but the modules must be finished loading <apteryx>unfortunately that's not a valid fix as guile-hall users may not use guix <apteryx>autoload allowed this use case (load the module if they are available) <apteryx>I guess I can use conditional imports instead <apteryx>then to retrigger the problem after reverting the change, one has to clear the .go files of hall <apteryx>maybe that's why I hadn't noticed until now <apteryx>the source files are at ~/.config/guix/current/share/guile/site/3.0 <cow_2001>oh, yes. it's a symbolic link, so find didn't find it, only find -L did <apteryx>the byte compiled ones at ~/.config/guix/current/lib/guile/site-ccache/3.0 <apteryx>just the source should be enough; performance is not really critical here <apteryx>but if you want it'll speed things to have the GUILE_LOAD_COMPILED_PATH set as well <apteryx>*maybe that's going to be part of the solution; if the guix *command* is found, add the above to the Guile load paths <apteryx>does anyone know if using #:autoload is problematic to load syntax? <cow_2001>hmm. now it complains i lack the (zlib) and (git) libraries, which i see are provided in guile-zlib and guile-git, and i guess those are coming from the guix library? i've grep -R autoload inside the hall libraries and didn't see any of those <cow_2001>installing those dependencies and running hall dist again <apteryx>seems propagated by guix, inside: 'guix shell --pure guix guile' <apteryx>but yeah, the GUILE_LOAD_PATH approach wouldn't account for those transitive dependencies <apteryx>yep, the guix package propagates a bunch of guile libraries <apteryx>is #:autoload problematic to load syntax (macros)? <apteryx>I'm getting a syntax-transformer object instead of a what should be a procedure <jpoiret>apteryx: iirc autoloads will not work with macros <jpoiret>since autoloads are processed at run-time, after macro expansion <apteryx>I see. What alternatives do we have to load some module that may or may not be available? <apteryx>I know of R7RS cond-expand that can do that <apteryx>but R7RS is still a bit broken on master without patches I've sent... so can't use it yet. <apteryx>ACTION tries it to see if this particular usage can work without the patches <apteryx>jpoiret: grr, won't work, because of bug 65363 "Missing support for (library ...) match condition in 'cond-expand' " <apteryx>actually it might, one comment of mine there was that the else clause was not supported <apteryx>has someone seen that kind of error? unknown location: _: bad use of '_' syntactic keyword in subform _ of _ <lilyp>apteryx: this probably means you borked a syntax-case or similar <lilyp>basically, guile thinks you wrote a _ as though it was code <apteryx>it was because I was using a define-library'd module (R7RS) and λ is not part of it <apteryx>can be imported from the (guile) module, but that's not portable <apteryx>jpoiret: actually I want the macro to be expanded by the time I use it; it's for package-name, which expands to a procedure <apteryx>what I'm seeing suggests it has *not* been expanded by the time it's used by code when autoloaded, as it fails saying 'package-name' is a syntax-transformer, not a procedure <apteryx>how can I check the path of the currently used guile interpreter? <old>apteryx: on Linux check /proc/self/exe <old>scheme@(guile-user)> (canonicalize-path "/proc/self/exe") <old>$1 = "/gnu/store/1gd9nsy4cps8fnrd1avkc9l01l7ywiai-guile-3.0.9/bin/guile" <jpoiret>Guile has to know at expansion type that your thing is a macro, but because it's autoloaded it can't know that yet <rlb>civodul: fwiw, just saw the tests hang with -j5 even without the parallel testing changes when the system was under a broader load (bootstrapping guile for the 3.0.9 debs in the background). And they printed the primitive-fork "multiple threads are still running" warning before hanging, so suspect it's the issue I tracked down. <apteryx>jpoiret: I see. I'll try adding some caution note to the manual, where #:autoload is documented <rlb>Yeah, it's somewhat repeatable even without the parallel test harness -- just swamp all the cores with whatever and then run "make -jN ...". <rlb>"make -jN check" I mean, for some suitable N presumably >= your actual core count. <civodul>rlb: hi! i should try and catch up on this, hopefully next week <civodul>apologies for not being more responsive <rlb>No worries -- I think where we are might be that the proposed branch does appear to fix the problem, but we'll have to see if we think that solution's good enough for now, and/or whether there are any concerns about "collateral damage". <rlb>I just happened to have a lot more time for guile this week. <civodul>it looks like you identified a real issue in ‘join-thread’ <old>there's race condition in join-thread? <civodul>old: it can return before before the underlying pthread has finished <old>last time I checked, join-thread wait on a condition variable associated with the detached thread. Is that why? <civodul>i’ll let rlb comment as i’m fuzzy on the details <old>rlb: You have a link on the patch? <rlb>old: yeah, it's (I think) the fact that join only waits for the thread to exit, but we still have nontrivial work pending in on_thread_exit, which runs as a pthread "destructor". <rlb>old: and that code grabs locks, pritmitive-exit only waits for join to finish before forking, and so "trouble". <rlb>But I think it's maybe a more general problem? I guess it depends on what we mean for thread-join to mean :) <rlb>And when under load, practically, what's happening, I believe, is that things slow down enough that fork ends up happening in the middle of on_thread_exit, while it's still holding lock(s). <rlb>And then you're "done". <old>POSIX itself is not very verbose on what "terminate" mean when pthread_join a POSIX thread <rlb>iirc the linux pages have more detail, but of course might not be officially posix... <old>typically I think that pthread_join will wait for pthread destructor (pthread key) ran <old>but the implementation in guile join the thread with something else. so ofc there can be race condition with destructors <rlb>old: I'm also a little unsure what's really OK in on_thread_exit, i.e. we're tearing things down, and I think have already left with_guile? So what's actually safe to call scheme function-wise? <rlb>Right, we don't use pthread_join for thread-join in main. <rlb>So I didn't even check it. <rlb>But I suppose that could be another solution -- would have to think about it. <rlb>(and check the promised semantics -- unless you already did) <old>the thing is that join-thread from multiple threads in Scheme is fine. but POSIX says that pthread_join from multiple threads is undefined behavior <old>that's maybe why the implementation is made around condition variable with broadcast <rlb>i.e. if we're out of with_guile, is it fine to call various scm_* functions? <rlb>ACTION doesn't know what the restrictions are, if any. <old>rlb: pretty sure it ain't fine <rlb>Well, we're doing it already, at least a bit? <old>I do not recall what with_guile does really, but I think it setup things on the stack somehow for the GC? <rlb>Which is why I favored pthread functions (not scheme) in the patch -- i.e. I wasn't sure what was allowed, otherwise. <old>or register the thread for the GC <apteryx>are define-library modules compatible with define-module ones? <rlb>To answer your earlier question. <old>rlb: thanks, I'll have a look ! <apteryx>I'm getting 'no code for module (hall common)' after rewriting it to use define-library and renaming its file to hall/common.sld <rlb>old: top two commits, and as a bonus, you can also try the parallel test harness if you like :) <rlb>apteryx: I imagine you know, but one thing that causes that is if the file the module loader picks, when loaded, doesn't actually define the module somehow. <rlb>old: oh, and thanks for poking at it. <old>rlb: pleasure. I like looking at parallel stuff :-p <old>or anything related to race condition <rlb>...my condolences? :) <rlb>(Though unfortunately, I find it interesting too :) ) <old>well it depends, but it is certainly challenging! <apteryx>rlb: ah, indeed, the .go is missing... I probably goofed the build system change part <rlb>(...like juggling knives...) <lilyp>apteryx: what happens if you use .scm instead of .sld? <lilyp>maybe guile just doesn't recognize the ending? <rlb>...not sure it's relevant, but I recall being surprised by the way guile currently handles/chooses the file to load when there are multiple choices (e.g. .go vs .scm) -- suppose that might apply to other extensions too. <rlb>If it was the behavior I saw, you can observe it via strace or similar to see what it actually chooses. <dsmith>strace is often very useful to debug weird "file not found" errors <rlb>I've wanted to propose a patch, but haven't had time to revisit it yet -- iirc it was that guile could load a "mismatched" pair wrt .scm and .go (in my opinion at the time). <rlb>Oh, right, the .scm was newer, and compilation failed, and it just went ahead and loaded the stale/wrong .go file. <rlb>I want guile to just exit with an error if the compilation fails. <rlb>...grabbed a random hell & damnation imperial stout at the groc store - not bad. <rlb>Hah, oops - wrong channel :) <ieure>"Pen and paper role playing game," I think. <rlb>Oh, hah - "real-ish life"? <apteryx>is it possible that Guile doesn't do dependency resolution when evaluating define-library? I think I have to specify the dependencies manually in the build system between the .go files. <apteryx>If I run 'make' multiple times in a row the problem resolves <apteryx>in guile I had to do e.g. in bootstrap.am when adding .sld SRFIs: srfi/srfi-178.go: srfi/srfi-151.go srfi/srfi-160/u8.go <apteryx>works the same with .scm as .sld, without that problem <apteryx>maybe it needs to be registered somewhere to be picked up implicitly as a source <apteryx>I had to use syntax-case because of the computed error message <lilyp>you could use let-syntax to save some horizontal real estate <old>apteryx: you could use recursion instead of #,@map and define the transformer instead of lambda <old>I find it more clear <apteryx>thanks! I agree that's more readable <dthompson>that datum->syntax is weird to me, rather than just using #` <jpoiret>old: you can just leave the (_ ids ...) case and just map over ids, no? <old>jpoiret: I think so yes <old>dthompson: datum->syntax keep the source location IIRC <old>dthompson: could you rewrite it in term of #' so I can understand better? <old>jpoiret: I'm just not sure about applying identifier? to all ids <dthompson>wouldn't expect all the atoms within that quasiquote to have accurate source locations though <old>dthompson: from my understanding, using datum->syntax using the `id' syntax object keep the syntax location to the expansion of it. I might be wrong and I do not know how do that with syntax-quote <dthompson>so I don't really know about the source info thing but just from a style perspective it would seem clearer to use #` since id is a syntax object <jpoiret>here's my version because I like flexing my macro foo <old>dthompson: It is cleaner yes, but I do not think that the resulting syntax-object will share source-location with #'id. That's my main concern <old>but perhaps syntax quoting is intelligent enough to do that already <dthompson>I'd have to experiment to know what the syntax objects end up looking like but my gut feeling is that #` wouldn't be very useful if it didn't have some reasonable handling of source locations <dthompson>especially since you could alternatively do with-syntax and regular #' <jpoiret>by the way apteryx, didn't you mean stub instead of stud 😳 <dthompson>jpoiret: your version looks like what I had in mind <old>jpoiret: lmao. I think I'm doing little too much renovation on my house <old>dthompson: after looking, it seems there's no difference at all <old>so I concur, using the syntax quote is cleaner <jpoiret>old: ah, I thought it was in the original paste :) <jpoiret>TIL begin with one clause expands to that clause <dthompson>I haven't been looking at source locations but I have been writing the gnarliest macro I've ever written a wide margin these past couple of weeks <jpoiret>when the whole project is just defining one macro <dthompson>so I've been getting pretty familiar with syntax-case and related things <jpoiret>basically what guix records are (please don't look at it) <dthompson>10 years ago, when I first saw define-record-type*, I had no idea where to begin with it. I think I stand a chance now. <jpoiret>we've got a macro-generating macro using all the damn tricks <jpoiret>people think G-exps are complicated, but they clearly haven't looked at that one definition :o <dthompson>yeah I'm in full procedural macros generating procedural macros territory <jpoiret>can't wait for "Dissecting Guix records" <old>I'm getting pretty confident with macro, but I'm still afraid of psyntax.scm <old>jpoiret: I once checked define-record of Guix. It's really nice. I do not remember how it checks for ABI compatibility, but that's nice to have <old>when you have 3 levels of with-ellipsis :-O <jpoiret>yeah there was a gnarly bug with ellipses in the body of match-record at some point, and my description of it in the ML is not informative :( <jpoiret>still, it's hard. Best of luck to all macro-wielders out there