IRC channel logs
2025-11-03.log
back to list of logs
<ArneBab>wingo: there’s a set of three old commits by you and we’re not sure what your intention was. Jonas may have discovered a bug in that which slows down compilation without any gain, but I am not sure whether there are subtle side-effects. <ArneBab>wingo: the question: in the bootstrap build: should stage0 prefer prebuilt binaries or the stage0 binaries that are just being built? <apteryx>would someone know a good way to define Linux constants like MAP_SHARED, etc. that differ depending on the target system? <rlb>dsmith: I'm finally back to wanting to file a bug wrt that timestamp issue -- can you remind me where you found the problem? (paste expired) <dsmith>rlb, Yes. .scm and .go have identical timestamps. Use the .go as is, or recompile from the .scm? <dsmith>rlb, In libguile/load.c around line 56x <dsmith>- && source_mtime.tv_nsec <= compiled_mtime.tv_nsec)) <dsmith>+ && source_mtime.tv_nsec < compiled_mtime.tv_nsec)) <dsmith>They can have identical timestamps becuse of filesystem timestamp granularity <dsmith>Also, because simple test code is faster to compile, there is a better chance of triggering this when testing than "real" code. <rlb>And I don't think we have a completely transparent fix? i.e. you could loop until the times vary when writing, but of course that's ugly and expensive with low granularity systems; you could try to bump the stamp into the future when they come back equal, but I suspect that might be asking for trouble; or you could maybe add a source hash to the .go file, or... <dsmith>Right. Many possible workarounds. <rlb>wingo: if you happen to get a minute, any thoughts? ^ (summary: we found the (a?) cause of the issue with guile running stale code that I'd observed in the past) <dsmith>I lean towards recompiling, but that could be a problem on a RO filesystem, or permissions <rlb>Also wonder if it could cause a lot of redundant compiles on a fast system with too low an fs granularity? <dsmith>Well, it depend on how fast the compile time compared to fs granularity. Again, I suspect this is not much of an issue in "real" code, only in artificially small test cases. <rlb>And yeah, it has roughly the same problem we did with clj and dh_strip_nondeterminism, i.e. if you end up with the same timestamp in say distribution packages, you'll cause all users to always rebuild their own copy (with clj it caused every run to recompile since there's no per-user cache). <rlb>i.e. if I uploaded a guile-3.0-libs with .scm/.go having the same timestamp. <rlb>(For clj, dh_strip_nondeterminism just changed to always put the compiled timestamp just after the source.) <rlb>But there it's not the fs, so you have better knowledge of the granularity (i.e. the files are inside a jar) <dsmith>Yes, but how much? 100ms? 1s? IIRC fatfs or something DOSish has 2s grandularity.. <rlb>You mean for scm (for jars I suspect we know the granularity)? <dsmith>So the failing test case: Compile the .go with a bug. Fix the .scm, but the .scm timestamp is the same as the go. Rerun the test and uses the "old" buggy .go and not the "new" fixed .scm. <dsmith>Where "old" and "new" are actually the same timestamp <rlb>It's at least theoretically fairly "bad". <rlb>depending on the bug :) <dsmith>So in that test, it's the creating the fixed .scm that takes a short time. <rlb>I thought it was just that if the .go compilation was "faster than the granularity", you may be in trouble? <rlb>Oh, you were just talking about my test script, maybe -- need to go find it and remember what I did :) <dsmith>Right. The test script triggers it. Doe it also get triggered in the wild? <rlb>I assume so -- since I've been concerned/complaining about it for a while, but until now I wasn't quite sure exactly what I was seeing. <rlb>If nothing else I think I saw it when working on lokke, where I was rebuilding lots of small files all the time. <rlb>I've also wondered if we might have issues around load vs compile path and/or the same file "name" in multiple path entries and "fallback" if we don't halt on the first error (not sure now). With luck, I'll just be the timestamp issue that I was hitting. <dsmith>My gut reaction is < is correct, not <=. The compiled file *has* to be newer. It's just that we can't tell because of fs inadequacies. <dsmith>Ah. So with the <= it's ambiguous. With < it's not <dsmith>And round and round the agruments go... <rlb>Right -- the (additional) fingerprint option is the only other one we've discussed that seems potentially both efficient and correct, but that'd still require < as a fallback for "pre fingerprint". <rlb>i.e. so far seems like the best option might be < for now, and maybe forever -- introducing a stable release risk of unexpected recompiles, but only for things that were ambiguous (maybe wrong). <dsmith>For more confusion, POSIX specifies it a certain way for make <ArneBab>That sounds like we should follow make, because the alternative has worse edge cases. <dsmith>Is it possible to know or discover fs granularity somehow? <rlb>Not portably afaik, at least not via anything other than heuristic probing. (I think we probably discussed this in some detail a good while back on the bup list.) <dsmith>When a timestamp is "stamped", and it's "in-between", is it written as the earlier time or the later time? In the past or in the future? <rlb>I suspect the answer to that might be "unspecified". <dsmith>Probably the older time. Whatever the current tick is. <dsmith>Hmm. That might be different depending on how the kernel is configured. <rlb>(compared to the previous where I also used modules) <dsmith>Yeah, that always eventually fails/exits <rlb>ACTION is filing a bug. <dsmith>Hmm. That is also eventually exiting even with s/<=/</ <rlb>Are you sure you're picking up the right guile? <rlb>Here I made the mistake that my "fixed" guile was guile-3.0, not guile :) <rlb>hence the GUILE= in there. <dsmith>guile (GNU Guile) 3.0.10.253-a1abc-dirty <rlb>Hang on, I had one other difference... <dsmith>Stuff flies by so fast I din't read much. <dsmith>foo.scm:2:1: unexpected end of input while reading string <rlb>Hmm, now it's wrong here too -- I did simplify the test, removing the cache in the tmpdir, wonder if that matters. <rlb>Yeah, that's fine -- the fix is intentionally broken syntax. <dsmith>I really would like to see all that autocompile noise go away <rlb>iirc that's probably on the agenda for the next X or Y release. <rlb>...I believe I had the test working with the adjusted code, but now it doesn't here either. Was I just mistaken before? <rlb>If you want to see the identical timestamps, you can add this in the "ran bad code" block: stat foo.scm "$(find -name foo.scm.go)" <rlb>oh, if you add back the <rlb>export XDG_CACHE_HOME="$(pwd)/cache" <rlb>just after the cd to tmpdir <rlb>Assuming I'm not getting something else wrong, wonder if there are other timestamp-related paths. <rlb>i.e. not just compiled_is_fresh() <rlb>It looks like maybe it's not calling compiled_is_fresh() for foo in that last guile invocation (added some writes to it, and don't see them, but do see them for threads, match, vm, etc. <rlb>I originally was testing a module, not a "top-level" script -- figured that might be more straightforward. Perhaps I'll try that again. i.e. wasn't exactly sure how scripts vary compilation-wise. <dsmith>I was somehow printing timestamps to the ns before. I forgot how I was doing that.. <rlb>think I'll just hold off filing anything for now <dsmith>I added some prints in there too, and not seeing them. <rlb>(One thing I'm still (perhaps incorrectly) concerned about is possible fall back behaviors, i.e. if the compilation fails, do we always halt, or might we fall back to the existing .go.) <rlb>(and here of course, the compilation better fail, given the broken syntax) <dsmith>Ok, something is fishy. I modified the ";;; note: source file" message and am still getting the old message <dsmith>rlb, See also fresh-compiled-thunk in boot-9.scm <dsmith>Actually more-recent? in boot-9.scm <rlb>looks like that might fix the test <rlb>I wonder if when I thought I saw it working earlier I was testing the module rather than the top level script, and that one takes the other path... <dsmith>Cause I did find identical stamps, and narrowed it down to that <= <dsmith>rlb, Nice catch! If you hadn't changed the test, would not have seen that in boot-9 <rlb>Got lucky -- was just trying to simplify. <dsmith>;; Return #t when STAT1 has an mtime greater than that of STAT2. <dsmith>But the code is actually greater or equal <dsmith>wingo, What was your intent here? The comment, or the code? <dsmith>"The make utility examines time relationships and shall update those derived files (called targets) that have modified times *earlier* than the modified times of the files (called prerequisites) from which they are derived."