IRC channel logs

2020-06-20.log

back to list of logs

<RhodiumToad>(let* ((port (make-custom-binary-input-port "the port" (const 0) (const (expt 2 42)) #f #f))) (port-position port))
<RhodiumToad>$2 = 4398046511104
<RhodiumToad>well that works better
<RhodiumToad>ERROR: asyncs.test: preemption via sigprof - arguments: ((wrong-type-arg #f "Wrong type (expecting ~A): ~S" ("resumable continuation
<RhodiumToad>" #<vm-continuation 228ea408>) (#<vm-continuation 228ea408>)))
<RhodiumToad>now what
<RhodiumToad>at least r6rs-ports.test passes now
<dsmith-work>Yeah, all the error/condition/exception reporting needs to be gone over carefully and tuned up.
<RhodiumToad>ERROR: srfi-69.test: SRFI-69: can use all arguments, including size - arguments: ((keyword-argument-error #f "Invalid keyword" () (3
<RhodiumToad>1)))
<dsmith-work>Many times, printing an error causes a segfault or something.
<RhodiumToad>the asyncs one passed on a retry, so I think that one might be just chance
<RhodiumToad>i.e. happens to get the sigprof at a bad place
<RhodiumToad>srfi-69 failure is consistent though.
<RhodiumToad>just to check, that passes for you?
<RhodiumToad>hm. this test doesn't fail if run standalone
<RhodiumToad>so the problem is that you can't run r6rs-hashtables.test and then srfi-69.test in the same interpreter
<RhodiumToad>which seems ... odd
<dsmith-work>All tests pass.
<dsmith-work>I put it in a loop and it ran for a while.
<RhodiumToad>hm.
<RhodiumToad>that worries me
<dsmith-work>But sometimes got a leftover file in /tmp that cause a failuer.
<RhodiumToad>because that pair of tests reproducibly fails for me
<dsmith-work>And a pipe test error too.
<dsmith-work>But that might be because of without-threads
<RhodiumToad>I'm currently using a with-threads build
<RhodiumToad>if I do just ./check-guile r6rs-hashtables.test srfi-69.test it fails
<dsmith-work>So one that that was a concern at one time, was is the rest of the system (libc) built as arm or thumb?
<RhodiumToad>arm
<dsmith-work>Mine is still rebuilding for with-threads
<RhodiumToad>aha.
<dsmith-work>So I can't check at the moment. But soon. On intset.go
<RhodiumToad>so the problem actually isn't with r6rs-hashtables.
<RhodiumToad>the problem is jit, somehow
<dsmith-work>So I heard at one time Debian was default thumb. But I think this rasbian is arm.
<RhodiumToad>it's just that without an explicit GUILE_JIT_THRESHOLD, and without r6rs-hashtables.test, it's not reaching the threshold on some path.
<RhodiumToad>GUILE_JIT_THRESHOLD=200 makes srfi-69.test fail on its own.
<dsmith-work>Do you know about GUILE_JIT_STOP_AFTER ?
<dsmith-work>Set GUILE_JIT_THRESHOLD=0 and then bisect on GUILE_JIT_STOP_AFTER
<dsmith-work>At some point n passses and n+1 fails.
<dsmith-work>Supposed to help identify the failing jit code.
<dsmith-work>Aww Fouey
<RhodiumToad>GUILE_JIT_THRESHOLD=0 GUILE_JIT_STOP_AFTER=681 passes, 682 fails
<dsmith-work>Forgot I wanted to pull to get the latest changes.
<RhodiumToad>any way to get it to show what function it decided not to jit?
<dsmith-work>And you can set GUILE_JIT_LOG = 1,2,3
<RhodiumToad>the log isn't helping
<RhodiumToad>GUILE_JIT_PAUSE_WHEN_STOPPING=1 helps
<RhodiumToad>can't easily see what function it's trying to compile tho
<dsmith>Need to look at hex addresses I think.
<dsmith>WHen I as using logging heavily, I also had code that was printing stuff. So I couild correlate.
<dsmith>The blx and veneer problems.
<dsmith>debugging on-the-fly generated code is a whole new level of "hard to do".
<dsmith-work>make check failed
<dsmith-work>ERROR: 00-repl-server.test: repl-server: simple expression - arguments: ((misc-error #f "~A" ("Unable to create directory in /tmp for 00-repl-server.test") #f))
<dsmith-work>
<dsmith-work>Cause /tmp was loaded up I guess
<RhodiumToad>probably
<dsmith-work>sneek: bugs?
<sneek>Last time I checked bugs is Send reports to bug-guile@gnu.org, and see bug reports at https://bugs.gnu.org/guile
<dsmith-work>Added this https://gitlab.com/wingo/lightening/-/issues/14
<dsmith-work>yey make check passes
<dsmith-work>With 0 threshold
<dsmith-work>yep
<dsmith-work>passed again
<dsmith-work>Ok, running the checks in a loop. will run overnight
<dsmith-work>with ramdom addresses enabled again
<daviid>I am going to add a g-golf 'mechanism', using a variable, so users may decide to not having any generic short names at all, or skip some, like map for example ... I wonder what a good name would be
<dsmith>shrtnm (not serious)
<daviid>%gi-no-generic-short-name-for, in which case one might passs 'all or a liat
<daviid>'all, or 'none', or '(map ...)
<dsmith>Remember to use your vowels sparingly
<daviid>or %gi-skip-generic-short-name-for
<daviid>or %gi-no-generic-function-short-name-for // %gi-skip-generic-function-short-name-for or %gi-no-gf-short-name-for %gi-skip-gf-short-name-for
<daviid>or some other name? :)
<daviid>gi stands for GObject Introspection, in this context
<daviid>or %gi-no-generic-function-short-names
<daviid>bah :)
<daviid>%gi-no-generic-function-short-name-for seems the best to me
<daviid>then, sould it be for 'none, or for 'all :):)
<daviid>I'll accept both, everybody will be happy
<daviid>hum %gi-skip-generic-short-names might be better
<daviid>dsmith: wdyt? :)
<dsmith>Personally, I don't like long names.
<dsmith>On the other hand, less chance of a collsion.
<daviid>ok, I am more into the name meaning something, hence long names, unless universally obvious :)
<daviid>but there are many long name possibilities :)
<dsmith>Can they be re-named? Kind of like module imports?
<daviid>no
<dsmith>But you do want user to adjust a set of names.
<daviid>I just wanted to offer to skip them all, or some
<dsmith>Hmm.
<daviid>renaming is really 'out of scope' so to speak
<dsmith>%gi-short-names ('on, 'off, 'all, #t, #f, #:on '(list of names)
<dsmith>Something like that would be my personal preference.
<daviid>I see, you would not use generic-function at all
<dsmith>Right
<daviid>in the variable name
<daviid>good sugestion, I'll think about it, thanks
<dsmith>Would there be any other "short names" than generic-functions?
<daviid>no
<dsmith>Naming is *hard*
<daviid>yeah, very hard :)
<dsmith>I can't count the times I've named some kind of struct "thing". At least as a placeholder.
<dsmith>YEah, had thing1 and thing2 too.
<dsmith>And then later, after working with it a while, a better name surfaces.
<daviid>yes, difficult - always ...
<dsmith>Sometimes, a good name used in isolation might seem weird, but when used in context, it's nice and readable.
<dsmith>Think about how it will be used. What other identifiers and functions will be used with it.
<dsmith>Grand ideas.
<daviid>my only objective is to offer those who do not want short names at all to be able to specify so, then those concerned that some guile core are transformed into generic, with an inevitable 'penalty', should it be small ... there is one ...
<daviid>map is the only real concern actually, but then the mnechanism should be 'general
<daviid>anyway, tx, i'll think a bit more about it
<dsmith>I'm not much for apps (gui apps), but I'm looking forward to playing with it some day.
<dsmith>s/it/g-golf/
<daviid>i hope you'll have fun :)
<dsmith>A long long time ago, scwm had an interface to gtk+. That has long since bitrotted.
<dsmith>I don't have much motivation, as awsome (what I use) works perfectly for me.
<daviid>sure
<daviid>and a wm is not a 'piece of cke' either
<dsmith>I sent in some patches and all of a sudden, they made the maintainer.
<daviid>:)
***terpri__ is now known as terpri
<roelj>How do the ‘http-get’ and others validate certificates, or what environment variables influence it looking for the CA store?
<roelj>I set both SSL_CERT_DIR and SSL_CERT_FILE, but that doesn't seem to work.
<leoprikler>daviid: do not forget about GLib functions with names sounding like Scheme syntax like "begin"
<leoprikler>Guile-GI has a way of protecting those bindings on a "per name" basis, as it uses modules
<leoprikler>not sure how G-Golf wants to handle this
***wxie1 is now known as wxie
<tohoyn>sneek, botsnack
<sneek>tohoyn, you have 1 message!
<sneek>tohoyn, daviid says: I pushed an important patch, to the G-Golf repo, devel branch: "Methods - Both long and short name generic functions" - please pull, make ... and see if everything is ok
<sneek>:)
<RhodiumToad>hm. so I made a testcase specific to this error I'm getting
<RhodiumToad>basically (define* (testfn #:optional a b #:key (c #f) #:rest d) (format ...)) and then call that function 100 times in a loop
<RhodiumToad>with the default threshold, after 34 successful calls, the 35th call fails with "Invalid keyword: 34"
<RhodiumToad>so the first thing that suggests to me is that whatever code path isn't being properly jitted is being called a lot, just under 30 times per call
<RhodiumToad>hm.
<RhodiumToad>setting the threshold to 0 makes it fail on the very first call. but a threshold of 1 makes it fail on the second?
<dsmith>threshold 0 means always jit
<dsmith>1 means (I think) jit if called more than once
<dsmith>Ya, it's in the manual
<dsmith>"Each function has its own counter that gets incremented when the function is called and at each loop iteration in the function. When the counter exceeds the ‘GUILE_JIT_THRESHOLD’, the function will get JIT-compiled."
<RhodiumToad>with a threshold of 0, it succeeds (program runs to completion) if GUILE_JIT_STOP_AFTER is set to less than 370
<RhodiumToad>i.e. GUILE_JIT_THRESHOLD=0 GUILE_JIT_STOP_AFTER=369 works, GUILE_JIT_THRESHOLD=0 GUILE_JIT_STOP_AFTER=370 fails at iteration 0
<RhodiumToad>GUILE_JIT_THRESHOLD=1 GUILE_JIT_STOP_AFTER=241 stops at iteration 1 (second call)
<dsmith>Maybe there is a way for force a bactrace or at least print the current function at the stop_after point.
<dsmith>As in instrument the stop_after code to do that
<RhodiumToad>you can make it pause to attach gdb, but I've so far failed to find out how to get the current function
<dsmith>wingo knows
<dsmith>No doubt he added that for this very purpose
<RhodiumToad>simplified it down to just (define* (testfn #:key a #:rest b) #t) (testfn #:a 1 1) (testfn #:a 1 2) (testfn #:a 1 3)
<RhodiumToad>removing the extra args didn't affect the stop-after threshold at all, curiously
<RhodiumToad>neither did removing the do-loop I was using before for the iteration
<dsmith>My loop over make check with-threads and address randomization is still running from yesterday
<dsmith>(I expected it to)
<civodul>RhodiumToad: i pushed your THUMB fix to the lightening repo (and just replied by mail)
<civodul>dsmith: we can add the test you posted yesterday in lightening if you're confident that it was a good reproducer
<dsmith>Yes. fails without RhodiumToad's fix. passes with it. Solid.
<dsmith>But remove the fprintf. (oops)
<dsmith>Idealy, should feed it into the jit code as a parameter, but I couldn't figure out how to do that.
<dsmith>So you could also verify the other encodings.
<civodul>awesome
<civodul>i'm trying to run the checks via qemu-binfmt here
<dsmith>Yes that's how I ran them yesterday. Worked well.
<dsmith>Just needed to look into the yaml file for the invocation.
<civodul>awesome
<civodul>i guess we need enter/leave_jit_abi like in the other tests
<RhodiumToad>civodul: I don't have any better test than dsmith's, so use that one
<civodul>cool
<civodul>i guess movi.c is a good name
<civodul>i confirm that it reproduces the bug for me
<civodul>dsmith: OK to push the test with your name/email address as the author?
<civodul>and me as co-author because of the small changes i made
<dsmith>Sure. Probably better to use dalepsmith@gmail.com
<civodul>alrighty!
<dsmith>Yey. Sneek will be pleased.
<civodul>:-)
<civodul>JIT for sneek
<civodul> https://gitlab.com/wingo/lightening/-/commit/24ef197b1269f8371b1f4a412caa6d2b99d66839
<civodul>RhodiumToad, dsmith: does that solve all the issues you noticed so far on ARMv7?
<RhodiumToad>no
<civodul>ah
<RhodiumToad>I'm debugging another one
<civodul>cool
<RhodiumToad>but dsmith hasn't seen it, so it may be a clang vs. gcc issue
<civodul>alright
<RhodiumToad>or a freebsd vs. linux or whatever
<civodul>if would be great if you could post a description & reproducer on gitlab/w/l or bug-guile@gnu.org so we have an audit trail
<civodul>that's on FreeBSD?
<RhodiumToad>I have time to spend on it today/tomorrow, so if you're planning a release I'd appreciate a few hours leeway
<civodul>sure
<RhodiumToad>freebsd on an RPI2b
<civodul>ok
<civodul>i'm going afk for a bit
<civodul>later!
<RhodiumToad>symptom is that this fails: (define* (testfn #:key a #:rest b) #t) (testfn #:a 1 1) (testfn #:a 1 2) (testfn #:a 1 3)
<RhodiumToad>but only with a sufficiently low jit threshold
<RhodiumToad>I'm trying to narrow down exactly what function being jitted is breaking it
<dsmith>civodul: Yes. All issues.
<dsmith>civodul: As I mentioned earlier. Have had a make check running in a loop over night. Still running.
<civodul>dsmith: yeah, nice!
***sneek_ is now known as sneek
***jonsger1 is now known as jonsger
<RhodiumToad>I should have checked out a whole bunch of different worktrees
<RhodiumToad>recompiling with different compile options, such as -O0, is time-consuming :-(
<civodul>RhodiumToad: you wrote that the snippet above "fails", as in it doesn't always return #t? or crashes?
<RhodiumToad>it throws an error about 1 not being a keyword, but only after doing some jit
<RhodiumToad>if the jit threshold is never hit, it runs fine
<RhodiumToad>I can narrow down which function being jitted causes it, but I can't tell which function it actually is
<RhodiumToad>i.e. it's the 370'th call to the jit code generator
<civodul>uh
<civodul>weird
<civodul>and only on armv7-freebsd?
<RhodiumToad>I've not actually tested it specifically on other platforms
<civodul>ok
<civodul>i guess we'd notice because it's a rather simple snippet
<RhodiumToad>it showed up for me when running the tests
<RhodiumToad>the srfi-69 test, specifically
<civodul>ok
<RhodiumToad>bleh. define* leads into a maze of macros
<RhodiumToad>I guess that's actually not important, because at the VM level there's just a bind-kwargs opcode
<civodul>yeah
<RhodiumToad>and it's definitely bind_kwargs in vm.c that ends up actually throwing the error
<RhodiumToad>hm
<RhodiumToad>if there's a #:rest, then bind_kwargs is supposed to be called with the "strict" parameter false
<RhodiumToad>which would prevent the error
<RhodiumToad>so somehow that parameter is not correct. progress!
<RhodiumToad>argh argh argh
<RhodiumToad>so I compiled it all with -O0 and of course the damn problem went away :-(
<RhodiumToad>so either clang is miscompiling something when optimization is on, or the nasal demons are at it again
<civodul>bah :-/
<civodul>it could be a stack marking issue
<civodul>if you compile the C code with -O0, you make it more likely that objects remain live
<civodul>as in <https://bugs.gnu.org/28211>
<RhodiumToad>time to make a new workdir and compile up another copy
<civodul>heh
<civodul>well, tty tomorrow
<civodul>maybe this fix will be for 3.0.4
<civodul>we'll see!
*civodul -> zZz
<mwette>RhodiumToad: You mean -O2 breaks libguile/lightening/*.c or libguile/jit.c?
<RhodiumToad>I mean that the bug I was seeing when it was compiled with -O2 could not be reproduced at -O0.
<RhodiumToad>investigation continues. but since I make the mistake of recompiling at -O0 in the original workdir, I now have to compile again to get -O2.
<RhodiumToad>(in a new workdir this time)
<mwette>So maybe worth compiling w/ -O2 and then selectively compiling other parts to -O0. Is there an __attribute__ to control optimization at a function level?
<RhodiumToad>haven't checked. but now that I know more about where to look, i can probably get useful info from gdb even at -O2.
<mwette>there is __attribute__((optimize, <level>)) but don't know what level arg is supposed to be
<mwette>it's a string, so I'm guessing __attribute__((optimize,"O0")) might work
<RhodiumToad>I don't see such an attribute in clang.
<RhodiumToad>there seems to be an __attribute__((optnone))
<mwette>Ah. I was looking at gcc manual.
<mwette>OK, I got gcc to accept this form: int __attribute__((optimize("O0"))) foo(...) { ...}