IRC channel logs
2025-12-06.log
back to list of logs
<rlb>Could I get a second opinion? In the srfi-64 spec it says in one place that an xfail result means "the test failed and was expected to", but for test-runner-xfail-count it says "returns the number of tests that failed, and were expected to pass". Is that second definition right, if the count is a count of the xfail results? https://srfi.schemers.org/srfi-64/srfi-64.html <rlb>Trying to figure out if I originally misunderstood the spec and/or if (srfi srfi-64 autoomake) has a bug. <rlb>Right now it considers a positive xfail-count to warrant a nonzero exit, which seems right according to the test-runner-xfail-count description, but perhaps wrong according to the xfail result description. <mwette>I assume the description does not express authors' intent. <mwette>If I read your concern right. i.e., if 3 fails are expected, return 3. <rlb>You mean if a test result is 'xfail three times then the xfail-count should be 3? If so, then that's what I'd have expected too, and makes it seem like the spec's test-runner-xfail-count description is wrong. <rlb>...also means I should probably fix (srfi srfi-64 automake), though I'm also now thinking that maybe its run-tests should always exit 0, i.e. I think the automake test harness/protocol doesn't expect the exit status to be relevant when there's a test driver. <rlb>(I don't know whether or not it just ignores it, or could be "confused" by nonzero.) <mwette>I don't know the details, but yes, "You mean if" is what I was thinking. <rlb>Looks like our new srfi-64 has a different description (that matches what I'd expect): "Return the number of tests that failed, and were expected to fail." <rlb>Though I could imagine you might also want to know about expected failures that didn't fail "somehow". But perhaps that's "out of scope" for srfi-64, proper.