IRC channel logs

<oriansj>and since ${ARCH}/bin works best, that is what will be done. I am also going to be adding a ${ARCH}/artifact to hold every build temp and hold file, to enable one to perform a manual step audit after the fact

<oriansj>and it'll solve where to put the mescc-tools-extra build artifacts problem when doing multi-architecture parallel builds

<stikonas[m]>sounds good, will be less messy

<stikonas>hmm, forgot to wire in after.kaem hook for risc-v...

<stikonas>well, we can do that for the next release

<stikonas>sounds like it might be soonish anyway

<stikonas>fossy: do you want to have exec in kaem?

<stikonas>might reduce the number of parent processes that are waiting

<stikonas>some 5 minute attempt at implementing it: https://github.com/stikonas/mescc-tools/commit/96e2d701b9853083ff88440c61e88ee1390de3d5

<oriansj>yeah, it is gonna be a full rewrite of the kaem scripts for stage0-posix but it'll be much cleaner

<oriansj>So I'll be doing each arch as a separate commit. It is *BIG*

<fossy>stikonas[m]: i have no problems with that

<oriansj>and x86 for stage0-posix has been entirely isolated

<oriansj>and I'll isolate the rest over the next couple days

<pabs3>"Supply Chain Security and Tar" https://www.openwall.com/lists/oss-security/2021/10/03/1

<oriansj>advantage of stupid tools. Less places for attacks

<oriansj>and AMD64 has been isolated

<xentrac>isolated?

<oriansj>xentrac: the stage0-posix bootstrap builds are now being isolated into their architecture specific directories to enable all of the architectures to be build in parallel to speed up cross-platform testing.

<oriansj>it is a big change so I am doing each architecture one at a time

<oriansj>as you can see here: https://github.com/oriansj/stage0-posix/commit/3e330783345d1ec5448df3de7cf5eb7c9734e11a

<xentrac>aha, cool!

<oriansj>and now all of the temp and hold files created during the bootstrap are now preserved to enable an audit to go in and find exactly where something might have corrupted a bootstrap

<oriansj>So one can archive (tar up) the artifact folder to provide a full audit trail for every single binary produced by stage0-posix

<xentrac>wonderful

<Hagfish>"It is possible to create a source distribution which leads to different files seen by the build environment than compared to a careful reviewer and other Linux distributions."

<Hagfish>oof, that's sneaky

<fossy>i think a very easy way to make sure this dosen't occur is to just extract a few times with a few different tar implementations and make sure they are all the same

<fossy>probably bsdtar and gnu tar

<Hagfish>that's a cheap solution, yeah

<Hagfish>i also think that there should be a --paranoid option to these tools which check that the archive doesn't contain any strange data

<Hagfish>for example, there shouldn't be any data in the archive that doesn't end up as a created file (other than metadata)

<Hagfish>when extracting

<stikonas>we also have mescc-tools-extra untar

<Hagfish>that would be a good place to add checks that nothing dodgy is hiding in the .tar files

<stikonas>well, but that tar thing just looks like a bug in implementation

<stikonas>it's not a problem in bootstrapping

<stikonas>because you can review untar source code

<stikonas>and make sure that behaves as required

<stikonas>it is a problem if you rely on tar blobs to unpack other tarballs (including tar itself)

<stikonas>mescc-tools untar actually returns 1 on that archive

<stikonas>so it does have a bug

<stikonas>which is expected I guess since that implementation is taken from libarchive

<Hagfish>interesting. thanks for checking

<xentrac>interesting source of nondeterminism I learned about yesterday: recent versions of GCC build everything as PIE by default, so that recent Linux kernels (or ld.so?) can employ ASLR with the segments of the executable

<xentrac>of course this doesn't have to result in nondeterministic externally visible behavior of the executable, but it can

<xentrac>happy Sputnik Day!

***pgreco_ is now known as pgreco

<oriansj>well untar is not defensively programmed yet.

<oriansj>So if someone wants expand it with input validation, it will probably be a good idea

<oriansj>xentrac: all I can think of when I see that name is: https://archive.org/details/GETLAMP-Supnik

<xentrac>haha

<stikonas>oriansj: can you merge https://github.com/oriansj/mescc-tools/pull/27

<stikonas>fossy said he is happy with exec built-in

<stikonas>hmm, M2-Planet has a crashy argument handler...

<stikonas>"M2-Planet -f" segfaults

<Hagfish>oof. that's something a fuzzer could find, right?

<stikonas>I guess depends on how you run fuzzer

<stikonas>if it doesn't fuzz command line arguments, then it won't find it

<stikonas>well, the code just uses argv[i +1] without checking argc

<stikonas>so a single if there to compare argc and i+1 would avoid the crash and then can exit gracefully

<fossy>stikonas: would you mind also adding a check that execve never returns? Just add a line below that exits if it ever gets there after the execve

<fossy>I would prefer to fail fast there in case the exec fails

<Hagfish>smart

<stikonas>fossy: don't we already have _exit(EXIT_SUCCESS); ?

<stikonas>or do you want meto fail in case of exec?

<stikonas>oh, actually we don't want EXIT_SUCCESS...

<stikonas>if execve fails...

<stikonas>yeah, I can add it

<stikonas>no too sure how to test it though...

<stikonas>fossy: did you mean this https://github.com/oriansj/mescc-tools/pull/27/files ?

<fossy>Yes

<fossy>I am unsure why the other one is EXIT_SUCCESS anyway

<stikonas>hmm, the other probably doesn't matter that much...