<stikonas>so I think we should stop at autoconf 2.69 for now <stikonas>although, 2.70 has better support for reproducible builds... <bauen1>hm, so i've made some changes to use unshare with chroot (a bit better than running everything as uid=0) and it seems that musl will fail the build, probably right after the configures script `creating config.mak... done` <bauen1>maybe something weird to do with how i setup devtmpfs in the chroot <stikonas>configure script there is in general mildly broken <stikonas>even if there is proper /dev/null device node <stikonas>might be some bash (possibly bash-mes) bug <bauen1>as in it will try to mknod /dev/null ? <bauen1>because afaik that won't be possible in a userns <stikonas>bauen1: oh, we do create /dev/null node later <stikonas>at that time we don't create device nodes yet <stikonas>later we create /dev/null which is actually used by the scripts <stikonas>fossy: argh, we are still using some generated files for GCC... <stikonas>ok, I think just two files, everything else is fine, but these two files will be really annoying <Hagfish>wow, what a nasty surprise lurking in gcc <Hagfish>i guess there are no obvious patches/shortcuts that could skip autogen/guile just for those 2 files <stikonas>well, unless somebody writes manual Makefile <stikonas>downgrading to gcc 3 might be the most practical for now <stikonas>haven't investigated yet, just found this issue... <bauen1>stikonas: in a userns you can't use mknod as far as i know, so i create a dev dir with empty files and bind mount the individual devices (null, zero, urando, random, tty, ptmx) <stikonas>bauen1: that might work temporarily for you, but I'm not sure what will happen with that configure bug that overwrites /dev/null. I guess in a user ns it won't be able to do it <bauen1>stikonas: i'm not sure really, because i think the configure script itself actually finishes without issue <stikonas>well, inside normal chroot it creates /dev/null file containing some text <stikonas>in later runs we already mknod /dev/null, and configure script overwrites it <bauen1>so if it already exists it shouldn't be a problem right ? <stikonas>it shouldn't, mknod should actually be skipped <stikonas>because all invocations are with test || mknod <stikonas>e.g. test -c /dev/null || mknod -m 666 /dev/null c 1 3 <nimaje>probably it isn't a character special file (from test(1) "-c file True if file exists and is a character special file.") <stikonas>no, mknod that we create is character special file <stikonas>it's something in musl's configure script that overwrites it <bauen1>stikonas: the configure script does a bunch of redirection to /dev/null and runs cc with outputfile as /dev/null a bunch time, so imo it would make sense to have /dev/null present from the beginning (or at least before building musl) <bauen1>and it also uses /dev/null as input when creating temporary files, so it might not except random junk in there <stikonas>bauen1: well, we can't create /dev/null too early <stikonas>presense of /dev/null block device does not matter <stikonas>they all replace /dev/null character device with random junk <bauen1>well they shouldn't if it's an actual block device ? <stikonas>they shouldn't, possibly bug in our bash <bauen1>but it's very weird trying to debug this error, looks like the `Subprocess error 256` is from kaem ? <bauen1>and the echo line i put after the configure doesn't get hit even though the configure script supposedly reached the end <bauen1>even printed the last line correctly `creating config.mak... done` so that's quite confusing <bauen1>i also don't see any ANOM_ABEND audit log entries in my hosts audit system which is what would happen if some bug caused bash to segfault <stikonas>Subprocess error 256 from kaem simply means that bash exited with non-zero exit status, ignore that <stikonas>so somehow configure exits eith non-zero code just before it finishes? <bauen1>i've added some stuff to print me the exit code of the configure script, hopefully that sheds some light on the why <bauen1>is that the first time bash is ever run ? <bauen1>well let's wait for this run to finish <bauen1>perhaps some sort of unit tests for the components could be helpful for debugging but i'm not sure <stikonas>yeah, some tests might be useful for debugging, although, most errors are apparent very quickly, as bootstrap process itself is kind of self-testing <stikonas>so any error and you are unlikely to build the next thing <bauen1>and echoing $? is of the opinion that the configure script exited with code 512 <gforce_de1977>i did not know, that this is even possible, was the opinion, that 255 is max 8-) <bauen1>if you've build live-bootstrap in a normal chroot before, what kernel version where you using ? <stikonas>chroot mode just uses whatever your distro is running <stikonas>although in qemu I use the same kernel too (but both are self-compiled gentoo kernels) <stikonas>bauen1: so 5.10.23-gentoo-gnu but I guess that alone wouldn't say much <bauen1>stikonas: both, i'm just trying to figure out if the kernel could play some role (linux 5.10.24-1 the latest from debian unstable) <bauen1>maybe my hardening is messing with it, but i also don't see anything audited in journald <bauen1>might be time to spin up a test vm and run it with a normal chroot there <gforce_de1977>what comes to my mind regarding /dev/null: when we compile a kernel with devtmpfs with automount and fill the initrd with /dev/null (which is a configure/compiletime option) than we dont have to mess with such stuff. we just have a kernel which just knows /dev/null <bauen1>how can i escape $ in kaem ? \$ ? <stikonas>bauen1: escaping in kaem is very limited <stikonas>that's why we build bash as soon as possible <bauen1>right, but not being able to actually express every ascii character in the file to pass to other commands seems a bit concerning <stikonas>but kaem is really just to get started, so it only has enough features to get us to next step <stikonas>as soon as we get make, we switch build srcripts to makefiles <stikonas>(called from kaem but it's then very simple call with definitely no escaping needed) <gforce_de1977>bauen1: which string exactly you want to build? or is it about $? <bauen1>gforce_de1977: i just wanted to quickly test bash, but i've now written a proper bash script to do the testing <gforce_de1977>because 'kaem' belongs to our initial binary seed, we should not extend, but minimize it <stikonas>kaem is not really an initial binary seed <stikonas>kaem-optional-seed is but it is different program from kaem <stikonas>kaem-optional-seed just runs command with arguments, no variables, no escaping, nothing... ***lukedashjr is now known as luke-jr
<bauen1>gforce_de1977: my line of thinking was more that it should be avoided to introduce such "bootstrap problems" ourselfs (like the inability to output a $ without an existing file that contains this $) ***lukedashjr is now known as luke-jr
<stikonas>fossy: pder: should we temporarily disable build of gcc in live-bootstrap? <stikonas>(and I guess bring back some older? bash) ***lukedashjr is now known as luke-jr
***lukedashjr is now known as luke-jr
<pder>stikonas: do you know if fixincludes is an issue, or just Makefile.in? <pder>I looked at the git history of Makefile.in and it looks like it started to be autogenerated in 2002 <pder>getting autogen working is probably a dead end at this point because we need guile and thats probably not possible to bootstrap without significant work <pder>maybe autogen could run on mes? Or we could write some perl to transform Makefile.def and Makefile.tpl into Makefile.in <stikonas>but in any case, we'll have to bootsrap guile if we want gcc newer than 3 <stikonas>unless we build C++ compiler with gcc 3 and jump to clang... <stikonas>this will very significantly deviate from guix bootstrap though... <stikonas>pder: but guile might not be too hard... <stikonas>it's just that I'm not very familier with it, there is also that pp-syntax issue that mihi sovled for 3.0.2 <stikonas>but we probably need to build some other guile version for autogen <stikonas>anyway, I think I can move some stuff from after gcc to before, like newer libtool <stikonas>but if we bootsrap guile early, that might also be useful to guix <OriansJ>bauen1: kaem, like all pieces I work on are need driven. If you "need" kaem to support \$ in some specific way to do something useful in bootstrapping. I'll add it or show the work around for it. (I try not to add support for features until someone expresses a need for them) <OriansJ>stikonas: deviations from the guix bootstrap while doing a no-pregen bootstrap seem just the cost of improving Guix's bootstrap <OriansJ>with a few minor tweaks to paths (into /gnu/store with sha paths) of course <fossy>stikonas: I will look into guile bootstrap today <fossy>I wonder how new it needs to be <fossy>If we can avoid using a version with psyntax entirely then its just another package <stikonas>well, at the moment I'm rebasing xz PR beforeGCC <stikonas>(it's the same static thing inside array) <stikonas>maybe we can find which patch from tcc mob fixes it <OriansJ>fossy: well the psyntax problem was recently solved. <OriansJ>(some porting might be required for older versions of guile) <OriansJ>now if there is other issues with bootstrapping guile without pregen I am unaware of them and they are probably something that need addressing in guix too. (guile devs seemed open to the psyntax bootstrap work; hopefully they upstream it soon and take the maintance work off our plates) <fossy>OriansJ: yeah i know, porting to <3.0 would proabbly be annoying <fossy>stikonas: FWIW, with your libtool in the xz pr, that is now new enough to work with autoreconf :) <fossy>hm, guile 3.0 is going to be pushing the limits of our versions of auto{make,conf} and libtool, needs very new things <fossy>stikonas: i'll just follow your formula for autotools lol