IRC channel logs
2024-04-15.log
back to list of logs
<Googulator>if I remove the #if YYDEBUG ... #endif block from the generated parse.c file, the error changes <stikonas>hmm, but strange, why would it not happen before? <Googulator>given the issues related to 0.9.27 self-hosting with different libcs, I'm not that surprised <Googulator>alright, the preprocessor is going absolutely haywire indeed <Googulator>tried generating with byacc instead - it succeeds (on parse.y), and the resulting parse.c does compile with tcc (invoked by hand), unlike the one made by heirloom yacc <Googulator>so much for not trying to make byacc + heirloom lex work; maybe it's the easy way out :) <stikonas>then we don't even need to build heirloom yacc? <stikonas>Googulator: also have you removed my heirloom patches? <stikonas>I did some nasty wstring -> string stuff to be able to build <Googulator>rm: cannot lstat `*/Makefile.in': No such file or directory <Googulator>rm: cannot lstat `*/*/Makefile.in': No such file or directory <Googulator>(this, of course, is the well-known WSL bash globbing bug, which means, flex bootstrap was successful) <Googulator>How do you even indicate "public domain" in a reuse-compliant way? <stikonas>Googulator: one option is to set it to CC0... <stikonas>(especially since you made some modifications...) <stikonas>Googulator: should we squash all those commits? <Googulator>IMO it's good development history, none of the commits are really fixups <Googulator>Meanwhile, pushed a workaround for Savannah being unreliable <Googulator>(backup mirror links to oriansj's files.bootstrapping.world server) <stikonas>and I checked that that weak symbols/exit musl patch is not needed <stikonas>got the new checksums too but now they are again out of date... <Googulator>in other news, openela kernel passed baremetal testing <matrix_bridge><Andrius Štikonas> Arch, base tarball sneaked in because it was printed at the end of bootstrap... <oriansj>Googulator: yeah I setup files.bootstrapping.world to be append only; so it'll have old version around as long as someone needs them and new versions as well; hopefully we don't need more than 200GB of tarballs. <oriansj>Googulator: no problem, what is the expected checksum? <oriansj> 708c943f89c972910e9544ee077771acbd0a2c0fc6d33496fe158264ddb65327 ?? <matrix_bridge><Andrius Štikonas> Hmm, we should be more careful with rnames in live-bootstrap <matrix_bridge><Andrius Štikonas> Right now we were renaming various tcc snapshots from janneke to tcc-0.9.26.tar.gz <matrix_bridge><Andrius Štikonas> But that would not work with files.bootstrapping.world <Googulator>That's basically something we had this way since time immemorial, but I agree, we shouldn't continue doing that, especially now that there's an established convention for situations like this <Googulator>BTW, #356 (mes-0.26 + nyacc regen) is ready for review again <Googulator>solved the double compilation with a single "replace" <Googulator>the downside of regenerating those nyacc files is - _it takes a while_ <matrix_bridge><Andrius Štikonas> Googulator: nice work! 6+ minuted on CI should be acceptable... <matrix_bridge><Andrius Štikonas> But at least we are properly bootstrappable there <stikonas>Googulator, fossy: have you tested recently with --external-sources? <stikonas>oh, that might be due to my dirty workspace... <stikonas>that I had unpacked some tarballs for inspection <Googulator>qemu or baremetal with --external-sources may be broken indeed, but I'd be surprised if we broke bwrap <stikonas>well, it was just "extra unexpected files" <stikonas>Googulator: mes 0.26 PR looks good but I left a minor question about commented out bits <stikonas>and I think let's leave out riscv checksums for now <stikonas>the annoying thing about riscv checksums is that you either need hardware (which is really slow with mes) or qemu. And qemu is slow too (just not as bad) but also some versions of qemu don't work with stage0-posix-riscv64 (random segfaults). <Googulator>pwn2de4d/riscv-toolchain Docker image seems to work for me <Googulator>(needs some trickery to set up binfmt_misc inside the container though) <Googulator>start it with `docker run --name "riscv-toolchain" -v ./target:/target -v /proc/sys/fs/binfmt_misc:/binfmt_misc --privileged -it pwn2de4d/riscv-toolchain:latest` <Googulator>in the container, issue `echo ':qemu-riscv64:M::\x7f\x45\x4c\x46\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\xf3\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/opt/qemu-riscv-static/bin/qemu-riscv64:CF' > /binfmt_misc/register` <stikonas>well, for now I just use an older qemu from my gentoo system (with binfmt configured) <stikonas>but I didn't fully understand the cause of the crashes... <Googulator>and finally `chroot . ./bootstrap-seeds/POSIX/riscv64/kaem-optional-seed` <Googulator>not sure what qemu version this is, I didn't check <stikonas>with other qemu versions the crash could happen at random point in bootstrap (though generally around the time M2 is first built, I never managed to reach end of stage0-posix) <Googulator>Copyright (c) 2003-2022 Fabrice Bellard and the QEMU Project developers <Googulator>mach.d regeneration took 95 minutes with this qemu on a Zen 4 <stikonas>though real HW board that I have is even slow (which we thought is due to slow memory bandwidth) <stikonas>anyway, nice that this is out of the way <stikonas>we are basically out of blockers for 1.0 release <Googulator>I guess it's more dependent on CPU speed / IPC, vs. mes->tcc, which is memory bandwidth limited <Googulator>we still have script-generator checksum, but that's easy <Googulator>NSS update is trivial (in fact, has been done at least once since #292 was posted) <stikonas>given that root certificate expires, it's best to be as up to date as possible <stikonas>yeah, most of the other stuff is trivial/documentation <Googulator>"Describe how to "latch on" to the end of live-bootstrap (eg, freedesktop-sdk)" - fairly trivial as well, just add an extra improve step to the manifest <stikonas>well, even before there was after.sh script <stikonas> Provide option for not all command output to be displayed on-screen <stikonas>for 1.1 it would be nice to package ncurses/readline too... <stikonas>since now a lot of our tools have fairly poor command line editing capability <stikonas>it's useful but probably needs more discussion, see what fossy thinks <Googulator>but all of these are really 1.x, not 1.0 material <Googulator>Bootstrapping without rootfs.py is gonna be a bit of a b*tch though <Googulator>we do have some instructions that could be updated, but they rely on manually maintained pre-network-sources file <Googulator>we have code to generate the equivalent - but it's written in Python and calls into generator.py <Googulator>and IMO writing "Only copy distfiles listed in sources files for build steps coming before improve: get-network in the manifest into this disk." would be a bit of a dick move <stikonas>or just write that you needto go over all sources files <stikonas>but it's not particularly complex thing to do <stikonas>Googulator: oh another thing that might be good to restore <stikonas>(given that kernel bootstrap only works on BIOS) <stikonas>but needs somewhat awkward manual editing of bootstrap.cfg <stikonas>you kind of want to follow chroot/bwrap bootstrap path in that mode <stikonas>or maybe we should focus on fixing UEFI bootstrap... But that's not super easy either <stikonas>I haven't done any work on posix-runner recently <Googulator>Won't that also disable kexecing into the built kernel? <stikonas>Googulator: well, that path is broken if you start with linux kernel <stikonas>I don't think we support kexecing from 64-bit kernel to bootstrapped 32-bit kernel <Googulator>I did several 32->64->32 tests when bootstrapping Gentoo <stikonas>hmm, fossy was under impression that it was broken <stikonas>anyway, this is something that we didn't test much and documentation is lacking <Googulator>AFAIK the originally intended usage of -qk is for booting with an old Linux kernel on BIOS, and then kexecing up to the newly built one <Googulator>performing essentially a chroot bootstrap on top of a recent kernel should be a new mode <Googulator>tried to test -qk here - qemu won't pass an e820 memory map to a kernel specified via -kernel <stikonas>well, it's probably fine to do 32-bit kernel bootstrap starting with 64-bit kernel if that works <stikonas>(since you can't directly boot 32-bit kernel on UEFI) <Googulator>stikonas: seems like that grub hash change is reproducible <stikonas>usually these things are in documentation, some month is hardcoded <Googulator>no, it seems to actually depend on that one patch somehow <stikonas>that patch is so early in the build chain <stikonas>anyway, I'll probably wait till after mes 0.26 and redo checksums again <stikonas>I don't want to delay that thing for some cleanup... <Googulator>actually - it turns out that the grub tarball that gets produced is unchanged from before the commit <Googulator>but somehow your commit includes a new expected hash for it <Googulator>do you happen to have the tarball that actually hashed to 0bf613f1f142d53baea81643383b66dcaf38438ee6ba33fb9ed1c135a0be9e47 ? <stikonas>and I think best to merge mes 0.26 first anyway <stikonas>after that tcc 0.9.27 has no riscv support <stikonas>if we want to continue with riscv, we need ot switch to tcc-mob <ekaitz>more than usable at this point yeah <stikonas>but I think it's not far from being done <ekaitz>i'm pretty happy with it at the moment <ekaitz>not perfect but pretty near from being stable-ish <stikonas>well for now we are doing the minimum needed for bootstrap and not doing any of the pregenerated file stuff <ekaitz>from my side i'm interested on having a proper tcc, so i implemented way more than we needed