IRC channel logs
2025-01-22.log
back to list of logs
<fossy>yes, you can hack around needing autogen for later GCCs. however i found that more difficult for GCC 10 (I did attempt autogen-less); but other people were already working on guile, and it became much easier to just use guile <fossy>live-bootstrap currently has no notion of dependencies. so it is difficult to do any kind of reverse-optionality <stikonas>there is a bit of info in parts.rst though <fossy>yeah, i mean programatically <fossy>replacing early stuff with busybox is probably viable <fossy>but i have no evidence for/against that <stikonas>I think I looked at busybox early in the live-bootstrap <stikonas>though meslibc now has more functions too compared to 3 years ago <fossy>hmm, yeah skimming over parts.rst again actually looks much less viable <fossy>by the time we get to musl we have most things we would want from busybox <fossy><lrvick> though, maybe I run m4 and company to generate the actual final compile commands each project needs <-- wrt this, I don't find this to be "fairly auditable" at all <fossy>something like GCC has many *thousands* of commands <fossy>so i think that you need to edit the commands run fairly significantly to make it auditable, something like what we do for early fiwix where it ends up being just cc -c -o main.o main.c over and over again <fossy>but then you run into the problem that if you ever want to change anything you have to redo all that work <lrvick>fossy: the rule of thumb for me is which is easier to audit. Cleaning up generated code, or reviewing all the code that does the generation <lrvick>I am not actually sure which is less bad in this case <lrvick>related to the above, I made a very aggressive fork of live-bootstrap with the sole goal of bootstrapping an x86 gcc13 container using containerized workflows to rapidly hack on and remove steps: https://codeberg.org/stagex/bootstrap <lrvick>some experiments I am planning on <lrvick>replace all versions of patch, gzip, tar, sed, bzip2, bash, grep, findutils, coreutils with busybox <lrvick>replace all builds of autoconf/automake with acr <lrvick>those feel like the lowest hanging fruit <lrvick>I would much rather review shell for anything nasty than c, etc <lrvick>though no getting around reviewing the actual code used in each step <lrvick>Also very happy to discuss/debate the merits or implausibility of any choices <stikonas>shell can be quite nasty though... Generally outside bootstrapping once your script grows a bit bigger it's often better to rewrite it in some other language... <lrvick>Yes. Now that we have golang just after the first version of tinycc, that might be an unlock to remove a lot of C code <lrvick>go is not my favorite language, but it does have a huge standard library, and I would way rather review some small utilities in go than c or shell <jackdk>Wait, what? That's awesome - when and how did that happen and has someone written it up as a blogpost or something? <lrvick>go in stage1? It was part of a series of experiments by a couple people on the stagex team <lrvick>you can inject it right after tcc-0.9.27 and it works <lrvick>though we think we can get it earlier <lrvick>but go 1.4 does require binutils, at least on a surface level <lrvick>but if we can patch that out, then we can move it to maybe step 3 <lrvick>but if we can replace everything in the path to binutils (automake/autoconf/coreutils/bzip2/grep/diffutils/gawk/bison) with acr/busybox then it might not be worth it <oriansj>stikonas: yeah, I probably made that bug for Knight (I will probably fix it up later this week) <fossy>"<lrvick> fossy: the rule of thumb for me is which is easier to audit. Cleaning up generated code, or reviewing all the code that does the generation" <fossy>one of my pain points with generated code beyond unauditability is its untrustworthiness; if i wanted to inject a (source code) backdoor into a well-written application, i would target committed, generated code first, where i can pass off the malicious code as committing a regenerated file (because i changed something in the inputs to said file) <fossy>and few people would actually review that <fossy>lrvick: would love to know how you go with busybox. did you see stikonas' comments about busybox not working with meslibc? do you have a plan to work around that, apart from attempting it? <fossy>with acr, is it actually compatible with autoconf???? <fossy>or do you have to rewrite all autoconf things in acr? <fossy>idk, acr does not look promising to me <matrix_bridge><Lance Vick> fossy: acr claims to be argument compatible, so we will see. untested <matrix_bridge><Lance Vick> Anything someone has not already tried ror proven cannot work, I might as well try <matrix_bridge><Lance Vick> plan B if busybox does not work out is u-root which is a busybox replacement written in go <matrix_bridge><Lance Vick> now that we have go very early it is maybe an option <matrix_bridge><Lance Vick> I do see afew examples of people building busybox with tinycc though <matrix_bridge><Lance Vick> though I may not be able to build it against M2libc <matrix_bridge><Lance Vick> tcc -> uclibc -> busybox is my current strawman <matrix_bridge><Lance Vick> surely m2libc/tcc can get me one of klibc or ucllibc <matrix_bridge><Lance Vick> seems tinycc has supported uClibc from 0.9.25, so one would hope it can also -build- uClibc <matrix_bridge><Andrius Štikonas> Yes, we looked at most libc and were only able to get musl to work <aggi>busybox can easily be compiled/linked with tinycc, although i hacked musl-libc for support with linux24 <aggi>i didn't track yet at which stage linux2/fiwix ABI is required, to keep a consistent bootstrapping path without circular dependency <aggi>musl-libc does need a linux2/fiwix abi of cause, mes-libc may not <aggi>as far as busybox was concerned, i had chosen version 1.2.2.1 since this didn't transition to the special kbuild system as later versions did <aggi>and there might have been something related to Perl needed with later versions <aggi>anyway, a busybox/tinycc bootstrapping ideally would not need a fiwix/linux2 abi <aggi>it's a little tricky if bootloader and kernel bootstrapping are considered; hence the proposal to re-write bootloader.S assembly parts (tccboot and kernel) for as86 <aggi>a kernel then can be compiled JiT with tccboot then (which i already confirmed this does work with an updated libtcc even) <aggi>if a static busybox could be linked against mes-libc, this should work with a linux2/fiwix runtime abi, i think <aggi>the blocker seems to be limitations of mes-libc and busybox that could not link against it (although tinycc can) <aggi>something else i didn't think of, to wrap a bootable image, besides tccboot some syslinux/isolinux is needed, which too contains 16bit assembly <aggi>i already confirmed syslinux-3.86 compiles/links with tinycc <aggi>i do emit (non-UEFI, PC-BIOS) ISO images; with those you would either need 1) cdrecord utility to write those to cdrom <aggi>or 2) isohybrid/perl utility, so such an ISO image can be dumped onto usb flash <aggi>for 1) cdrecord i confirmed cross-compilation with tinycc already (statically linked against hacked musl-libc compatible with linux-2.4) <aggi>this might introduce another blocker to link cdrecord against mes-libc <aggi>2) isohybrid/perl seems not feasible with limitations of mes-libc <aggi>with regards to build-time dependencies during bootstrapping, in principle, it seems all of this had to be accomplished with mes-libc and _before_ a fiwix/linux2 abi was available <aggi>seems there is no way around the intmerediate kexec towards fiwix <aggi>fiwix does support PATA-IDE, maybe cdrecord support could be added to live-bootstrap <aggi>a completely different approach which slipped through my mind: bootstrapping some FreeDOS type system as intermediate development host <aggi>historically, that's how linux could be loaded (umsdos or however that was called) <aggi>i've not had a look at 16bit real-mode DOS for ages <fossy>lrvick: at that point it is mes libc, not M2libc <fossy>lrvick: let me know how you go with those, would be interested to hear <fossy>tmg1|michelson: what do you mean by that? <matrix_bridge><Lance Vick> ah yeah so the actual path would probably be more like: m2libc/M2-Planet -> mes/mescc -> tinycc/mescc -> uClibc -> busybox <stikonas>but how do you configure uClibc build system? <stikonas>don't you need to run "make config" or something like that <stikonas>I think we did look at all possible libc's... <stikonas>and musl was the only option that we were able to proceed with <stikonas>well, I guess we do build bash before musl too <stikonas>but then you need to build bash before busybox too <stikonas>and bash will automatically pull in all those earlier tools between tcc and musl <stikonas>perhaps you could avoid bash with gash... <matrix_bridge><Lance Vick> for uclibc the only way forward I could think of really early is just running hacking up the makefiles and make config to produce the final set of actual tcc commands that would be run as a kaem script. <matrix_bridge><Lance Vick> but is that 100 commands that could be easily dealt with, or thousands, I don't actaully know <matrix_bridge><Lance Vick> the easiest wins then may be to build busybox just after the first musl when we build go, then can drop a lot of builds after that point <matrix_bridge><Lance Vick> It is a hard puzzle I know you guys spent a lot of time on. I appreciate you talking me out of known dead ends so I can focus my attention on paths not fully explored yet. <aggi>what would the benefit of uclibc be, if musl-libc can be hacked for linux2.4 api/abi? 