IRC channel logs

<aggi>getting closer, tccboot initially linked against an even older tcc-0.9.21...

<aggi>hence didn't need various symbols

<aggi>that's for JIT compilation of linux-2.4

<aggi>yet for AoT compilation, a test-case was reported somewhere on github, that referred to a tcc-0.9.26..27

<aggi>last time i tried, the kernel aot-compiled with tcc-0.9.27 loaded, booted and crashed

<aggi>and i recompiled/linked a tccboot-loader against that version too... in this case tccboot spawned, and got stuck/crashed

<aggi>tccboot-loader introduced a circular dependency, but i want a confirmation a linux-2.x can be compiled/linked/loaded with tcc, and does not crash

<aggi>because, tccboot.iso does work as it should

<aggi>and isn't reproducible yet

<aggi>nor was the AoT compilation of linux-2.x reproducible, without a kernel-crash

<pabs3>Googulator: just sent a request for SWH to auto-save ghc.git https://archive.softwareheritage.org/save/ https://archive.softwareheritage.org/save/list/

<aggi>yet, if i can identify the correct tcc-version for tccboot, then i may bisect/hack AoT compilation of it too... bingo.

<aggi>that's why, i may want to carefully review tccboot, to see exactly which tcc-version was desirable

<aggi>according to my memories, #bootstrappable used tcc-0.9.27, and had some patch for fiwix kernel too

<aggi>yet this recent tcc seems a slightly different beast than the old one from year-2004

<aggi>and i cannot conclude yet, which was the best tcc-version to pick, to support whatever: bootstrapping gcc/binutils, JIT compile/link kernel, AoT compile/link kernel

<aggi>how had #bootstrappable avoided a potential circular dependency for any bootloader to compile with whatever?

<aggi>probably stupid question... nonetheless

<stikonas>aggi: what do you mean by bootloader?

<stikonas>we don't use any bootloader in #bootstrappable

<stikonas>at least not until your final system is ready for disk

<stikonas>but yes, tcc 0.9.27 could build fiwix

<stikonas>and then kexec into it

<stikonas>we'll probably switch to tcc 0.9.28 soonish for riscv64 support

<aggi>stikonas: imagine if there was any computer (real-hardware), which starts, and knew how to read some bootsector, and jump to anything written in ASM for example, calling bios routines

<aggi>a system does need to load something, even if it was some tinybasic punched paper strip

<aggi>or, in case of tccboot, it's first syslinux, then tccboot (with related circular dependencies)

<stikonas>well, yeah, ideally you would have something that can read some simple machine code from paper tape or something like that

<stikonas>in the absence of that, we just use prepared disks / USB devices or something like that

<stikonas>which of course introduces some dependency on another machine

<aggi>this answers the question, insofar there is potential circular dependencies for the official bootstrapable itself too

<aggi>such as, an assembler is needed, the assembler has to be compiled with gcc, and then the assembler can process some early-stage bootstrapping code to arrive at some gcc again later

<aggi>although you did implement various miniature assembler, compiler, libc etc.

<aggi>don't know how that's loaded by a x86 BIOS (16bit real-mode) to proceed anywhere

<aggi>besides, tcc hasn't got support for x86 16bit real-mode

<stikonas>no, assembler is not needed

<stikonas>assembler we can build with hexers

<stikonas>e.g. hex2

<stikonas>it can go down to as little as 200 bytes of code

<stikonas>(see builder-hex0-mini)

<stikonas> https://github.com/ironmeld/builder-hex0/blob/main/builder-hex0-x86-stage1.hex0

<stikonas>well, this builds to 512 byte MBR sector

<stikonas>but large part of it is padding

<Googulator>yeah, 192 bytes of actual code & initialized data, plus some boilerplate to convince stubborn BIOSes that it's a valid MBR

<stikonas>can probably be compressed a bit further

<stikonas>e.g.

<stikonas>EA 11 7C 00 00

<stikonas>this is jump to 0x00007C11 address

<stikonas>I guess one can use short 8 bit relative jumps

<stikonas>that's 1 byte for opcode and 1 byte for relative jump

<stikonas>there is also something similar for UEFI, though quite a bit larger

<Googulator>no, that's actually there for a reason

<stikonas>UEFI hex0 is maybe 800 bytes

<aggi>MBR layout and asm hasn't been my strenght (although i did dig a little into this more than 20years ago), i was curious nonetheless

<Googulator>sets CS to a known value

<stikonas>though large part of it is PE32+ header

<stikonas>Googulator: oh I see

<Googulator>a healthy BIOS doesn't need it, but some buggy ones do

<aggi>meaning, #bootstrappable beginning with stage1.hex0 can run on real x86 hardware, and utilize bios-calls?

<stikonas>yes, it can

<Googulator>basically yes, stage1.hex0 is converted to binary, and written to the MBR

<Googulator>by rootfs.py

<stikonas>though you can convert it yourself into binary

<stikonas>that's not a problem

<stikonas>the problem is only getting it onto actual machine

<Googulator>that's literally the only binary machine code that's on the bootstrap disk

<Googulator>everything else is source

<Googulator>stikonas: more speciflcally, the problem is getting only that onto the machine, and nothing else, guaranteed

<aggi>interesting...

<stikonas>well, that too

<Googulator>a malicious actor infecting the host system could pretend to write just those 512 bytes, but actually put more binary code there in secret, thus propagating to the bootstrap system

<Googulator>and compromising the bootstrap

<stikonas>you needfront panel with PDP-8 switches :)

<Googulator>which is why I devised the "trusted Flash drive" with swappable SPI flash

<Googulator>smallest SPI flash on the market is 256 bytes, largest is 256 MiB, and their basic read commands are compatible

<aggi>it's not a PDP-8, and i was born too late to be raised with such a device:

<aggi> https://en.wikipedia.org/wiki/LC80

<stikonas>well, I'm too young for PDP-8 too

<stikonas>just heard about it

<stikonas>haven't actually seen one in real life

<Googulator>rekado: having a go at reproducing your Haskell results with a sufficiently old gcc (https://elephly.net/posts/2017-01-09-bootstrapping-haskell-part-1.html#org360295f), do you remember ever having to call greencard manually?

<Googulator>I'm running into an issue where building the prelude fails early because it can't find the module NhcFloats

<lanodan>I'd say host infection is countered by reproducibility, after all the BIOS (or whatever other boot firmware) can be infected. And I guess if live-bootstrap would have a smaller payload it would be more reasonable to emulate from another architecture.

<janus>GoogulatorMobile: what i did is just use RedHat 6.2, then you don't need all this patching, NHC98 works almost out of the box

<janus>i don't understand why you'd want to try to port compilers to platforms they didn't run on back then, it's unnecessary work... just take the old system

<janus>i talked to Simon Marlow and he mentioned that GHC was most likely bootstrapped on SPARC... i haven't started going down that route yet

<janus>GoogulatorMobile: are you at zurihac?

<stikonas>fossy, Googulator: so should we bump to 0.26.1 in master or also in the branch?

<stikonas>at the moment branches are still equal

<stikonas>so I would lean for 0.26.1 in both

<aggi>i see now what happened to tcc... it supported linux-kernel compilation with a rather early version in conjunction with tccboot

<aggi>then a notable amount of important patches were added, _and_ then it was re-structured internally splitting .c/.h files

<aggi>thise broke compiling/linking of tccboot, and a few more important patches were added too

<aggi>since then, there was no more confirmation of kernel-compilation with tcc either JIT (tccboot) or AoT (custom scripting)

<aggi>which is problematic, because i may have to bisect my way through 15years of changes, including the fact tcc was re-structured internally breaking a relevant test-case: JIT-compile of kernel

<aggi>bootstrappable's interest is a chain of trust for tcc itself, and bootstrapping gcc/binutils then with it

<aggi>while i consider linux-2.4 (and/or fiwix) an at least equally important criteria for tcc/tccboot

<aggi>inside git-log of tcc, i spotted a few developers which i noticed elsewhere before, such as susematz who tried to bring tcc-support to a recent linux-4.4 (and couldn't finish/failed)

<aggi>in the process of this, support for linux-2.4 was broken too, unnecessarily i think

<aggi>at least i couldn't confirm/re-produce some test-cases

<aggi>and there was rumors in another channel elsewhere, tcc may not be appropriate as system-compiler... of cause, if relevant test-cases were broken

<aggi>and priorities set elsewhere, for gcc and/or llvm/clang... which was OK, if a good known-working state for linux-2.x support had not been abandoned/broken

<aggi>so, i had aggressively forked against #gentoo already, to bring back a c-only/no-c++ system profile

<aggi>i had forked against linux-kernel already, to roll-back to linux-2.X

<aggi>and now it seems, i have to fork against tcc-toolchain too to bring back what was broken!

<stikonas>the problem with tcc is though that tcc is hard to read

<stikonas>especially considering that it is non-optimizing compiler

<stikonas>so shouldn't be too complicated

<stikonas>it's full of 1 or 2 letter variables...

<aggi>ACTION fears for the worst

<aggi>and even when i re-produced/re-compiled tccboot for linux-2.x, i am not convinced of the whole approach anymore

<aggi>another consideration was, to stay at gcc-4.7 (no-c++), yet i had a little too much "fun" inside it's autotools/automake scripts already

<aggi>and this is one corner-stone, where i am worried about #bootstrapple approach, to focus on bootstrapping GNU-toolchain exclusively (if i may say so)

<aggi>for example, if i chose an earlier version of tcc to fork against, and wanted to rely upon #bootstrappable procedure, i may have to fork against some bootstrappable stuff too

<aggi>because this was an overwhelming argument for tcc, that it's supported by #bootstrappable, yet this applies to a recent version which failed against mentioned test-cases for linux-2.x

<stikonas>aggi: it's not like we focus on bootstrapping gnu toolchain exclusively...

<aggi>stikonas: had any version of tcc up until tcc-0.9.24 ever been tested for bootstrapping?

<stikonas>other paths are welcome too

<stikonas>aggi: yes

<stikonas>cosinusoidally has a bootstrap path that skips mes/mescc

<aggi>stikonas: i don't intend to skip mes/mescc

<stikonas> https://github.com/fosslinux/live-bootstrap/pull/407

<stikonas>it's x86 only

<stikonas>though much faster

<aggi>i am worried i chose a tcc-versions which wasn't tested for bootstrapping;

<stikonas>as for building other versions of tcc, no, we can only build some patched versions of tcc with mescc

<stikonas>none of them are buildable directly

<stikonas>we need to #ifdef away floats, and on 32-bit arches long long's and possibly a few other things

<stikonas>that are not supported by mescc

<aggi>well, may not be as problematic, because a tcc-0.9.27 patched for mesc probably will be capable to compile a tcc-0.9.24 later anyway

<aggi>this wouldn't break the bootstrapping itself, because i won't move to tcc-toolchain if the bootstrapping path was broken

<aggi>which it isn't and was a most important criteria

<aggi>it's not the first time, compilers were forked/split (at the time of gcc-2.96, EGCS fork), such as a kcc-version (for kernel), and some other cc-version for anything else

<matrix_bridge><Andrius Štikonas> Yeah, you should be able to compile older versions

<matrix_bridge><Andrius Štikonas> At least on x86

<matrix_bridge><Andrius Štikonas> Definitely not on more open arches like riscv64...

<aggi>that's the reason why i abstain from riscv completely, because it requires most recent compilers/kernel with all the problems those have nowadays

<aggi>i'll get another coffee, and will try to cleanup some bisecting-chaos

<aggi>what's biting me with tcc is another fact, i am on BUILDHOST=aarch32 and this seems to leak into tcc build-system for various versions of it

<aggi>i rather keep that, as another sanity check, for what's leaking into tccboot for example (since unresolved arm symbols showed up all of the sudden with it, even with -nostdlib and -nostdinc precautions taken)

<matrix_bridge><cosinusoidally> I've not tried to build tccboot myself, but a long while ago I did once try to run stage0-posix-x86 under tccboot. iirc there was some weird crash early in the bootstrap process. I didn't get to the bottom of that (and tbh I didn't try too hard).

IRC channel logs

2024-06-08.log