IRC channel logs

<fossy>haha, "who disassembles doors"

<oriansj>fossy: yeah, it is super funny. Until it is your kid and you have to explain to the babysitter that closing the bathroom door to have privacy will likely result in the door falling onto their head by said child wishing to continue playing.

<doras>So where do we stand in regards to kernel bootstrap? Is a Linux kernel still a seed requirement for building sysa?

<j-k-web>> I am starting work on the Fiwix to Linux kexec. - rickmasters 2023-04-13

<j-k-web>also

<j-k-web> https://logs.guix.gnu.org/bootstrappable/2023-03-20.log#070004

<j-k-web>with live-bootstrap when building mes it grabs the mes source + nyacc source but then not much of a mention after that. is it unpacked to the same dir as mes?

<j-k-web>*same dir mes source

<gforce_d11977>here my asciinema recording from head/trunk of yesterday evening with qemu and builder.hex0 kernel: https://creepy77.playat.ch/live/

<doras>I see. So kexec to Linux is still missing.

<rickmasters>doras: yes it only works up to building Linux. We integrated what works so far to make it easier to test changes and make sure keeps working.

<rickmasters>Of course if you use a Linux kernel it will still run to completion like before.

<rickmasters>j-k-web: if you are asking about the nyacc source, it is unpacked in a sibling directory next to mes.

<doras>rickmasters: impressive progress :)

<rickmasters>doras: thanks!

<rickmasters>j-k-web: Check out GUILE_LOAD_PATH here to see how nyacc is located: https://github.com/fosslinux/live-bootstrap/blob/master/sysa/run.kaem

<oriansj>yeah, the combo of mes/nyacc and M2-Planet/M2libc is a good bit touchy

<Mikaku>rickmasters: next week I'll open a new issue for kexec with a commit where you can easily switch from Fiwix to a new Fiwix kernel

<Mikaku>rickmasters: this will put the foundations to add support to be able to switch From Fiwix to a Linux kernel

<Mikaku>I'm preparing the documentation that will help to understand how is kexec implemented in FIwix

<rickmasters>Mikaku: Thank you! That will be very helpful and I'm looking forward to it.

<river> https://matklad.github.io/2023/04/13/reasonable-bootstrap.html

<j-k-web>thanks for the nyacc details both. is there anywhere the `libc-mini.a` etc builder parts live standalone? or do they only live in live-bootstrap mes-xyz.kaem?

<stikonas>j-k-web: no, it's not standalone...

<stikonas>there is also makefile in mes repo that builds libc-mini.a

<stikonas>j-k-web: what you described is not that different to what we already have for rust, just webassembly instead of C

<stikonas>there is a rust to C compiler that is independent of the host architecture

<j-k-web>well I'm still at the mes part trying to work up to tcc still. no make or gcc etc

<stikonas>j-k-web: oh, sorry, that was river's post

<stikonas>I suspect most of the complexity is not in what we output, i.e. webassembly or C

<stikonas>but in implementing features of the language that is compiled

<stikonas>e.g. only a few percent of M2-Planet's code is target specific

<stikonas>though perhaps webassembly can be compiled or interpreted with less RAM than huge C files

<msavoritias>that plus webassembly is probably easier to work with(?) at least it should be

<stikonas[m]>Unlikely to be much easier...

<stikonas[m]>C is well known anyway

<stikonas[m]>So it's not hard to spit C code

<msavoritias>hmm. makes sense

<stikonas[m]>Web assembly is probably easier to parse than C

<stikonas[m]>But we alrwady have GCC

<stikonas[m]>If we consider rust bootstrap

<stikonas[m]>Adding something like web assembly to stage0 might make some simplification though

<stikonas[m]>Maybe it can be smaller than cc_x86

<msavoritias>personally i see myself moving to webassembly for cross compiling stuff. and a low level vm

<msavoritias>that basically C is right now

<msavoritias>mainly due to the smaller amount of overhead writing it.

<msavoritias>at least I have been discouraged for learning C because of that ^^'

<msavoritias>plus i probably dont even have to use gcc at that point

<oriansj>river: yeah, that is the we are just distributing a binary and you need to use that if you want to build our crap.

<oriansj>but supporting the generation of web assembly output is reasonable as it is effectively portable object code

<oriansj>and the support of running web assembly also makes sense as it is just supporting a standard object code format.

<oriansj>honestly there isn't much of a difference between web assembly and IBM's TIMI in regards to just being a portable binary format

<doras>fossy, stikonas: do you think the description of the minimal seeds in README.md is no longer representative given the recent kernel work? Is it possible to switch to sysb and continue the bootstrap without kexec support? I'd like to mention the minimal seeds in my presentation and would prefer to share the up-to-date information.

<doras>Actually, it's README.rst :)

<stikonas[m]>doras: reboot might be possible bug needs other work

<stikonas[m]>I.e. we need to mount hard disk

<doras>Ah, I see. So we can't actually continue to sysb for now.

<stikonas[m]>Well, not yet

<stikonas[m]>But if you have time, you could say that we are only kexec away from full bootstrap of use space and kernel

<stikonas[m]>And for now we start with 4k builder-hex0

<stikonas[m]>It might be possible to add "kexec" to builder-hex0-mini

<stikonas[m]>Which fits in 1 sector...

<doras>stikonas: so post-kexec effort you'd expect to have ~4KB seed with a Linux kernel no longer required?

<stikonas[m]>Yes

<doras>Nice :)

<stikonas>doras: keep in mind that builder-hex0 basically have something like hex0 and kaem built-in

<stikonas>so you don't need to double count those

<stikonas>at least that's my recollection

<stikonas>but I can quickly check

<stikonas>yes, it has hex0 command https://github.com/ironmeld/builder-hex0/blob/main/builder-hex0.hex2#L62

<stikonas>(this is hex2 prototype of builder-hex0, so a bit easier to read)

<stikonas>doras: and these are command that builder-hex0 runs on boot https://github.com/fosslinux/live-bootstrap/blob/b0a2747ada52a6f2486d202e15f9f4db01f5caff/sysa.py#L180

<stikonas>so builder hex0 from stage0-posix

<stikonas>then builds kaem

<stikonas>and kicks it off

<stikonas>so strictly speaking hex0 and kaem-optional are not seeds in kernel bootstrap

<doras>Oh.

<doras>Well, I guess. If they are just embedded into builder-hex0, one can claim that builder-hex0 is the only seed.

<stikonas>well, some simpler versions are embedded

<stikonas>rickmasters: speaking of builder-hex0 mini, did you have any ideas how it can boot the next step?

<stikonas>I wonder if we can somehow chain something like: builder-hex0-mini->builder-hex1-mini->builder-hex2-mini and then just boot into hex2 version of your kernel (builder-hex0.hex2)

<stikonas>(perhaps if this is possible builder-hex0 wouldn't be the right name...)

<rickmasters>stikonas: builder-hex0-mini just needs to write to memory and jump, it becomes a boot loader / hex0 compiler.

<rickmasters>stikonas: the name builder-hex0 refers to the fact that it builds with a built-in hex0 compiler.

<stikonas>I see

<stikonas>so presumably we can write something that has hex1 built-in

<stikonas>which might be smaller amount of hex0 code than full builder-hex0...

<stikonas>though given that we already have builder-hex0.hex0, maybe it's not worth it

<rickmasters>If it can compile hex0 and launch executables, stage0-posix can provide everything else, so it would be redundant.

<rickmasters>So anyway, if we have bootloader-hex0 compile and jump to builder-hex0 then maybe we can remove a bit of code from builder-hex0 but not a whole lot.

<rickmasters>(I'd probably create a new name "bootloader-hex0" if it has a different purpose)

<stikonas>well, I was just thinking that builder-hex0 implements quite a few syscalls

<stikonas>and we don't need that many for the first steps

<stikonas>but anyway, that's something for future, we don't even have fiwix->linux kexec yet

<rickmasters>stikonas: I do think there is merit to transitioning to a higher-level kernel earlier.

<stikonas>no, I agree with that

<stikonas>I don't think that we should run a lot of stuff with builder-hex0-mini

<stikonas>but I was thinking whether it makes to have another medium-level kernel...

<stikonas>anyway, it's up to you as you would have more insight on what makes sense there

<rickmasters>stikonas: Yeah, just doing a boot loader and hex0 compiler is a challenge in one sector.

<stikonas>yeah, I know...

<stikonas>let me see how big is hex0 compiler in stage0-posix

<stikonas>well, that one is 172 bytes, but you might not be able to have it so compact in baremetal environment

<stikonas>as you might need to use some registers for other purposes, etc...

<rickmasters>stikonas: builder-hex0-mini is only 384 bytes total and the hex0 compiler part is 96 bytes I think.

<rickmasters>stikonas: Anyway, a much smaller bootloader seed is definitely possible.

<rickmasters>I mean much smaller than builder-hex0 and probably smaller than builder-hex0-mini.

<stikonas>yes, I think so

<stikonas>even 4K is probably smaller than I thought possible

<stikonas>(for kernel that can run POSIX binaries)

<rickmasters>Even so, its a lot of hex0. We could probably strip it down to only run hex2 and a kexec like function to another kernel in hex2.

<stikonas>yes, that was exactly my question above

<stikonas>(though maybe I was not super clear)

<stikonas>and maybe we can do everything in just builder-hex0-mini, without any intermediate kernel

<rickmasters>No I understood. There were other questions too. :) yes, its something I've thought about.

<rickmasters>mini?

<stikonas>well, this thing https://github.com/ironmeld/builder-hex0/blob/main/builder-hex0-mini.hex0

<rickmasters>Right, that currently runs in 16-bit mode and writes to disk then reboots. So its enough to build and reboot to builder-hex0.

<rickmasters>I'm not sure what you mean "do everything in just builder-hex0-mini" ?

<stikonas>well, somehow build hex2 in it and run it

<stikonas>or like I said before another option is build an intermediate kernel that does not have to fit in 1 sector, is much smaller than 4K but can run hex1 or hex2 inside it

<rickmasters>I'm thinking start with the tiniest hex0 compiler with kexec to satisfy the tiny seed idea and building up from there.

<rickmasters>That's pretty easy to do - just some modest changes to builder-hex0-mini.

<rickmasters>Then the next kernel can run hex2 or exec hex2 - I'm not sure.

<stikonas>yeah, something like that

<stikonas>(exact details to be determined later)

<rickmasters>Yes, there are options for later. I initially went with exec hex2 because I was focused on writing a kernel to support stage0-posix rather than try to replace any of it.

<stikonas>oriansj, rickmasters: how about this idea for "hex1.5". We can have two hex character labels that are encoded into byte

<stikonas>that should still be easy enough to do without any of those hex2 lookup tables

<stikonas>and 256 labels are enough for builder-hex0

<stikonas>right now hex1 can support maybe 90 labels...

<rickmasters>Sounds fine to me

<oriansj>stikonas: actually hex1 supports 256 distinct labels as anything after : is the label offset in the label table

<stikonas>ok, so theoretically enough to build builder-hex0 in hex1 form

<stikonas>though we would have to use non-printable characters

IRC channel logs

2023-04-15.log