IRC channel logs

<genr8eofl_>sam_: do you read what goes on in this channel

<genr8eofl_>i will return to my normal hideout

<fossy>oh i didn't realise free was not noop now that's neat

<fossy>rickmasters: i was also looking at Limine before - it does look like much nicer code than GRUB!

<fossy>grub's build system and codegen is horrible, i don'

<fossy>t really think it's feasible for use in live-bootstrap without significant work, so having linux kexec in fiwix makes a lot more sense, imo

<fossy>(build system uses autogen templates generated by python LOL) (and there's extensive python codegen)

<rickmasters>Here is a script that uses Limine to manually complete sysb/sysc after running the current partial kernel bootstrap:

<rickmasters> https://gist.github.com/rick-masters/8a68efcb9cb7c8797dcf85d861802c3f

<pabs3> https://www.joachim-breitner.de/blog/802-More_thoughts_on_a_bootstrappable_GHC

<rickmasters>rickmasters: looks like I forgot `mkdir mnt` so do that before running the script

<rickmasters>oops, talking to myself now

<fossy>nice!

<Irvise_>Just saw the new bootstrap blog regarding Guix, they have really improved a lot throought the years :D

<janneke>stage0 -> m2-planet -> gnu mes has landed on guix master \o/

<janneke>=> https://guix.gnu.org/en/blog/2023/the-full-source-bootstrap-building-from-source-all-the-way-down/

<civodul>🎉

<next4th>congrats and thank you for bootstrap guix!

<river>that is incredible!!!!

<stikonas[m]>janneke: we should switch to 256 byte bootstrap :)

<stikonas[m]>Though that is not yet released...

<stikonas[m]>oriansj: maybe make a release when you have some time?

<janneke>stikonas[m]: something to look forward to

<fgaz>congrats!

<janneke>it's always nice to have some good news in stock ;)

<theruran>janneke: I am wondering, why is the bootstrap-seeds not at the bottom, and guile-bootstrap is instead?

<janneke>theruran: bootstrap-seeds would be at the bottom, but in guix it must be untarred which is done by bootar, which is written in guile

<janneke>so if you would ignore the unpacking, it would also be at the bottom

<theruran>ohh got it :+1

<theruran>great work!

<janneke>thanks, yeah possibly that dependency should be kept out of the graph, dunno

<stikonas[m]>theruran: that's the bootstrap driver... You could use a smaller one, e.g kaem-optional but that's standalone solution. Might not be the best option for bistro

<stikonas[m]>Where users expect some integration with package manager

<stikonas[m]>And plus kernel is also necessary but it's slightly more hidden than guile or kaem

<janneke>+1

<janneke>(or hardware, even)

<theruran>does that mean there's a circular dependency not depicted? that guile-bootstrap is still DDC, but it just deployed as a binary for convenience

<theruran>"If we pull this off, guile-bootstrap will only be used as a dependency of bootar and as the driver for Guix."

<janneke>have we heard of DuskOS?

<janneke> https://git.sr.ht/~vdupras/duskos

<stikonas[m]>theruran: yes, but like I said that's unavoidable in package manager

<stikonas[m]>Unless you want to use kaem-optional-seed as package manager

<stikonas[m]>And bootstrap your package manager on each run...

<stikonas[m]>But in principle the whole process can run without guile

<stikonas[m]>Which we have done

<stikonas[m]>(Just not in guix)

<theruran>OK I think I understand except the "bootstrap your package manager on each run." I thought it would only need to be done once, upon install of Guix?

<rickmasters>janneke: I've seen DuskOS. It looks cool. I'm not indoctrinated into Forth but it seems to be a favorite of many do-it-yourselfers.

<janneke>yeah, i've heard a lot of talk about how bootstrapping would be "almost trivial" with forth, but haven't seen anything before

<janneke>*anything real before

<rickmasters>janneke: Ultimately its a POSIX world out there so he'll need to get to tcc/Fiwix to meet up with a reasonable path, IMHO.

<janneke>right

<janneke>of coourse, DuskOS addresses stuff below our bootstrap that we ignore currently, but we have a full gnu/linux distro

<rickmasters>Could you elaborate? Yeah I'm a fan of many approaches we can all learn from. Maybe DuskOS can do things better at some layer.

<janneke>oh, i just read the readme; but they go from bare hardware and a small binary to an "almost c" compiler

<rickmasters>Well the new kernel bootstrap does that too. He says DuskOS is smaller though.

<rickmasters>The kernel bootstrap starts with a 3.5K POSIX kernel binary but with some work that could be compiled with an approx 384 byte bare metal binary.

<vagrantc>that sounds suspiciously similar in size to hex0

<rickmasters>vagrantc: It has an equivalent hex0 compiler as stage0's but it is an entirely independent implementation.

<rickmasters> https://github.com/ironmeld/builder-hex0/blob/main/builder-hex0-mini.hex0

<vagrantc>it just was interesting to see a similar size ... sort of like a constant of the universe or something :)

<stikonas[m]>vagrantc: hex0 is a bit smaller...

<stikonas[m]>We have hex0 that is 256 bytes including elf header

<vagrantc>oh, wow.

<rickmasters>oriansj also mentions bootOS as possible starting layer: https://github.com/nanochess/bootOS

<stikonas[m]>This baremetal stuff has no header, so can pack more

<stikonas[m]>But elf header is mostly data, not code

<vagrantc>in the grand scheme of things, that's getting to be on the same order of magnitude of smallness :)

<stikonas[m]>Yeah, anything sub 1K is OK...

<vagrantc>thought i saw 357 bytes being mentioned for hex0 in several places

<stikonas[m]>Well, hex1 is still written in hex0 and that is also less than 1K

<stikonas[m]>vagrantc: 357 was before I made it smaller

<vagrantc>:)

<rickmasters>re: small ELF programs there are unholy hacks that can be done: https://www.muppetlabs.com/~breadbox/software/tiny/teensy.html

<janneke>guix uses an already ancient stage0 and m2-planet

<stikonas[m]>E.g. by using stuff like cmp al, 8 bit number instead of cmp eax, 32 bit

<stikonas[m]>256 byte stuff is only from git...

<stikonas[m]>It has been a year since the last release...

<janneke>yeah, iwbn to have a consistent set of releases, not easy

<vagrantc>on the other hand, good progress is nice too :)

<stikonas[m]>This is new unreleased stuff: https://github.com/oriansj/stage0-posix/blob/master/CHANGELOG.org

<genr8eofl>janneke: nice blog and congrats on guix bootstrap

<janneke>thanks!

<janneke>stikonas[m]: very nice

<fitzsim>yes, great milestone, congratulations!

<fitzsim>have to ask; is a ppc64 bootstrap planned?

<stikonas>fitzsim: we don't have anybody doing it

<stikonas>but feel free to start it

<stikonas>stage0-posix at the moment is x86/amd64/riscv32/riscv64/aarch64

<stikonas>and aarch64 is in slightly worse state than other 4

<stikonas>well, there is also knight architecture but there is no hardware for it

<stikonas>or any POSIX kernel for knight...

<stikonas>and then of course there are higher layers like mes and tcc...

<fitzsim>cloning stage0-posix...

<fitzsim>does gcc 2.95 have ppc64 support?

<janneke>no, gcc 2.95 is 32bit

<fitzsim>ah, ok

<stikonas>fitzsim: gcc 2.95 is not really necessary

<stikonas>tcc can build at least gcc 4.0.4 (probably newer) and binutils 2.30 (current stable tcc can't go any newer but 2.30 is reasonably modern)

<fitzsim>oh cool, hadn't heard that

<stikonas>fitzsim: that's what live-bootstrap builds...

<stikonas> https://github.com/fosslinux/live-bootstrap/blob/master/parts.rst

<stikonas>and gcc 4.0.4 is from the times when we only had binutils 2.14

<stikonas>since then we upgraded to binutils 2.30

<stikonas>gcc 4 is not that old, I think only riscv support is missing but ekaitz backported it to gcc 4.6

<ekaitz>i did but I can't say it's perfect

<ekaitz>it did compile a really small file

<ekaitz>no linking or anything like that

<ekaitz>and C only

<stikonas[m]>Yeah, OK... Still there is some progress

<stikonas>fitzsim: there are a couple of ways of starting a new arch, it's not clear what is better, you could start with C tools and add ppc support there, or you could start with early hex0 steps

<stikonas>although I guess the very first steps would be to figure out how to write elf header

<fitzsim>hmm, yeah

<stikonas>i.e. all the stuff until ELF_text in https://github.com/oriansj/bootstrap-seeds/blob/master/POSIX/x86/hex0_x86.hex0

<stikonas>I'm not very familiar with ppc, but then one has to think how to go from opcodes to hex (or octal or binary...)

<fitzsim>yeah, just looking at the riscv64 version of that file

<stikonas>so e.g. riscv64 is a fixed word size architecture

<fitzsim>how far up the stack must one go before ppc64le is already there?

<fitzsim>e.g., cc_ppc64le would need to be written from scratch, I guess

<stikonas>cc_ppc64le definitely needs to be written

<fitzsim>and M2libc routines

<stikonas>and after than you need to add support to M2-Planet/M2libc

<stikonas>but those are smaller bits

<stikonas>only a few % of M2-Planet is arch specific

<stikonas>fitzsim: so e.g. on risc-v you can see M1 defines are like RS1_SP RS2_A0 SB and then each of these is converted to somethign like .somehex .somehex2 opcodehex

<stikonas>and . means add it to the next thing

<stikonas>on the other hand on x86 the whole thing is one define

<stikonas>e.g. mov_edi,eax

<stikonas>there is no separate mov or register

<stikonas>so stuff like that is arch specific

<stikonas>and one needs to figure out the easiest way of working

<stikonas>fitzsim: it looks like ppc is fixed word size architecture

<stikonas>so you probably want risc-v method

<stikonas>which is more tedious in early steps

<stikonas>though is nicer later

<stikonas>(by the time cc_ppc64le has to be written)

<fitzsim>it looks like lots of work has been done on kernel minimization since I last looked at this

<stikonas>fitzsim: and we also have stage0-uefi (though nothing after that)

<stikonas>fossy: any reason why we are using dhcpcd and not dhcp?

<stikonas>maybe that one has more stable tarballs

<stikonas>fitzsim: see https://git.stikonas.eu/andrius/stage0-uefi this runs directly on UEFI

<fitzsim>neat!

<nektro>janneke, and everyone, congrats 🎉

<fitzsim>I guess eventually there will be a libreboot minimization effort?

<stikonas>fitzsim: libreboot is in a bit of a sad state, it does not run on any new hardware...

<stikonas>I guess eventually all old hardware will break

<stikonas>not sure what will happen to libreboot then

<stikonas>you might have more luck with non x86 arches

<fitzsim>sure, but for bootstrapping purposes old hardware can play a role

<stikonas>if you can find it...

<stikonas>e.g. it's hard to dinf PDP-8 these days...

<fitzsim>haha

<stikonas>some arm or risc-v boards don't have any non-free firmware (only bootrom)

<stikonas>but something like builder-hex0 would be a few times bigger there as we would need to write some drivers (I/O, memory, etc...)

<fitzsim>yeah; one idea is to synthesize a Microwatt-based SoC and start from there

<fitzsim>with UEFI, I guess the idea is that someone could fully bootstrap their "daily driver"

<fitzsim>which would be really cool

<muurkha>janneke: congratulations! bootstrapping guix from a 357-byte program is a truly magnificent achievement

<mihi>janneke, I do know DuskOS (and also CollapseOS by the same author) and incidentally I made my first contribution to it yesterday :)

<mihi>rickmasters, "Ultimately its a POSIX world out there": Yes, I think it is sad that today to have an OS as a daily driver it needs to have a "modern web browser" (more complex than many whole niche OSes) and that to speak any network protocol (except gopher and maybe irc) you need a TLS implementation, which pulls in all the X.509, ASN.1, XKCD.2347 stuff. Yet I wonder whether SerenityOS could have

<mihi>been successful as well if it wasn't inspired by POSIX architecture...

<mihi>That TLS requirement effectively forces the small community that still want to use Native Oberon as their daily driver to either tunnel all their traffic through stunnel on another machine, resort to using gopher, or start using a different OS for web activity...

<muurkha>it's not successful yet

<muurkha>you could imple

<muurkha>you could implement TLS in Oberon

<muurkha>sorry

<mihi>muurkha, as "successful" as it is today, I mean. From what it started from, its success is surprisingly large.

<mihi>muurkha, "you could implement NDISWRAPPER on Linux/Hurd/BSD". Would you want that? :D

<mihi>from the Oberon simplicity standpoint, X.509 is part of the problem :D

<muurkha>yes, I think ndiswrapper was a good idea

<muurkha>yes, I agree

<muurkha>maybe you could start with tweetnacl

<muurkha>BER isn't nearly as bad as it looks at first

<mihi>RSA, AES, Diffie-Hellman exist in Oberon already. So you can encrypt stuff :)

<mihi>but I'm digressing. To get back to topic, Native Oberon is not boostrappable from itself (there is a cross compilation path via obnc, but to go that route you first need to bootstrap a POSIX system...)

<mihi>anyway, congrats to janneke and all the guix folks for getting the Full Source Bootstrap merged :)

<muurkha>A bootstrap pa

<muurkha>th for Oberon from a tiny seed would be pretty great

<muurkha>And could plausibly be much shorter than a C system

<muurkha>well, a POSIX system

<muurkha>and it might be an interesting alternative to Fiwix

<dongcarl>civodul, janneke, oriansj: Congrats to all for completing this monumental undertaking. Bootstrappable builds for all!

<muurkha>I found this post about sourciness interesting in this context: http://inamidst.com/whits/2014/source

<muurkha>hmm, actually "next steps" says that a 25MB Guile executable is still part of the binary seeds needed

<muurkha>I thought we had Guile's bootstrapping problem licked?

<mihi>muurkha, that Guile binary is used by Guix to retrieve, unpack, etc. the source files and to orchestrate the bootstrap. It is needed the same way as a git client and python are needed to run live-bootstrap. That Guile executable will not affect any file of the resulting system.

<stikonas>muurkha: these are different problems

<stikonas>mihi: that's only partially true

<stikonas>guile is used to run some scripts inside bootstrap in guix

<stikonas>in particular gash is used that does not yet run on mes

<mihi>ok, but those are not necessarrily "needed" (both live bootstrap and my own bootstrap-from-git script demonstrate that these scripts run well enough with coreutils/bash as well)

<stikonas>but guile usage inside bootstrap is mostly limited to running shell and unpacking stuff with bootar

<stikonas>mihi: indeed

IRC channel logs

2023-04-26.log