IRC channel logs
2021-06-19.log
back to list of logs
<oriansj>clemens3: well stage0 comes in two pieces; stage0-posix which by definition depends upon a posix kernel (a rather minimal one at that) and stage0; which doesn't depend upon any kernel and runs on bare metal. The big problem is once one gets to mes-m2 (MesCC) one needs a POSIX kernel to extend further into the bootstrap process. So any kernel one would want to write in C and bootstrap from bare metal would have to be in the M2-Planet <oriansj>that being said, this is all a big work in progress and despite 5 years of effort, not even close to being *DONE* <oriansj>ripping out the kernel for x86 and other platforms being bootstrapped. Porting stage0-posix to more architectures. Building custom fpgas to minimize possiblity of hardware compromises. Building a custom computer out of individual gates to finish off the entire chain. There is a mountain of work to do but all of it fun. <oriansj>where we are now is if you have a posix kernel you trust that supports: exec, fork, waitpid, read, write, exit and a file system. (virtual memory would be a nice extra) you can use it to bootstrap to the end of stage0-posix. After that things get more complicated. <oriansj>(chmod, mkdir, open, close possibly too depending on the filesystem used) <clemens3>well, the hardware security is another whole and can of worms of its own.. just the software stack would be a great progress <oriansj>clemens3: well the software stack has already been reduced to something that could be solved in less than a year by a single programmer. Just build a POSIX kernel in assembly that supports ext2 and just enough syscalls to bootstrap TCC, then use it to build Linux and be done. <oriansj>It doesn't have to be portable, just barely good enough to do the job and small enough to manually audit. <oriansj>stikonas: well fat16 might be simpler but ultimately that would be an implementation detail done by the implementer. <stikonas>well, what I meant do we even need filesystem? <stikonas>well, there is a question of how we feed sources in... <oriansj>stikonas: ultimately if the binaries don't get written to some physical storage media eventually. Then there was no point in doing the computation. <stikonas>but it might be even simpler than fat16... <stikonas>that's true, but you can probably build ext2 driver later <stikonas>and then use that to write binaries somewhere <oriansj>stikonas: a RAM file system is still a file system <oriansj>The only savings is in regard to writing blocks to a Logical Block Address and loading from a Logical Block Address, which could fit in a couple simple functions. <oriansj>if you look at BootOS, it does disk read and write along with everything else it does in 512 bytes <oriansj>So it isn't some impossible task, just one that gets complicated the more disk drive controllers you wish to support. <oriansj>now mind you that BootOS has the advantage of being in real-mode and making bios calls to keep the complexity down to the floor. <oriansj>But remember all engineering is the balancing of needs and building within acceptable tolerances. <stikonas>oriansj: so I am back at getting mes-m2 from stage0-posix to work. So we discussed one problem before (included nyacc version is wrong). I think the other error I'm getting iserror:mescc: file not found: "linux/x86-mes/elf32-footer-single-main.hex2":() <stikonas>(that's when linking tcc, so hopefully the last issue) <stikonas>oriansj: I can't find that file anywhere in the stage0-posix/mes-m2 <stikonas>there is another file though ./lib/m2/x86/ELF-i386-debug.hex2 <stikonas>yaeh, it's only to use mescc to link other binaries <stikonas>I'm trying to get it to work but somehow need more debugging (even after I put the file there, it still can't find it, probably looking in the wrong path) <oriansj>it is a hex2 ELF header if I remember correctly. <stikonas>ok, I'll deal with it later, since it's more or less clear now, what to build tcc first. So third problem is <stikonas>/after/tcc-0.9.26/src/tcc-0.9.26 # ${MES} --no-auto-compile -e main ${bindir}/mescc.scm -- -o mes-tcc -L ${libdir} tcc.s -l c+tcc <stikonas>mescc: failed: hex2 --LittleEndian --architecture x86 --BaseAddress 0x1000000 -f /after/lib/linux/x86-mes/elf32-header.hex2 -f /after/lib/x86-mes/crt1.o -f tcc.o -f /after/lib/x86-mes/libc+tcc.a -f /after/lib/x86-mes/libmescc.a -f /after/lib/x86-mes/libc.a -f /after/lib/linux/x86-mes/elf32-footer-single-main.hex2 --exec_enable -o mes-tcc <oriansj>stikonas: well that is a hex2 linking error <oriansj>which in this case means :qsort is not in any of those files being linked together <stikonas>oh, that's probalby because I removed qsort during previous debugging <stikonas>anyway, I'll rerun everything from scratch <oriansj>So even a stub function will produce that label <stikonas>by the way, do you want to keep nyacc inside mes-m2? <stikonas>we need to either fix embedded copy or just use release tarball <stikonas>even mes-m2 in principle can be used from tarbal now <oriansj>well the idea was to make new developers lives easier when they choose to start hacking on MesCC as the previous method required guix to setup a build environment before you could get it to run <oriansj>but you are right, perhaps it is time to move mes-m2 out of stage0-posix and just tarball it in live-bootstrap. <oriansj>it is very touchy and can't do cross-platform builds like the rest of stage0-posix <stikonas>yeah, mes/mescc is very picky about environment <stikonas>and I had to precisely time Ctrl+C keys to see these error messages <oriansj>we could just abandon mes-m2 entirely as mes should be including the pieces needed to be built by M2-Planet and maintaining those as it developes <stikonas>otherwise the buffer gets filled in with useless backtrace <oriansj>stikonas: I find piping 2> log extremely helpful with MesCC <stikonas>yeah, hopefully in future can just build mes directly <oriansj>stikonas: well mes.c and mes-m2 are supposed to be clones with the only difference being mes-m2 was my attempt at making mes.c into a form M2-Planet could build and janneke was to backport into mes.c <oriansj>Turns out janneke solved the mes-m2 bootstrap problem first in mes.c; so at this point there isn't much reason to keep mes-m2 <stikonas>well, for now as far as I understand this is just a snapshot of that working version (with a few extra fixes) <oriansj>I can certainly backport the low level primitives that janneke provided for mes-m2 into M2libc <oriansj>stikonas: well it is more complex then that unfortunately. <oriansj>or buildable by GCC but MesCC doesn't work anymore <stikonas>oh I see. Well buildable for GCC is nice to have but not essential for bootsrapping purposes. But I guess might be blocker for official mes <oriansj>and there are only two paths forward: 1) janneke gets MesCC running on the GCC branch of mes-m2 and we finish the M2libc porting work for mes-m2 or 2) I sort out the differences betwen mes.c and mes-m2 to keep both working. <oriansj>neither will be fun for me or likely janneke <oriansj>as 1) requires MesCC to support FILE as a struct and 2) requires me to do functional isolation in a tightly wrapped scheme. <stikonas>ok, it builds now, I'll prepare mes-m2 changes then <stikonas>ok, I think I've got live-bootstrap to build with tarballed mes and nyacc. Although, that will need mkdir, so we'll need a new stage0-posix release <stikonas>oriansj: sha256sum.c prints "Wanted" and "Received" hashes when there is a hash mismatch and they seem swapped <stikonas>Wanted is checksum of the file and Received is the recorded checksum in the digest file <stikonas>is that deliberate or should I swap those in mescc-tools-extra? <clemens3>re,ok,just look at the mes and stage0 projects on the page.. question to me is how much old source/projects from say the 80s 90s can be rejused.. is like we try to reproduce the history, because the history got lost in the bloat.. <clemens3>now with emulators, can we go back in time..to meet somewhere from the bottom up mes/stage0 hex byte appraoch to the, well, we have been in the computing stone age already before, why can't we use it again.. <stikonas>clemens3: once you have tiny C compiler a lot can be reused <stikonas>although we tend to use stuff from early 2000-2005 maybe <clemens3>oki.. reading up a bit on the stage0 project.. <stikonas>if you look at live-bootstrap project that builds on top of stage0-posix and mes, it fairly quickly switches to modern C library <vagrantc>what's the first version of gcc in the bootstrap chain these days? <stikonas>vagrantc: in guix it's 2.95.3, in live-bootstrap its 4.0.4 <stikonas>new version is unlocked because of musl libc <stikonas>which is much more complete than mes libc <stikonas>vagrantc: yeah, gcc 4 is not bad indeed. although at the moment with a small caveat, it's C backend only <oriansj>stikonas: I think I did those backwards feel free to fix. <stikonas>or maybe right now before I go, it's a quick fix <oriansj>clemens3: well the thing about history, sometimes it is best to avoid repeating it. <stikonas>(draft because it will need update in stage0-posix modules) <oriansj>I'll try to udpate stage0-posix later today with your enhancements <stikonas>although, that is independent of stage0_posix <stikonas>that draft commit build mes-m2 completely from tarball <oriansj>stikonas: just noticed that and it is merged