<Profpatsch>I’m trying to compile mes on an x86-64 machine, and gcc doesn’t like the assembler <Profpatsch>/nix/store/1wb31gzrjw055xl6d3igzj1gnajzlwbk-gcc-wrapper-9.2.0/bin/gcc -c -D HAVE_CONFIG_H=1 -I ../lib -I include -I ../include -I ../include/linux/x86_64 -static -nostdinc -nostdlib -fno-builtin -static -g -o mini.o ../lib/linux/x86_64-mes-gcc/mini.c <Profpatsch>../lib/linux/x86_64-mes-gcc/mini.c: Assembler messages: <Profpatsch>../lib/linux/x86_64-mes-gcc/mini.c:31: Error: unsupported instruction `mov' <Profpatsch>and : "rm" ((uint64_t)filedes), "rm" (buffer), "rm" (size) <Profpatsch>whoo, it compiles mescc and it’s immediately segfaulting. Maybe I should turn off all hardening flags for now. <Profpatsch>mes-0.22/gcc-lib]$ ../pre-inst-env mescc -m 64 -c -D HAVE_CONFIG_H=1 -I ../lib -I include -I ../include -I ../include/linux/x86_64 -static -nostdinc -nostdlib -fno-builtin -o crt1.o ../lib/linux/x86_64-mes-mescc/crt1.c <Profpatsch>builtins.h config.h constants.h lib.h lib-mini.h macros.h mes.h <Profpatsch>janneke: ^ If you have any pointers, that would be great <Profpatsch>I’m assuming the header that is not found make a function return #f and that is passed down to expr->register, which crashes on the pattern match <Profpatsch>hmhm, it doesn’t look like mescc is even trying to look at the include paths: <Profpatsch>[pid 13687] access("./mes/lib-mini.h", F_OK) = -1 ENOENT (No such file or directory) <Profpatsch>[pid 13687] access("./mes/lib-mini.h", F_OK) = -1 ENOENT (No such file or directory) ***stikonas_ is now known as stikonas
<janneke>Profpatsch: that's a lot of hacking already! <janneke>Profpatsch: xwvvvxw has been looking at packaging mes for nix; that could be a helpful pointer? <janneke>that mes/lib-mini.h include error above looks strange <Profpatsch>Ah, I’m just trying to build it with a normal gcc toolchain at the moment, so all default configure/make/install <Profpatsch>And maybe later make sure it’s really bootstrapped from a minimal distribution. <Profpatsch>janneke: where is the include logic defined? Somewhere in mescc.scm? <janneke>Yes, first a regular build would be best thing to do <janneke>yes, in module/mescc/mescc.scm and module/mescc/preprocess.scm <janneke>Profpatsch: this: "mes-0.22/gcc-lib]$ ../pre-inst-env mescc" is not going to work <janneke>/gnu/store/s5ljhv95c2zka3ry6qnyjq349g7xhlw8-profile/bin/gcc -c -D HAVE_CONFIG_H=1 -I ../lib -I include -I ../include -I ../include/linux/x86_64 -static -nostdinc -nostdlib -fno-builtin -static -g -o crt1.o ../lib/linux/x86_64-mes-gcc/crt1.c <janneke>/gnu/store/s5ljhv95c2zka3ry6qnyjq349g7xhlw8-profile/bin/gcc -c -D HAVE_CONFIG_H=1 -I ../lib -I include -I ../include -I ../include/linux/x86_64 -static -nostdinc -nostdlib -fno-builtin -static -g -o crt1.o ../lib/linux/x86_64-mes-gcc/crt1.c <Profpatsch>I just ran the default `make`, so it plugged in the wrong compiler and it should be compiled by gcc instead of mescc? *Profpatsch hasn’t actually looked at the boostrap chain yet, was just hoping for the make to work magically <Profpatsch>The nix configure phase does some magic setup, I’ll try stripping that out. <Profpatsch>Ah, the configure.sh sets paths depending on cwd, that might not be what it expects. <janneke>you need guile, but configure.sh is strictly for a full bootstrap *janneke tries ./configure.sh for fun <janneke>Hmm, ./configure.sh does not work for me in a regular build; looks real silly, though: @CPPFLAGS@ does not get substituted :-( <Profpatsch>I’ll try with ./configure for now and maybe later I can remove the dependency on make. <Profpatsch>But mescc-tools already depends on configure/make, so I don’t know how that is intended to look. <Profpatsch>guile depends on GCC already, so that’s a rather big dependency, I haven’t yet understood that part of the bootstrap <janneke>Profpatsch: yes, we have been applying some cheats that we intend to remove later <janneke>currently, mescc-tools, mes and guile are added to the guix bootstrap binaries <janneke>and: gcc, binutils, glibc have been removed <Profpatsch>But how would you break the cycle? Or does it make a difference from a theoretical point that the binary is guile compiled with gcc rather than gcc compiled with gcc? <Profpatsch>Or is it irrellevant because mes starts from a very small blob anyway? <janneke>a next, in-between step is to remove the mescc-tools and mes binaries from the bootstrap <janneke>that needs some work, but they can be bootstrapped from stage0's hex0 <Profpatsch>That’s kind of a big one, because e.g. byacc and a C parser. <janneke>yes...of course, in guix we use guile as a driver; so i would like to avoid counting guile in that role (for now) <janneke>we injected a dependency on guile that we intend to remove by using Mes instead of guile <Profpatsch>I’m not partial to any runtime, nix itself doesn’t care. <Profpatsch>But mes itself is built with guile again, right? <janneke>no, we build mes using m2-planet then, really from source <Profpatsch>Ah, that’s where m2-planet comes in. That makes sense, thanks. <janneke>to reach that stage is still a couple of month's work <Profpatsch>So stage0 -> m2-planet -> mes -> tinycc -> {make|guile|gcc} -> guix <Profpatsch>and we can replace guix by nix and it should work just the same. <janneke>fossy here has been working to implement the "guix" bootstrap without guile, using shell scripts (or even stage0's kaem?) as a driver <Profpatsch>you could even use both package managers and compare results if you are very paranoid. <Profpatsch>I’d like bash to be replaced by something lighter, e.g. dash <janneke>so what we did, was to build a cross-distro bit-for-bit reproducible mes <OriansJ>also the stages in stage0 also can be done on bare metal to eliminate all other software from the TCB <janneke>we have been working to add "gash" a shell in guile <OriansJ>and yes fossy is using kaem as it is the most minimal shell possible and was written in 737bytes <Profpatsch>So you would bootstrap it without a kernel, just on a plain CPU with a chunk of memory? <OriansJ>Profpatsch: everything in stage0 already has a bare metal version <Profpatsch>Now I’m thinking of a caste of priests, bringing the source code to the processor altar to produce the holy reproducible reference binary :) <OriansJ>using an ancient CPU architecture, which removes the need for firmware, microcode or even BIOS <Profpatsch>“From which the world was bootstrapped and the singularity can reboot” <OriansJ>and every port of stage0 can cross-verify all the stages for all of the other ports as well using mescc-tools <OriansJ>dddddd: I think we forgot a minor piece in our M3 plan <OriansJ>janneke: M3-Star M3-Comet (for converting TCC into something we can directly compile with M2-Planet) but we probably need to add a M3-Meteoroid to provide a drop in replacement for ld as M3-Comet is supposed to be a drop in replacement for as and M3-Star is supposed to be TCC converted to only output assembly to simplify future porting work. <OriansJ>That way we can use the standard binutils ld and as during development to save time chasing bugs of our own making <OriansJ>plus when we get it solid, it provides MesCC a much better assembly language to work with <OriansJ>with 2 different possible back-ends for generation depending upon if bootstrapping or just standard building. <janneke>Oh, you are adopting/forking TCC, really? <janneke>i really like to have more than one bootstrap path anyway; we need to figure out what works <janneke>and there are various perspectives to "works" <OriansJ>mostly, I need a break from trying to figure out how to write syntax-case in C for mes-m2 <janneke>syntax-case in C .. that would be quite a feat <OriansJ>janneke: well it just isn't possible to implement it in scheme without it <OriansJ>I might be a terrible programmer but I just can't find a different solution to that problem. (Aside from asking scheme programmers to give up macros) <janneke>I have only a mostly (cough) complient syntax-rules using define macro <janneke>and I spent a couple of weeks trying to bootstrap Adre van Tonder's syntax-case, which presumably should run on a correct syntax-rules <janneke>i could not get that to work, and i don't dare to think about performance <OriansJ>now simple macros that one would do in define-macro <OriansJ>which mes-m2 has but it might not even be possible, which raises the question of how the heck did they manage to bootstrap it in the first place <janneke>yes...possibly using Andre van Tonder's all source code way once and burn all bridges after that? <OriansJ>which would be a great way to hide a backdoor <OriansJ>a big complex block that is virtually impossible to understand and about as hard to implement <OriansJ>I'm just glad information about ELF relocatable object files is readily available and most of what is needed to be known were covered when we did the hex2 ELF headers <OriansJ>I however thought it was weird that ELF never supported a .source section it seems like an obvious GNU extension <OriansJ>now mind you it would be optional and the resulting binaries would be inflated by the size of the source code required to build them but it would solve the binary/source distribution issue forever <fossy>guix is great and all, janneke has done super good work on the upper levels of the bootstrap <fossy>guix is the best distribution for bootstrapping <fossy>but there are some things a full fledged distribution cannot do. <fossy>ie, everything from the one seed <fossy>gcc-seed will be a scripted build of the full gcc/binutils/glibc triplet from the 357 byte seed <fossy>with auxiliary tools to go from.there <dddddd>OriansJ, noted. I'll add it to my notes-to-digest-the-idea. <dddddd>janneke, you can read about the M3 very early plan in the log of the 14th. <janneke>truly bootstrapping from 357 bytes is what we set out to do <vagrantc>would you be satisfied with 358 byte bootstrap? <vagrantc>i haven't been following closely ... what's the constellation of bootstrappiong looking like right now? <fossy>if its readable and adds value sure <vagrantc>would like to inject a few slides about the state of things in a reproducible builds talk i'm giving <fossy>that is, between M2-Planet and tinycc <fossy>two paths: mes-m2 (scheme) or m3-* (modified tinycc) <vagrantc>i know where "gnu mes" is but i'm a little shaky on the lower level stuff <fossy>guix is nearly at just guile for the bootstrap <fossy>kaem has been expanded and can run fully scripted builds up to bash (not configure scripts tho) <vagrantc>think i've looked at that map before ... though maybe a smaller number of red lines :) <vagrantc>the arrows seem backwards ... but i guess it's a dependency graph rather than the origin of the bootstrap graph <janneke>vagrantc: 358 bytes, that's out of the question! *vagrantc thanks janneke for playing into the trap <janneke>well, vagrantc we have to draw the line somewhere! <Profpatsch>if it’s arbitrary, at least make it arbitrary arbitrary <janneke>fossy: i guess these big distributions will have to play catch-up rsn ;-) <Profpatsch>People are asking: what about reflections on trusting trust, why not just compile a gcc twice from binary blob and compare the outputs? <janneke>Profpatsch: it all starts with asking questions... <fossy>Profpatsch: where did the binary blob come from? <vagrantc>and we've kind of sort of not-quite-right done that with GNU mes <vagrantc>actually, i suspect GNU Mes might fill the role described in david a wheeler's DDC papers <Profpatsch>But what does bootstrappable give that DDC does not? <vagrantc>you almost can't have one without the other <vagrantc>or at least, they're both stronger if you have both, obviously <Profpatsch>> The method requires a different compiler and the source code of the compiler-under-test. <vagrantc>all of them add confidence to each other <vagrantc>you can always go a little lower down the stack for even greater levels of confidence, auditability, etc.