IRC channel logs
2026-06-14.log
back to list of logs
<alganet>"shpack bootstraps from a few hundred bytes of binary seeds into a reproducible GCC 4.7 toolchain and utilities" <alganet>that unlocks a wide ass list of software that already depends on musl-cross-make into a supply chain cleaner than the typical usage of mcm <alganet>mcm itself is not up to the standards of live-bootstrap (regens and all), but it's an obvious and quick practical improvement that many people could adopt without friction <tobtoht>stabbles: wow! you weren't kidding. it runs in just under 2 minutes on my machine (in a VM). <aggi>alganet: a linux syscall abi is a tad bit more than a few hundred bytes, believe me <roconnor>If the worry about compilers is about Trusting Trust, there is no choice but to include the kernel. A Thompson virus could infect a kernel so that when it runs a compiler it augments the output to include itself, which in turn spreads to a kernel when it compiles a kernel. <roconnor>That said, I do respect that bootstrapping a kernel is not easy and it is a matter of building up the bootstrappable infrastructure. <alganet>you could trust someone you trust, also. Let's say 100 people do the kernel bootstrap, and all sha256sums agree. Then, any tool that reproduces those sums is good enough, as long as there is communal consensus on the checksums. <alganet>this can go further still. For example, if one is able to build the debian kernel from live-bootstrap full, it estabilishes that chain (although that is a hard thing to do, likely very difficult) <alganet>if you want to verify for yourself, then yes, you need to do it :D at that point probably you'd be worried about the firmware, disk controller, disk imaging software and so on <roconnor>Unfortunately running a sha256sums on a Thompson infected kernel is meaningless because of course the virus is smart enough to lie on the output when it knows you are running sha256sum program. <aggi>a complete system bootstrap including kernel nust must be distinguished, simply because it must be done one way or another <aggi>if any approach can't do it yet, it shouldn't claim so <alganet>We can take the concept as far as we want, as I mentioned with firmware and so on. It's nice, and I totally enjoy that side. <alganet>In practice though, there are a lot of systems that would benefit from a more pragmatical bootstrap today, and mitigate all sorts of risks <alganet>I see the infected compiler as a more pedagogical thought exercise that exemplifies issues with supply chain. <alganet>It has been reproduced, and that's also nice, but still, mostly pedagogical. <tobtoht>"Let's say 100 people do the (...) bootstrap, and all sha256sums agree." <- this is the typical approach for projects that need an auditable build process (e.g. bitcoin). <tobtoht>let well-known contributors run a "driven" bootstrappable build (using e.g. guix) and have them attest to their build outputs. <tobtoht>if all hashes match, you can rule out compromised firmware, kernel, OS, etc with some degree of confidence. <xentrac>aggi: achieving a full system bootstrap *not* including the kernel is also a very important achievement, because the kernel can likely be replaced without disturbing the rest <aggi>xentrac: when kernel is excluded and relied upon with a circular dependency then it's not a full system bootstrap by definition <aggi>of cause, it may not be necessary always to accomplish a full system bootstrap to achieve the level of trust which was dicussed <aggi>hence my objection was only this: not to mislead other developers with false premises into thinking a complete system bootstrap was done when it wasn't <aggi>simply because, for several other reasons than trusting trust, a kernel bootstrap must be done one way or another <aggi>same applies to bootloader, BIOS/firmware, macro assembler <aggi>furthermore, taking all the way down the rabbit hole into this requires notable knowledge and skills which was lost otherwise <aggi>then again, maybe it wasn't obvious <aggi>the dependency chain which was implemented with x86_32 in live-bootstrap permits both: a full system bootstrap and /chroot <aggi>which is rather practical, if some pieces can be skipped for faster testing inside /chroot <aggi>but, if the whole dependency chain was altered which makes it impossible to transition it back onto a full system bootstrap then this is a huge tradeoff made <aggi>that's why i was asking about mes-libc, if and how that got removed from the shpack variant <aggi>because mes-libc does not need a linux2/fiwix syscall abi and can operate atop the builder-hex0 kernel/loader runtime environment <aggi>hence the dependency chain with mes-libc involved permits both, a full system-bootstap and testing inside /chroot <aggi>so far, this was implemented for x86_32 "only" <matrix_bridge><Jeremiah Orians> DDC shows that if you have a trusted path, you can use it to verify if other paths include a Trusting Trust Attack. As the firmware and microcode trusted bootstraps have not yet been done; we can only speak of relative safety if the alternatives produce identical results <aggi>which makes it even more interesting to retain x86_32 with shpack/mes-replacement too - seeing to hashes match when intermediate mes-libc got removed <aggi>furthermore, a full system-bootstrap is an engineering discipline in it's own right, to get up and running any system from scratch - regardless of trust <matrix_bridge><Jeremiah Orians> Also if a human didn’t write it, then it falls under precompiled code and wouldn’t count as properly bootstrapped (as the precompilation could have embedded a trusting trust attack) <mihi>stabbles: Impressive work! Do I see it right that stack_c_amd64.c (port of Frans' stack_c.c) is your original work and that you did not port tcc_c.c, but instead use the ~350KB preogenerated sl64 files instead? <mihi>Also, I'm curious why you cannot run 32-bit userland on your laptop, but you can run x86_64 code? I assumed that 64-bit kernels, with the right setarch switches, would support 32-bit (i386) just fine, and while CONFIG_IA32_EMULATION can be disabled, there is no hardware limitation that would prevent enabling it again if you can recompile your kernel... <xentrac>aggi: it sounds like you are trying to impose a definition on the problem that conflicts with the definition that the rest of us have been using for the last several years. It seems to me that that could add confusion to conversations you participate in if you don't clarify that you're using the terms in a different way than others do <xentrac>when you say 09:16 < aggi> xentrac: when kernel is excluded and relied upon with a circular dependency then it's not a full system bootstrap by definition <alganet>regarding the "if a human didn’t write it" thing, I think there's a lot of polemic there, even before AI stuff <alganet>my perspective is "can and will a human maintain it unassisted?", and if the answer is true, it's good enough <aggi>xentrac: there seems not been a defintion before to distinguish a full system-bootstrap <aggi>meanwhile several bootstrapping variants exist: live-bootstrap, cosinusoidally/JS, pnut-cc, shpack/mes-replacement, and many other language/compiler-bootstraps <alganet>for example, my builder-hex0 variants currently do not qualify for "can and will a human mantain it unassisted?", my own criteria. I cannot maintain them alone and there are parts of it I don't understand <alganet>In principle though, there's no conceptual block (a human *could* understand it, no harder than typical code) <alganet>I'm happy to keep it as a working reference if that never happens <aggi>so far it seems live-bootstrap and the pnut-cc variants qualify as full system-bootstrap on x86_32 only (excluding BIOS/firmware still) <aggi>all other ARCH and variants seem to miss some more pieces <alganet>aggi: there's no definition, we do the best we can <alganet>people have expectations (auditability, no blobs, etc) and it's a movement that unifies around approaches to meet those without frustrating too much each volunteer <alganet>that said, there's a lot of free software thinking rooting it <aggi>alganet: i've just had a glimpse at almost all of what's been published here, plus all the other things i placed into my notes.txt in recent decades <alganet>we can think it of an extension of the libre movement that reveals some blind spots (having the code is not enough if something that I cannot inspect is used in the mix) <aggi>if the full system-bootstrap definition can be agreed upon, this too clarifies upon some shortcomings with the most complete option that exists today, live-bootstrap <aleksi>I've been bitten by the compiler bug. For the past few days I've been working on a C99 compiler that could be compiled with M2. My goal is to at least bootstrap tcc <alganet>hate to be that guy but the best definition are the four essential freedoms, extended to previously undisclosed/unexplored aspects of software <aleksi>So basically, C99 minus _Atomic and _Complex but including long double and long long, even on i386 <aleksi>I guess if we do get to that subset of C99, it's close if not even sufficient for gcc 4.6... I hope <roconnor>I think the question is less about whether or not a human wrote the source, but whether a human could review the source. Writing code by hand is simply one way of making sure the source is human scale reviewable. I *presume* all the low level .hex and .M0 files have been generated by computer to ensure the machine code and assembly code comments line up correctly. <xentrac>after the Underhanded C Contest, I'm not convinced that humans can reliably find backdoors in source code <xentrac>all the low-level .hex and .M0 files were written by hand, mostly by oriansj <roconnor>xentrac: by hand as in manually transcribing each assembly instruction to machine code? <roconnor>I get that the assembly was authored by hand. <matrix_bridge><Andrius Štikonas> roconnor: yes, hex files in stage0-posix are handwritten <roconnor>I was about to say, the RiscV ones looks horrible to do by hand. <roconnor>I thought manually compiling my haskell functions to combinators was bad... <matrix_bridge><Andrius Štikonas> Perhaps with the exception of builder-hex0... <matrix_bridge><Andrius Štikonas> That's fairly easy, just some mechanical work <matrix_bridge><Andrius Štikonas> Immediates are a bit annoying at this step but you can create some bit templates <alganet>The "High_level_prototypes" folders tell the story. There was assistance from existing tools, it's insane doing without it. <roconnor>The hand conversion from M1 to hex2 seems tedious and error prone. <alganet>That said, at some point, they got independent from the prototypes <matrix_bridge><Andrius Štikonas> You look at opcode and copy paste appropriate hex value <matrix_bridge><Andrius Štikonas> It's the constants that are annoying and take time <alganet>yep, refactorings and changing ideas is brutal without some regen <alganet>but once it crystallizes, it gets easier <matrix_bridge><Andrius Štikonas> And you only ever need to write 3 programs in hex0 <roconnor>Even writing hex2 by hand seems tedious, especially as the programs get larger and larger. <alganet>there's a kind of wall on cc_*.M1, I think it's the biggest jump