IRC channel logs
2023-04-27.log
back to list of logs
<fossy>stikonas: do you have a link to dhcp? <fossy>this dhcpcd tarball thing appears to be a one-off - i don't think we'll see this again <fossy>yeah, it'd probably be ok, we use much lder software, however dhcp client is one of the things that does need to keep up with the various networking trends lol <fossy>as soon as i finish one the next one pops up <fossy>oh well, shouldn't be more than an hours work (hopefully i can drop all the backports) <stikonas>yeah, hopefully we won't need those patches <stikonas>but yes, at this stage things should just build <stikonas>assuming they didn't add any new pregenerated files <oriansj>roconnor: kaem also includes logic related to the crashes found in fuzzing kaem heavily. <stikonas>well, all stage-posix0 C tools are fuzzed... <stikonas>only early hex/assembly tools are more crashy <oriansj>the biggest problem was IBM went off the rails with e_entry <oriansj>so my solution was to put :__e_entry <stikonas>so basically one can already start writing hex0 for ppc... <oriansj>and make a few tweaks to M1 and hex2 so that it better maps to the architecture <stikonas>and probably borrow riscv's ".hex" style <oriansj>but I really don't like how it came out and it was prior to the RISC-V work <oriansj>so it would only take about a couple days to clean up the M1 and hex2 to better match the PowerPC64LE architecture <oriansj>I would have done PPC64BE and PPC32LE and PPC32BE too if I could have figured out how to make VMs for testing or had the hardware <oriansj>fitzsim: if that work doesn't interest you; just providing instructions for getting a QEMU virtual machine into a state for running a linux distro would be enough for me to do that work for you. <oriansj>(or any free as in freedom POSIX which has SSH support in a pinch) <stikonas>oriansj: though before we start adding ppec64le, we should get a release out :) <stikonas>(well, maybe faster if ppc64le has a nicer immeditae encoding than riscv) <oriansj>if you look at the M1 example, you'll find except for its bit alignment it is a much nicer immediate encoding. <oriansj>janneke: yes I've discussed with the DuskOS author a few technical points and in many ways it is quite compatible with stage0. The primary disagreement we had was I preferred smaller roots and they wanted to only depend upon 1 simple FORTH. <oriansj>which given their comfort with FORTH is quite expected. It honestly is the first FORTH that I ever saw as a reasonable bootstrapping root. Pity it was done after cc_* and the rest were done. <stikonas>yeah, those ppc immediates resemble x86 more than riscv <oriansj>it is nice to see that someone took another whack at the GHC pinata <oriansj>if only there was a haskell half step like tcc is a half step to GCC <muurkha>well, that's what nhc98 and yhc were supposed to be <roconnor>maybe hugs is more like mes in this analogy <roconnor>some of the files generated by tcc are stripped and others are not. I'm not sure what dictates this. <roconnor>maybe it depends on if they get linked with any library or not <oriansj>muurkha: unfortunately they haven't hit the stage of maturity of being able to build the next step -_- <oriansj>and we don't even have a solid guess of how many little steps remain before it is done <oriansj>muurkha: well the next step in Haskell would be GHC; unless they are not up to the TCC level in which case they would be the MESCC step building TCC but for haskell <muurkha>well, you can build Hugs with the stuff that's already bootstrapped <muurkha>GHC might be the next step, I don't know <janneke>oriansj: nice; yes it's inspiring they have something real and for a big part complementary to what we have <muurkha>janneke: what's the obstacle to getting Guile in Guix built from source too? <janneke>muurkha: no worries, guile is built from source too, in Guix <janneke>just not the "guile-bootstrap" package that is the driver for package builds <muurkha>janneke: right, why isn't guile-bootstrap built from source? <muurkha>i thought we had the guile bootstrapping problem licked <janneke>muurkha: for a similar reason why the linux kernel isn't built from source at the start of the bootstrap <janneke>the first package "bootstrap-seeds", which holds the 357 (soon to be 256) bytes initial hex0 binary, is built using...guile-bootstrap <janneke>guile-bootstrap can only be built from source after we have gcc and glibc, which is waaay further down the bootstrap chain <janneke>it's kind of an "external" dependency, outside the build graph <muurkha>right, it's driving the whole process <janneke>the live bootstrap effort does not use guile-bootstrap <janneke>(but it uses plain git/tar/bash to unpack everything initially) <janneke>i always think it's "unfair" not listing guile-bootstrap, although it is confusing and we also don't list the linux kernel <fossy>ok wth is going on with gh actions and nongnu <Raito_Bezarius>Hi there, is there anyone working on a "in UEFI" bootstrap to the Linux kernel and (whatever) to wire it up with existing bootstrap efforts from Guix for example? <Foxboron>Raito_Bezarius: what does "in UEFI" bootstrap mean? <Raito_Bezarius>Foxboron: You boot into a UEFI "compiling image" rebooting into the Linux kernel at the end, potentially with the required binary to continue the bootstrap process <Raito_Bezarius>UEFI compiling image which has to be prepared somehow somewhere else or to be trusted of course <Foxboron>Raito_Bezarius: I don't think UEFI itself is sophisticated enough for that <Foxboron>You could uefi -> linuxboot -> compile kernel -> kexec <Raito_Bezarius>How do you source the compiler for compiling the kernel Foxboron ? <Foxboron>Raito_Bezarius: This is all very abstract so I'm just guessing; I'd simply boot a minimal reproducible linuxboot image, fetch the bootstrapped and reproducible gcc and kernel sources. Build stuff and add everything into the TPM eventlog <Foxboron>You get attestation of the work, the tools and a bootstrapped server <Raito_Bezarius>okay Foxboron while you are here (krkrkrkr), in all the bootstrappable seeds stuff, I am not sure I have seen a clear answer about the "fetch" process, e.g. how do we ensure integrity, etc. ? <Raito_Bezarius>I am not familiar enough with Guix to understand how they perform the stuff, is it a fixed output derivation à la Nix or is there signatures involved? <stikonas[m]>Feel free to continue bootstrap chain on uefi (next step is to port mes) <Foxboron>Raito_Bezarius: I havent paid enough attention to the ideas and details on this topic to know :) <janneke>Raito_Bezarius: it's quite similare to nix, just packages <stikonas[m]>But I haven't implemented any input functions in m2libc <rickmasters>I was thinking one possible path for UEFI is to compile a bootstrap kernel, load source into memory with UEFI functions, then kexec. <stikonas[m]>And yes Foxboron is right uefi is not capable enough to build Linux. It even can't run make. UEFI has no pipes... <Foxboron>I suspect the linuxboot project would fit well into this picture, really <janneke>ACTION just heard about it this morning <janneke>Raito_Bezarius: hehe, okay -- thanks for that! <Foxboron>The lobste.rs thread on this is popular fwiw <janneke>this, #bootstrappable channel has been an amazing support for me <stikonas[m]>All input tarballs and all output products are checksummed <Foxboron>Things like reprobuilds, bootstrappable builds, binary transparency and these topics are sometimes a bit hard to explain to people <Foxboron>If anyone wants to engage on the thread i can send invites obviously <fitzsim>oriansj: I can probably help with ppc64be qemu setup; I maintain a builder for sourceware.org <gforce_de1977>@oriansj: for PPC 32bit i used the QEMU-board 'mpc8544ds' and compile a linux kernel with 'ARCH=powerpc' and 'mpc85xx_defconfig' - is this what you ask for? than I can provide a commandline + settings for PowerPC64LE too <stikonas[m]>Though "never been achieved, to our knowledge, since the birth of Unix" is a bit out if date :). Current release of freedesktop SDK was already using our bootstrap seeds <vagrantc>how ... complete of an OS is the freedesktop SDK ? <stikonas[m]>There is even a talk by doras at Linux app summit 2023 <janneke>stikonas[m]: where can i read more about that? <stikonas[m]>janneke: probable best to watch the talk, I also can find the link to merge request <janneke>stikonas[m]: i'd like to see both :-) <vagrantc>guix had thte full-source bootstrap for quite a while before, no? it just wasn't merged into guix master? <janneke>yeah, but anyway, selling bootstrappable builds to freedesktop.org is the sort of thing that we need <janneke>kinda interesting/unexpected angle; i had the impression freedesktop.org was moving towards flatpacks/snaps: non-upgradable binaries all the way? <vagrantc>you still have to get the non-upgradable binaries somehow <janneke>"This removes the recursive dependency of freedesktop-sdk on its <janneke>previous versions as binary seeds for bootstrap, and from Yocto as <janneke>its original binary seed, allowing the project to be rebuilt from <janneke>scratch even if its previous versions are completely lost." <janneke>that's so interesting -- one of the things we predicted people would eventually come to guix (or nix) for, getting their build environment / build system managed properly <janneke>it's not immediately clear to me what kind of packaging system they used, though <stikonas[m]>janneke: they take output of live-bootstrap and then use buildstream to build on top of that... <janneke>seems guix is/was discussably the earlier, though ;) <janneke>"● The upcoming Freedesktop SDK 23.08 will be bootstrapped from the new binary seed" <janneke>stikonas[m]: right (never heard of it ;-) <stikonas[m]>It was actually back ported to stable, so it's already out... <roconnor>I'm not sure how musl-1.1.24's configure manages to run without grep available. <janneke>stikonas[m]: right, they released end of March <stikonas[m]>yes, though it's not really a race. We all want more stuff bootstrapped. <rekado>janus: sorry, you are right about HBC being written in Lazy ML. <rekado>the recursive imports of GHC are only a minor obstacle; they can be rewritten. <rekado>in fact, that’s what I did in my attempts to host GHC 4 on Hugs. <rekado>these notes are incomplete and don’t detail everything I tried, but it’s all that’s left <roconnor>hmm my tcc is choking on the static array size in musl's <roconnor>hidden void __procfdname(char __buf[static 15+3*sizeof(int)], unsigned); <civodul>that freedesktop-sdk/live-bootstrap merge looks neat! <civodul>it's about updating hashes of pre-built images, roughly? <rekado>I’m not really motivated to do the work necessary to clean up the notes, because the result is such a bummer. <janus>right, I understand. You still have more notes than anyone else on this <rekado>perhaps it’s better for people to ignore what I did <rekado>others are better networked and more social than me, so perhaps they can find ways that I didn’t. <janus>Right, in a way, if people are reattempting the same things, there is a chance that they succeed on something everyone else 'thought' was hard <rekado>would be a shame if my notes were discouraging to anyone. <janus>Right, right... To me they are not discouraging at least, but I am going down another avenue (just trying to get GHC 0.96 compiling on NHC, nothing more) <rekado>I’m stepping away from bootstrapping; got myself a new synthesizer and I’ll have to reverse engineer the USB protocol <doras>janneke: congrats on the nice progress with Guix's bootstrap. <stikonas>janus: with so many people reattempting the same things, at some point it might be quicker to rewrite GHC in another language... <janus>Yeah, Joachim is well connected within the Haskell/GHC community, so if a new compiler needs to be written, maybe he can help motivating that <stikonas>though GHC is probably something like 600k LOC... So not exactly small <janus>stikonas: I was thinking that too! Writing a Haskell interpreter that just knows how to interpret NHC seems like an interesting hobby project ;) <janus>so if I can get GHC-0.96 compiling on NHC, that could be the next step :P <janus>rekado: synthesizer, that does sound fun :) Music is nice because you get the gratification more regularly and consistently <janneke>didn't know about your effort, very nice getting bootstrappable into freedesktop.org! <janneke>ACTION hasn't been very observing here, lately <stikonas>janneke: do you know any estimate when gash might be ready? Something like for mes 0.26? <janneke>first mes-0.25 with pending m2-planet 0.10 and risc-v patches <stikonas>ideally mes-0.25 should work with current unreleased m2-planet <stikonas>(which will probably be released before mes-0.25) <stikonas>I think I've sent you those defines patches before <stikonas>and also there was some struct issue which I think is also fixed in wip branch <janneke>stikonas: right, i think they're on mes wip <stikonas>M2-Planet git became a bit more advanced and started choking on some hacks from m2 folder... <doras>janneke: you had a significant role in it, it couldn't be done without mes :) <stikonas>anyway, it's probably going to get harder to extend M2-Planet much more... <janneke>doras: my "role" was well hidden, like it's supposed to be at that level of abstraction <stikonas>yes, mes and mescc is the big glue between stage0-posix and tcc (which is the first compiler actually capable of building most of the stuff) <janneke>did appreciate the mention though, of course <janneke>yeah, it's easy to underestimate the work that went into the bootstrappable-tcc patches to get it built with mes+mescc <janneke>not the most elegant bits of the bootstrap either <roconnor>I suppose all those patches are hidden away in the mystrious tcc-0.9.26-1136-g5bba73cc.tar.gz file. <doras>And actually builds it in a sandbox containing nothing but the source files and seeds. <civodul>doras: oh i see; so /bootstrap-seeds/POSIX/x86/kaem-optional-seed is a kaem script that builds it all? <doras>It's a minimal shell written in hex0 that initiates the build process of stage0-posix <stikonas>though there isn't much in this file, mostly calls to other files <civodul>and the scripts that build all the higher-level packages are in live-bootstrap, right? <stikonas>yes, though you need to add a hook on top of stage0-posix <stikonas>stage0-posix ends a run with after.kaem script that just prints something <stikonas>but you can replace with your own script <stikonas>that's how live-bootstrap hooks into stage0-posix <stikonas>but oriansj cleaned up stage0-posix a lot <stikonas>now it's much easier to use in subprojects <civodul>i guess bootstrappable.org needs an update! <janneke>nice twas quite tricksy to get the right set of tarballs for stage0-posix-1.4 <stikonas>yeah, it would be good to add more projects there <stikonas>civodul: and we have other projects too now, e.g. last year I worked on stage0-uefi <stikonas>civodul: then there is of course rickmasters work <stikonas>he and Mikaku basically solved Linux kernel bootstrapping problem <stikonas>via builder-hex0 -> Fiwix -> Linux kernel chain <stikonas>we can now build Linux, though they are still working on kexec support for Fiwix, so that Linux kernel can be loaded <stikonas>so there was a lot of stuff going on recently <roconnor>heh. kaem's alias command is broken when the alais name is longer than the command. <stikonas>we mostly used to to replace long compile commands <stikonas>roconnor: though I suggest just running kaem in gdb <roconnor>I think add_alias might be fine. If I do alias touch=catm, followed by alias to print a list of all the aliases, the printed list is fine. <roconnor>so I'm guessing the issue is in the command intepreter. <stikonas>hmm, we loop there until token_done == FALSE <stikonas>and this loops over input string i.e. touch in your case <roconnor>yes, collect_alias_token just overwrites the existing token. <roconnor>but if the orginal token was longer, then it will have those characters still there. <oriansj>Foxboron: if you send me a lobste.rs invite, I'll answer any questions people there have <roconnor>token_done needs to overwrite a null at the end. <Foxboron>oriansj: Sure, privmsg me the email you want an invite on <oriansj>Foxboron: my email is Jeremiah@pdp10.guru (as is listed on all my git commits) <oriansj>joined *surprised passwords are limited to 72chars* <stikonas>72 is quite good, I've seen websites that are limited to 15 or even 12 <stikonas>looks fine to me but I can't merge myself, so let's wait for oriansj to review <stikonas>and then after that we need to update stage0-posix (and changelog) but I have write access there... <oriansj>stikonas: the worst are banks which limit passwords to 6 digits >.< <stikonas>though they sometimes have some code generators too... <stikonas>I had one of those, last week battery died, I've replaced it but then it doesn't work <stikonas>roconnor: and I've updated stage0-posix now <oriansj>RSA hard tokens would have been fine; given that the password changes every 60 seconds and 1 guess locks you out until the next cycle <stikonas>well, those also expire after a few years <stikonas>somehow none of the banks I saw have support for something more modern, not even TOTP. And I'm not talking about webauthn <oriansj>stikonas: the number of banks running on IBM's ZOS will stun you