IRC channel logs
2024-01-02.log
back to list of logs
<stikonas>number of open bugs and PRs in live-bootstrap skyrocketed recently... <stikonas>though I guess it means project is active <stikonas>Googulator: I went over your PRs, mostly looks fine, but maybe let's wait for fossy to review too <stikonas>well, with more people, we can afford more review too <stikonas>you are also reviewing some other PRs... <stikonas>it's definitely better than me reviewing fossy's PR and fossy reviewing mine <stikonas>hmm, though after today I'm a bit scared of stuff like Variable *last = NULL; <stikonas>especially since this is constant and not function... <stikonas>but there is some bug in M2-Planet where assigning doesn't work <stikonas>took me a while to understand why "child" variable had some bogus value... <Googulator>if "struct process* child = current_process" didn't work, but "struct process *child = current_process" inexplicably did :) <Googulator>although it seems more likely that the issue is related to referring to a global variable in a variable declaration <stikonas>Googulator: but the same was happening with assigning calloc result <Googulator>hmm, where is script-generator's checksum located? <sam_>i'm not sure what to think about this, but <stikonas>though programs that print their own source is nothing new... <stikonas>and actually hex0 kind of also does that, it's just that source is not stored in the binary itself... <stikonas>but how can we know that source printed by that executable is real <Googulator>next step would be a compiler flag, e.g. -Q, to turn the generated executable into a quine :) <stikonas>oh yeah, I was trying to remember this word "quine". Thanks! <stikonas>hmm, so M0 in my kernel is broken by something the last two commits... <stikonas>but at least I'll should be able to "bisect" it by applying changes in smaller bits <stikonas>ok, got it! I need to keep brk area calloc'ed rather than malloced... <stikonas>as posix version of M0 assumes that newly allocated memory is zeroed <stikonas>hmm, and probably need to zero it on execve too... <fossy>Googulator: looking over your PRs now <fossy>Googulator: are these ok to be merged in any order <fossy>well, apart from the "Depends on" you are writing in descriptions <Googulator>this doesn't affect the PRs, I'm still using the old builder-hex0 there <oriansj>stikonas: yeah calloc works for any type defined prior to the sizeof expression and will also work well for arrays of structs <fossy>Googulator: 393 & 395 are just the sum of other PRs, yes? <Googulator>I'm currently running 393 on one of my boards, and 395 on the other <fossy>thanks for taking the time to upstream a relatively large set of changes :) <Googulator>There's one more piece to come - the builder-hex0 upgrades <Googulator>once I figure out why hda writes aren't working right now in qemu <Googulator>(irrelevant for our use case, but required for upstreaming) <Googulator>is builder-hex0 used anywhere outside live-bootstrap? AFAIK Guix doesn't use it. <Googulator>janneke: (hoping you know this) how did the static "driver" binaries for Guix get built? That is, the static guile, tar, xz, etc. used to drive the full source bootstrap. <Googulator>I'd be interested in reproducing them in the live-bootstrap environment. <ekaitz>Googulator: i might be able to answer <ekaitz>Googulator: they are in a tarball <ekaitz>you can read them in gnu/packages/bootstrap.scm <stikonas>it basically needs guix to build them... <stikonas>Googulator: so basically you should try to run guix with different bootstrap binaries that you produced, that will cause all "guix" checksums to be different, but hopefully actual file checksums will be the same <stikonas>and then it should be able to build those bootstrap binaries <Googulator>ekaitz: the bootstrap _binaries_ are themselves in a tarball <Googulator>I'm trying to reproduce that tarball from sources <ekaitz>that's what stikonas told you i think <ekaitz>make-bootstrap is where those are built <Googulator>I can see guile, xz and tar here; mkdir preusmably comes from coreutils <stikonas>yeah, ironically guix is hard to bootstrap from source... <Googulator>My plan is to rebuild the 5 bootstrap binaries (bash, xz, tar, mkdir, guile) without using Guix itself (using the Guile 3.0.9 we already have is fine, so long as it doesn't pull in the original bootstrap binaries) <Googulator>they should be binary identical to the ones normally downloaded by Guix <Googulator>and then somehow override Guix's download location to load the rebuilt ones instead of the originals <Googulator>I wonder why tar, xz and bash are even involved in the bootstrap, given gash and bootar <stikonas>Googulator: it's not that simple I think <stikonas>let's say you get all the guix dependencies built <stikonas>then once you install guix binary, you can add it to /gnu/store with "guix download" command <stikonas>patch guix sources to use those binaries <stikonas>and then you should be able to start building with --no-substitutes <stikonas>I'm not sure if you'll be able to get it to be binary indentical without guix... <stikonas>if yes, then maybe everything is simpler <stikonas>but you'll have to somehow simulate all the build environment that guix has <stikonas>I'm sure all those guix paths with hashes end up affecting hashes of bootstrap binaries <Googulator>If the bootstrap binaries are reproducible within Guix, then rebuilding them in a patched Guix that will accept "wrong" initial binaries can work too <Googulator>& then stage the rebuilt binaries in /gnu/store, and rebuild+bootstrap a clean Guix <janneke>Googulator: the bootstrap binaries were built doing: guix build bootstrap-tarballs <janneke>however, they were last built some 4y ago <janneke>so, to reproduce them, you'd have to checkout an older guix <janneke>let me check which guile binary exactly is used right now <janneke>guix build -e '(@@ (gnu packages bootstrap) %bootstrap-guile)' <janneke>=> /gnu/store/lgi9x15a0w35mcpd7g1kb9274r6wy4pv-guile-bootstrap-2.0 <janneke>you can see that guile-2.0 is still used, the original guile driver from 2013 <ekaitz>I sent you some patches for mes and tcc <ekaitz>janneke: if you could apply the tcc one and re-upload I would make a patch for guix to include our riscv work <janneke>yes thanks, i updatet mes wip and tcc@mes-0.25 and uploaded a new tarball <janneke>ACTION hasn't gotten round to answering email -- nice work tho <stikonas>janneke: not sure if you saw it yet, but we might have a new option for baremetal bootstrap on UEFI soonish <stikonas>janneke: so rather than porting meslibc to UEFI, we can just use this kernel, though I'm still debugging it on stage0-posix binaries. Mes would be next <stikonas>though the limitations would be that this is x86_64 only. And will only run position independent binaries <ekaitz>janneke: but I need the link to the tarball please <ekaitz>let me see if that's the latest one <ekaitz>i'll add it to guix and send patch <ekaitz>janneke: i'll cc you in the patch <ekaitz>i'll keep 0.25.1 for the moment until 0.26 has the double compilation fixed <janneke>you need 0.26.1 for that, right? (git has your patch, but yeah -- bootstrap) <ekaitz>but as long as 0.25.1 works, there's no hurry <pder>Googulator: On current master for live-bootstrap (commit 133c05426c2133c60) running ./rootfs.py -q hangs every time on booting Fiwix. Applying commit 812b41d4f8593bcfd29 (Update Fiwix sources link to version with PAE passthrough support) fixes the issue for me. Are there any plans to include this updated Fiwix source? <Googulator>AFAIK the plan is to move to Fiwix 1.5 eventually, but if my fork fixes an actual issue in the wild, then maybe we should include it until then <Googulator>I'm gonna test if I can reproduce this on my system <Googulator>QEMU emulator version 6.2.0 (Debian 1:6.2+dfsg-2ubuntu6.15) <Googulator>pder: I guess your version of QEMU & SeaBIOS uses a different memory map by default <Googulator>(I still don't see how PAE passthrough would fix it though, since it merely preserves PAE information for later use by Linux; Fiwix still can't access PAE addresses, and it essentially sees the same memory map as before) <oriansj>well, guix changing its "root" binaries is a problem as it'll change all of the hashes for all of the packages (a sort of reset world); so they are likely to avoid doing that as long as possible and we will probably have to give them a good reason to do so <Googulator>Which is why my goal is to rebuild the exact binaries currently in use <Googulator>IMO the only good reason to deviate from that goal would be if upon rebuild, we couldn't get the hashes to match exactly, and the remaining unavoidable difference turned out to be an actual Karger-Thomson retrovirus embedded in the current binaries <oriansj>which would be surprising but certainly a good reason for them to change <Googulator>pder: ./rootfs.py -q (well, sudo ./rootfs.py -q since on WSL2, only root has KVM access) on a clean checkout of 133c05426c2133c60 booted cleanly into Fiwix here, bootstrapping Perl as we speak