IRC channel logs
2023-12-20.log
back to list of logs
<oriansj>Christoph: bootstrapping changes can be merged slow or fast depending on the people and the lift required. The but progress is always embraced once it proves itself to be a net benefit <oriansj>GoogulatorMobile: well download from zero is relatively rare but local mirror fallbacks might be a good idea; Software heritage links probably are something to consider. (IPFS or gnunet might be interesting future options for ensuring source availability) <GoogulatorMobile>i.e. for each project, create a source repo mirror under the LB project on GitHub (like we do for Fiwix) <GoogulatorMobile>stikonas: from the messages that get printed during tcc compile, it looks like mescc 0.26 is now two-stage <stikonas>but in my case it seems much more than 2 times slower or so <oriansj>well using GitHub as an object store as a one of many is a valid option but I wouldn't trust that Microsoft wouldn't alter any of the tarballs in the future. <Googulator>well, Microsoftness is obviously an issue, but unless they can preimage SHA256, the worst MS or an APT compromising them could do is DoS the bootstrap. <Googulator>(for reference, even MD4 hasn't been successfully preimaged) <Googulator>BTW, looks like the perf regression was almost entirely from $JOBS not propagating <Googulator>just over 3 hours into the bootstrap (on the Q6600), and it's already in guile <Googulator>(note to others replicating on a Q6600 or similar: this is *with* the 3.6GHz overclock this CPU had back when it was in my gaming PC, expect about 30% perf loss with the default 2.4GHz clock) <stikonas>Googulator: ok, so that's good ($JOBS thing), should be easy to fix... <Googulator>The fix is already in my simplify-playground branch, although it needs cleanup <Googulator>(most of the script-generator changes are really irrelevant, what makes it work is the update-jobs imrove step) <Googulator>also, I was thinking of reducing the role of script-generator to just the kaem era, and parsing the manifest directly using helpers.sh once bash is available <stikonas>though what is the advantage of doing it this way? <Googulator>& at that point, we can do things like proper runtime redefinition of variables, or multiple manifests <Googulator>another thing it makes possible is optional build steps that can be called from terminal once the initial bootstrap is done <Googulator>e.g. we could have a guix.manifest describing how to bring up Guix <stikonas>probably source based distros are the easiest target <stikonas>guix needs a lot of hackery to not depend on its bootstrap seeds <Googulator>and then once you get to that first terminal prompt, you can just type "manifest git" to additinally bootstrap Git <stikonas>yeah, stuff like git and a few other things might be very useful on baremetal bootstrap <Googulator>having manifest parsing be in Bash would also enable better error handling <Googulator>the "Holy Grail" would indeed be bringing up guix or emerge <stikonas>Googulator: we already have text editor :) <stikonas>though I don't really know how to use it <stikonas>and ideally we move to a another bc implemntation <Googulator>given what our early bash behaves like, I doubt ed is any better <stikonas>Googulator: bash is simply because we don't have required headers at that time <Googulator>Gentoo is sadly moving away more and more from bootstrappability, to the point that stages before stage3 aren't even published anymore <stikonas>Googulator: those don't matter too much anyway <stikonas>they just don't exist, I think stage generation is different these days <Googulator>the early bash is actually worse than your typical "non-interactive" bash - if you build a bash normally without readline, it will still give you some kind of shell if you just go "bash" without any arguments <stikonas>but also once emerge is working (even with no packages installed) <Googulator>no line editing, tab-completion, or even prompts <stikonas>it should be possible to install other things <stikonas>Googulator: probably depends on defines... <stikonas>the same bash 2.05b built later (when musl is available) is interactive <stikonas>which rickmasters didn't have time to investigate <stikonas>of course once you get to distro, you can forget not using pregenerated files and stuff like that... <stikonas>but theoretically we would alraedy have all the tools to rebuild them <Googulator>yeah, solving auditability there is another challenge <stikonas>Googulator: also your separate manifest files thing might be useful for different arches <stikonas>e.g. that would allow us to cleanly have slightly different paths <stikonas>though I guess we can have it with full manifest like we have now too... <stikonas>e.g. there could be x86 manifest that skips mes <stikonas>(though I'm not sure what to do with checksums...) <stikonas>they might differ in some intermediate steps <stikonas>perhaps output checksuming is not scalable... <stikonas>in which case it really makes sense to have pre-helpers.sh and post-helpers.sh manifests <stikonas>though hopefully we won't need any more splits <stikonas>otherwise we'll just go back to run.sh -> run2.sh -> run3.sh which caused fossy to do that simplification... <GoogulatorMobile>Pre-bash, script-generator copies the checksums to their correct locations for kaem <sam_>[00:37:19] <Googulator> Gentoo is sadly moving away more and more from bootstrappability, to the point that stages before stage3 aren't even published anymore <sam_>no, it's that users kept using them when there's no real point <sam_>yeah, the scripts are all public and such <sam_>catalyst can still do stage1/stage2 and we do that when something has different ABI <sam_>what I really want to do is make prefix have a more auditable bootstrap path <Googulator>How does gentoo even deal with things like Haskell, which has no known bootstrap path? <Googulator>I somehow doubt the stage3 tarball includes a Haskell compiler. <sam_>we take the L with haskell, yes <sam_>which is nothing that's changed ever <sam_>not sure what your point is <sam_>it's not like stage1 ever had it or dealt with that <GoogulatorMobile>Because I was thinking that to bring up a guaranteed-trusted Gentoo, all that's needed is regenerating the boot ISO and the stage3 tarball using the live-bootstrap environment, since the rest is all source... but binaries for e.g. ghc must come from somewhere even in Gentoo, and the stage3 tarball seems too small for that <Googulator>Bare-metal test done, took 7 hours and 40 minutes. <Googulator>It was just a hair over 7 hours before simplify & the 2 mes upgrades - that roughly 40 minute difference is about equal to how long the mes phase took before the 0.26 upgrade <Googulator>Which is consistent with my theory that mescc is doing everything twice <matrix_bridge><Andrius Štikonas> Googulator: in ghc ebuild there is flag to use ghc binary seed or use system compiler <matrix_bridge><Andrius Štikonas> Sometimes it's not the flag but another ebuild <sam_>what i want to do is introduce binary tagging <matrix_bridge><Andrius Štikonas> But those can be bootstrapped via extra packages using overlays... <sam_>to make it easier to say "really, please no binaries" <oriansj>sam_: sad to hear that gentoo is moving away of bootstrappability <oriansj>ok, I guess I read that wrong. Need more coffee -_- <sam_>but nah i promise it's very important to me <matrix_bridge><Andrius Štikonas> And it's not like building stage1 from live-bootstrap would be any easier than stage 3 <oriansj>and the Haskell problem is definitely going to take a bunch of dedicated work. <oriansj>to repeat the cycle we only need janneke to write a haskell compiler written in haskell; that I am then responsible for bootstraping that subset of haskell. <oriansj>^written in haskell^written in scheme that runs on an interpreter written in haskell^ <sam_>a big challenge for us is actually even stopping more totally-avoidable haskell deps too <sam_>like pipewire nearly started using pandoc <oriansj>well, guess Haskell has to be my next project <AwesomeAdam54321>Is it easier to modify an old version of GHC to use equivalent features in Hugs? <AwesomeAdam54321>because Hugs seems just as capable as GHC, but the syntax is a bit different