IRC channel logs
2023-12-26.log
back to list of logs
<fossy>i think the 6 hrs is new... i think it was lower in the past <fossy>oh, no archive.org says it has alwyas been 6hrs <fossy>oh, that's not all too uncommon, sometimes the log bugs out a bit during CI runs. when the run finishes the full log will be there <fossy>what's the status of your PR <fossy>cause i see gradually more and more things being added to it <stikonas>hmm, I think after returning from syscall, we end up with some unclean state which works fine while we are in our code, but randomly bombs out (probably on timer interrupt) while in UEFI code (e.g. printing to console) <Googulator>Hopefully it's finished now, I'm just waiting for one final clean CI run <fossy>my brain had #354 and #361 as the same PR <fossy>also - just one thing im noting in the 361 commit history, when getting changes from one branch into another branch, prefer rebase over merge - merge creates these merge commits which pollute the commit history <fossy>dont worry about removing the one that's there but just for future reference <Googulator>I rather disagree with the notion that merges "pollute" the history - in fact, having merge commits enables a more accurate way of representing development history, as opposed to falsifying it to fit an imperfect VCS tools's time linearity requirements <Googulator>probably has to do with the fact that I'm coming from a Mercurial background <stikonas>I know plenty of people from mercurial background who don't like merges... <Googulator>merges are indeed a PITA to get right in something like SVN (was a big reason why we switched from SVN to Hg at my old employer) <fossy>i agree with that in terms of representing development history - i'm ok with them for that purpose. but the typical git workflow i've used in most projects ive contributed to is that commit history *is not* time linear, but rather each commit represents a distinct change to the codebase. personally i like that more because it makes commit history much more ergonomic to work with; i know <fossy>that if i cherry-pick XYZ commit then I am getting exactly what I think I am and i don't have to read all of the surrounding context in the commit log to determine if i need to care about merge commits etc <stikonas>well, we used Hg at my work when I started, but soon switched to git (probably within a year) and most people somehow don't like merges <fossy>(another thing is that git merges aren't implemented very well :P) <fossy>i haven't had the displeasure of working with svn too much <stikonas>it's not too bad, there are way worse systems... <stikonas>I know hardware engineers often use perforce, that one is a total disaster <fossy>yeah i have not heard good things about that one <Googulator>perforce is the one that requires a constant nailed-up network connection, right? <Googulator>can't even list working copy status without talking to The One Great Server <Googulator>SVN can at least make do with requiring a connection only when running commands, and even then only some commands need to talk over the network <stikonas>also it is often used as glorified FTP server rather than version control... <stikonas>ok, so I've got some eveidence that UEFI timer interrupt is causing me problems... <Googulator>at my old employer, after an incident when someone committed several GB of intermediate build artifacts, we came to calling such content "adultery" <Googulator>we maintained a list of "known adulterous commits" in a push hook that would block such commits from being pushed upstream again <stikonas>setting UEFI process priority to the highest level seems to "temporarily" avoid hang, though as soon as I call in back into UEFI, timer interrupt is reenabled <Googulator>some good news: I now see CI logs up to 2023-12-26T00:32:12.5381402Z in raw mode <fossy>Googulator: thanks for your help with maturing simplify btw, i appreciate it alot, you identified some problems it would have taken me a fair bit longer to recognise (the benefit of second pair of eyes!) <Googulator>...and it's done, uploading pass2 image as we speak <Googulator>(looks like it might be worth adding compression, rathe than just tar, at this point) <fossy>for the transfers between passes? <fossy>quite possibly, a gzip would probably be optimal <Googulator>the final tarball has compressed images inside, so it's not that important there <fossy>uncompressed = network is too slow, xz = too long to compress, gzip = probably sweet spot <fossy>bz2 can be even slower than xz <fossy>especially for decompression i believe <Googulator>I was thinking, wouldn't it be enough to do a 2-stage GCC 13 bootstrap when --update-checksums isn't given? <fossy>what do you mean by 2-stage? <Googulator>GCC 13 stage 3 is meant to rebuild the exact same binaries as stage 2 - indeed, a mismatch is considered a build failure -, but the binaries are already verified by the checksum <fossy>as in removing the third stage from gcc build where it compars? <fossy>does GCC configure supoprt that <Googulator>(plain "make" is equivalent to "make bootstrap") <Googulator>fossy: what's the use case for the current default of deleting the bootstrap ("tmp") directory after bootstrap is done? <Googulator>As I understand it, there's nothing temporary about it, since it's where the sole final product of the bootstrap process ends up created. <Googulator>Is this residue from a concept similar to how builder-hex0 was originally meant to be used (i.e. writing the result back to the virtual disk at sector 0, shutting down the VM, then extracting the build result from the disk, and finally deleting the virtual disk as cleanup)? <Googulator>IMO the directory name should default to "build" (by analogy with GNU build system) or "target" (by analogy with Maven) <Googulator>"fosslinux tagged this Nov 7" - no, I've tagged it, right now... <fossy>yes, it's a residue, that is a bad default <fossy>tmp is also a residue, i just haven't thought of a better name <fossy>target i think is probably my preference? <fossy>seems to be taking the commit author and date?! <Googulator>$ git tag where-i-started 47feb75; git push origin where-i-started <Googulator>("origin" is Googulator/live-bootstrap in this case) <Googulator>"target" sounds good to me, especially since even the directory layout ends up very similar to Maven's "target" <fossy>ohh.. git doesn't store author information for a "lightweight" tag, the default, only for a "annotated" tag, created with -a <fossy>github, should not be interporlating author information from a lightweight tag <Googulator>I'd definitely default to preserving target, the question is, do we even need an option? <fossy>nah, i reckon let's just kill --preserve and make it the default <Googulator>yeah, an option for "waste several hours building, then nuke the result from orbit" doesn't seem to make any sense <Googulator>maybe ignore --preserve for backwards compatibility <fossy>eh, we have enough churn at the moment that i don't really care if it's not backward compatible <Googulator>I kept tmpfs support for now, as it's not actively harmful, and may even be useful on systems with slow storage and lots of RAM <Googulator>looks like total CI time right now is just barely below 6h <Googulator>"BuildStream/2" working as a user agent suggests that requests specifically is blacklisted <Googulator>in which case we can just say "User-Agent: live-bootstrap/2023" <fossy>i use tmpfs regularly, RAM is faster than storage, i have a fair bit available, and reduces wear on my SSD <fossy>(even though SSDs do admittedly have a fairly high TBW these days) <Googulator>although a simple change I included in that patch (mkdir -> makedirs) means it's now possible to do "-t /tmp/bootstrap" <Googulator>I guess it's time for me to rebase & clean up the bare metal work on top of that now :) <fossy>github actions, you suck, "The workflow is not valid. .github/workflows/bwrap.yml: Anchors are not currently supported.", this is just a YAML parsing thing, literally just use a yaml parsing library that supports it -.- <fossy>Googulator: hm, do you think there is any point in #367 then? <Googulator>I was thinking maybe if someone wanted to download the internal stage2 artifact for any reason, it would be smaller - but actually even that's not the case, because GH zips artifacts for download, even if said artifact is already itself an archive <Googulator>I'm downloading one of those zips just to be sure <Googulator>if it's STORE, then there's a benefit there, if it's DEFLATE, then it doesn't matter at all <Googulator>GH now has conflict resolution on the Web, but it messes things up terribly :( <fossy>oh, yes, GH conflict resolution is poor on the web <Googulator>still had the "tmp" default expected in bwrap.yml, when it's actually "target" now <stikonas>hmm, yeah, I don't see it in tarball either <stikonas>fossy: yeah, gnulib does have those files... <fossy>oh, okay... but do they actually make it into the libtool-2.4.7 tree? <fossy>lets have a look at a finished build, brb <fossy>yeah, i don't think it gets in there.. or at minimum, something changed so that it doesn't anymore <fossy>even if i completely nuke the uniwidth directory everything still works identically <fossy>wth, but they are used somehow?! <fossy>oop, and something is wrong with binutils 2.41 ?? <fossy>looks like my package cache is outdated, time to do a full run i guess' <stikonas>I was just going over the list of new gnulibs <stikonas>then there are still nyacc pregenerated files... <stikonas>hmm, looks like I might have figured out how to do syscall on UEFI... <stikonas>first thing after syscall I need to adjust SS register... <GoogulatorMobile>What's currently the major blocker for upgrading to a newer Linux kernel? <GoogulatorMobile>4.9 is EOL, and it's directly interacting with attacker-controlled TCP streams <stikonas>GoogulatorMobile: probably blockers are already fixed <stikonas>GoogulatorMobile: but you'll only be able to upgrade to 5.6 or something like that <stikonas>oh actually we might need to first upgrade GCC 4.0.4 to GCC 4.6 <stikonas>that's the last before GCC 4.9 is required <stikonas>so I think that's the first thing we should target... <stikonas>(it's also the version that ekaitz was targeting for riscv backport...) <GoogulatorMobile>4.14 is the only thing we can build rn that's still in support - for 1 more month :( <stikonas>we'll eventually be on versions that are out of support <stikonas>otherwise we need to build multiple GCC's before linux... <GoogulatorMobile>"building just one GCC before linux" is IMO a less important target than "not giving attackers network access to a vulnerable kernel" <GoogulatorMobile>Especially with plain HTTP downloads, where even an MITM compromise is possible <GoogulatorMobile>The alternative is maintaining a fork of 4.14 with bootstrap-relevant security fixes <GoogulatorMobile>Or implementing an option to download from a local HTTP source instead of from the original mirrors themselves <stikonas>well, should probably switch to 5.15 first <stikonas>5.15 will bring us more features/fixes too <stikonas>hmm, I guess we don't have function pointers... <stikonas>and goto *ptr; is not even standard C, it's GCC extension <oriansj>stikonas: M2_Planet has FUNCTION; so you only need to use FUNCTION <oriansj>if you look in M2-Planet's cc_core.c you'll see function pointers are passed and used in: common_recursion and general_recursion <oriansj>but yeah goto *ptr; will work sorta but it isn't one of our test cases <oriansj>you need FUNCTION_name if you want to use goto but I don't recommend it <stikonas>well, FUNCTION_name wouldn't work here... <stikonas>it's really jumping into address that was previously malloc'ed <stikonas>oriansj: anyway, thanks for reminding me about "FUNCTION" <stikonas>I have a few other assembly functions, though not sure whether they are suitable for M2libc... <stikonas>it's very x86_64 specific... Even if we put assembly code to M2libc, it wouldn't be portable <oriansj>stikonas: M2libc has architecture specific directories explicitly for that purpose <oriansj>but yeah, only should be adding functions to M2libc if it would be generally useful and not task specific <matrix_bridge><Andrius Štikonas> You can't even read or write to those registers from ring 3... So yes... <matrix_bridge><Andrius Štikonas> Though this posix-runner won't be switching to ring 3... I think it would be simple to stay in ring 0 <matrix_bridge><Andrius Štikonas> Just like builder-hex0 does on BIOS systems <Googulator>just managed to reproduce it - configure.ac is properly patched to remove the offending check, but configure isn't <fossy>Googulator: is it running autoreconf? <fossy>stikonas: i think we can just close #203 then, while there are pregened files in the gnulib tarball they very obviously don't touch the final system <fossy>(well very obviously now, not before :D) <Googulator>it uses real autoconf-2.69 and autom4te-2.69, but Perl versions of aclocal and automake in place of the real ones <Googulator>autoconf is called like this: $AUTOCONF -Wall -Werror <Googulator>Usage: ./automake/usr/bin/autoconf-2.69 [OPTION]... [TEMPLATE-FILE] <fossy>yeah i was about to say the same <fossy>oriansj: re that blog post, not really sure how that means go is distributing rust binaries, this is just showing a way to integrate rust into golang using webassembly <fossy>an increasingly common paradigm for FFI, particularly in golang, which is typically horrible at FFI <fossy>this wouldn't ever be used in core golang though too many problems with it <Googulator>all I want to know is, Why... Isn't... That... The... Default?! <stikonas>fossy: yeah, that's fine regarding #203...