IRC channel logs

2024-01-19.log

back to list of logs

<lrvick>finally. got an x86_64 cross compiler built from live-bootstrap
<stikonas>lrvick: nice, any tips?
<lrvick> https://git.distrust.co/public/packages/src/branch/main/src/bootstrap/cross-x86_64/Dockerfile
<lrvick>that pivots from a live-bootstrap container to a x86_64-cross-gcc scratch container
<lrvick>that can at least tell me its own version number
<lrvick>TBD if it can build native musl, binutils, and gcc yet
<oriansj>lrvick: I am sure you can make it happen ^_^ keep up the good work
<GoogulatorMobile>pder: so the new code enabled by that config option doesn't even execute (apart from the initial check)
<GoogulatorMobile>Mikaku: that sounds like some kind of alignment issue
<GoogulatorMobile>But why does a Rocket Lake CPU care about alignment in this situation, where no other x86 CPU does AFAIK?
<GoogulatorMobile>*Raptor Lake
<GoogulatorMobile>(Rocket Lake was older)
<Mikaku>GoogulatorMobile: if I could just reproduce it here
<GoogulatorMobile>Do you have a Raptor Lake system to test on?
<GoogulatorMobile>Seems to be highly specific to Raptor Lake or at least newer Intel "Lake" architectures - AMD Zen 4, for example, has no problem here, even when given 192GiB of RAM
<GoogulatorMobile>(another theory was that it's related to high host RAM, beyond even the PAE limit)
<Mikaku>GoogulatorMobile: no, I only have old computers around me (i386, i486, Pentiums, PC/104, etc.)
<GoogulatorMobile>I do have some newer ones to test on, but all AMD or ARM
<GoogulatorMobile>Newest Intel I have access to is Skylake
<GoogulatorMobile>Maybe Coffee Lake
<GoogulatorMobile>But nothing post-2020
<GoogulatorMobile>(Intel announced breaking 8086 compatibility in their future CPUs in 2020)
<GoogulatorMobile>Mikaku: just had this idea - are we overwriting the GDT or IDT maybe at some point in the builder-hex0 to Fiwix kexec?
<GoogulatorMobile>If that happens, we can get really varied behavior between CPU makes and models, due to different caching approaches
<Mikaku>GoogulatorMobile: I don't know, as to rickmasters
<GoogulatorMobile>The version of builder-hex0 we currently use puts the GDT at 7EF8 to 7F20
<GoogulatorMobile>And the IDT at 10000 to 10800
<GoogulatorMobile>Those addresses are marked as usable in the handcrafted e820 map in kexec-fiwix
<GoogulatorMobile>pder: can you try adjusting the start address of the first AVAILABLE region in kexec-fiwix.c to 0x10800 (instead of 0), and removing 0x10800 from the size of the same region?
<GoogulatorMobile>(use a Fiwix kernel that normally crashes when testing this)
<GoogulatorMobile>Hopefully this is enough to protect builder-hex0's GDT & IDT from getting overwritten before Fiwix loads its own GDT & IDT
<Mikaku>RISC-V mini laptop: <https://3.14.by/en/read/RISC-V-Sipeed-Lichee-Console-4A-Alibaba-T-Head-TH1520-review>
<janneke>Mikaku: that's pretty cool -- the keyboard is unusable, unless maybe you're still stuck on qwerty
<Mikaku>janneke: heh, you mean I'm last few still using qwerty?
<Mikaku>all my IBM keyboards model M are qwerty and considering how solid they are, I guess I'll be stuck on qwerty forever :-)
<janneke>Mikaku: dunno, i've not been keeping track
<nimaje>well, you don't need to use the layout printed on the keys, you can configure your system to use any other layout
<janneke>usually yeah, but not really with the risc-v mini
<Mikaku>yep, just thought on it, even they keys on these keyboards are interchangeable easily
<pder>GoogulatorMobile: I reverted back to master and made the following change to kexec-fiwix.c
<pder> https://paste.debian.net/1304648/
<pder>It hung at kexec-fiwix: jumping to trampoline..
<pder>Let me know if thats the change you intended
<GoogulatorMobile>pder: that's exactly what I meant
<matrix_bridge><cosinusoidally> Is there any way to debug builder-hex0. I was trying my alternative tcc bootstrap path and it seems to crash here https://github.com/fosslinux/live-bootstrap/pull/407/files#diff-cc6e390ef115302dcd339b5d634107e16ad985d974300ad2b51732aa948a7484R203 . It's running the following command: tcc-boot0 -static -nostdlib /usr/lib/mes/crt1.o ../mes-0.25/unified-libc.o ../mes-0.25/libtcc1.o tcc-boot1.o -o...
<matrix_bridge>... tcc-boot0-new
<matrix_bridge><cosinusoidally> the crash seems to happen in console_putc called from handle_syscall_write after reading crt1.o
<matrix_bridge><cosinusoidally> I've been using qemu with -gdb tcp::1234 and then connecting using gdb.
<matrix_bridge><cosinusoidally> gdb gets pretty confused though when it transitions into 16 bit mode
<matrix_bridge><cosinusoidally> I think it fails somewhere in console_putc_16, but I can't see what's going on.
<matrix_bridge><cosinusoidally> Or why that would stop working suddenly. That function will have been called multiple times before as characters are being printed to stdout up until that point.
<matrix_bridge><cosinusoidally> works fine under chroot and bwrap
<rickmasters>cosinusoidally: debugging builder-hex0 is pretty difficult. I'm not sure there is any simple advice I can offer.
<rickmasters>cosinusoidally: I think it is worth trying the latest code from https://github.com/ironmeld/builder-hex0
<rickmasters>cosinusoidally: There are some improved instructions for generating hex0 and testing with live-bootstrap in DEVELOP.md
<rickmasters>cosinusoidally: near the bottom.
<rickmasters>cosinusoidally: It will be especially difficult to debug the output routine because you would need it to work to help debug itself...
<matrix_bridge><cosinusoidally> Ok, thanks. I'll see if I can use a newer version of builder-hex0 in live-bootstrap
<rickmasters>cosinusoidally: If I just checkout and run your branch should that reproduce the problem?
<matrix_bridge><cosinusoidally> unfortunately possibly not, unless the newer version of builder-hex0 is unaffected by the following issues. There were 2 other issues I ran in to (which I hacked around locally). First my cdrun utility seems to fail https://github.com/cosinusoidally/tcc_bootstrap_alt/blob/0.1/tcc_js/cdrun.c (seems to fail on execve). I worked around that by updating after.kaem to use cd (and kaem.x86 to call kaem...
<matrix_bridge>... --file ./after.kaem). The other issue was the initial call to brk is not aligned to 4 bytes. I also hacked around that issue. I can push branches with the hacks applied if that helps?
<matrix_bridge><cosinusoidally> Running under the newest version of builder-hex0 still seems to fail
<rickmasters>I'm willing to make edits to get it working but of course a branch would be easiest...
<matrix_bridge><cosinusoidally> I'm just checking if the latest builder-hex0 still needs the workarounds. If it does I'll push a new branch of tcc_bootstrap_alt
<rickmasters>ok, I'm running the PR branch as well. It's curious that it still runs mes-m2. I guess I was expecting mes to not be there...
<matrix_bridge><cosinusoidally> You need to run ./rootfs.py .... --tcc_bootstrap_alt
<rickmasters>I haven't updated builder-hex0 in this test. It appears to have stopped with last output: "if match xFalse xTrue"
<rickmasters>cosinusoidally: oh, well that explains that :)
<rickmasters>i'm trying again with latest builder-hex0 and --tcc-bootstrap_alt but I expect that I'll need the branch or more detailed workaround instructions
<rickmasters>The newest builder-hex0 has disk I/O improvements from Googulator and there are some interrupt handling changes that might be relevant, but I doubt it
<rickmasters>Yeah, its crashed on cdrun, so I'd need to workaround
<matrix_bridge><cosinusoidally> rickmasters: ok https://github.com/cosinusoidally/tcc_bootstrap_alt/tree/builder-hex0-issue should repro the issue
<rickmasters>ok I'll try it
<rickmasters>It's also worth mentioning the list of limitations: https://github.com/ironmeld/builder-hex0?tab=readme-ov-file#limitations
<rickmasters>Some of them may be surprising. Like you can't reopen a file for writes.
<rickmasters>cosinusoidally: running now. seems to be getting pretty far. lot's of debug output.
<rickmasters>ok, failed on tcc-boot0 checksum. will run again with --update-checksums.
<lrvick>Built x86_64 cross-gcc and used that to build musl, and now cryptic issues with "conftest" building binutils if anyone has had to debug this before: https://dpaste.org/BjJW8
<stikonas>lrvick: have you not installed that gcc?
<stikonas>why is it still xgcc?
<rickmasters>cosinusoidally: so it wasn't able to cp the generated checksum file. I'm suspecting a file system issue. I'll keep digging.
<lrvick>I built xgcc then used that to build musl, and now am using it to build binutils
<lrvick>I too have only used crossdev and have never done this by hand, so I may be misunderstanding the order of operations
<stikonas>yeah, i understand that, but you didn't run "make install"
<stikonas>that probably shouldn't cause any problems
<stikonas>but who knows...
<lrvick>Yeah I was trying to use everything "in place" so I can install all the things at a separate build stage
<lrvick>in a dedicated rootfs
<lrvick>trying to see how conftest works so I can try to do whatever is exploding by hand
<lrvick>Here is the full build setup right now: https://dpaste.org/LTKnc
<lrvick>musl at least built happily without me doing a "make install" as I have "--prefix" set to nothing so it should be portable, if I understand correctly
<stikonas>well, musl is much simpler
<stikonas>binutils and gcc have multiple projects inside the repo
<stikonas>there are top level and subdirectory configure scripts, etc...
<lrvick>fair. I will install to a "cross" directory as DESTDIR to see if that helps, but I stil limagine I have to somehow tell binutils about all the stuff in there manually
<lrvick>build is currently unprivileged trapped in /home/user
<matrix_bridge><cosinusoidally> rickmasters: thanks. The debug logging is a bit verbose in my code. When I ran it on my side I remove the checksum check https://github.com/cosinusoidally/live-bootstrap/blob/tcc_bootstrap_alt-refactor/steps/tcc_bootstrap_alt-0.1/pass1.kaem#L33-L41 since it won't be correct with the tweaked version of tcc_bootstrap_alt . The shasums should match for later stages.
<rickmasters>cosinusoidally: ok, I just thought the checksum generation problem might indicate a problem with the kernel.
<rickmasters>cosinusoidally: but FYI it's actually a wrong file name in steps/tcc_bootstrap_alt-0.1/pass1.kaem
<rickmasters>cp ${pkg}.checksums ${SRCDIR} should be cp ${pkg}.${ARCH}.checksum ${SRCDIR}
<rickmasters>cosinusoidally: ok, reproduced. last line: open: /usr/lib/mes/crt1.o flags: 0 mode: 134215840 flags_new: 0 mode_new:
<rickmasters>cosinusoidally: were you debugging why no output after mode_new: ?
<matrix_bridge><cosinusoidally> Is that just due to https://github.com/ironmeld/builder-hex0/blob/main/builder-hex0-x86-stage2.hex0#L135-L137 ?
<matrix_bridge><cosinusoidally> "# Note: with QEMU+Seabios this does not flush the last character or
<matrix_bridge>CRLF of a line until the first character is output on the next line
<matrix_bridge>and that character cannot be another CRLF."
<rickmasters>Yes
<rickmasters>So, the open is succeeding and then something freezes after that.
<matrix_bridge><cosinusoidally> It seems to get stuck when it attempts to print an "o" on the next line