IRC channel logs
2023-12-06.log
back to list of logs
<GoogulatorMobile>Rock 5B is known to underclock the RAM to 528MHz when it's not sensing load <Googulator>StarFive's AVL also lists the H54G56BYYVX046 as supported <stikonas>yeah, possibly newer kernel will speed it up at some point later... <oriansj>Googulator: well modern hardware doesn't have as secure boundaries as we like to imagine. And Neural net branch predictors do wonders for pipeline fill rates/IPC averages <muurkha>how big a difference do they make in practice? <Googulator>Looks like I may have finally gotten live-bootstrap with simplify to work fully; building gcc-4.7.4 as we speak <Googulator>(bare metal system is lagging behind, it's still creating initramfs for Linux, but I've seen that succeed already) <Googulator>bare metal system has now booted into Linux as well <Googulator>(network was the final issue to tackle, or rather, curl not working properly) <pabs3>hmm, a VM crashing the host OS sounds like a pretty bad security bug :( <pabs3>if its reproducible, would be a good idea to report that <muurkha>such a bug would not be that surprising though <muurkha>if there were only four programs that people ever ran in practice on the JVM, you wouldn't expect it to be very well tested <muurkha>VMs are mostly tested with a very small set of OSes: Linux, FreeBSD, FreeDOS, and Microsoft Windows <fossy>GoogulatorMobile: ah, nice catch for script-generator; useful snippet too, thanks <fossy>GoogulatorMobile: re your host system crashes, do you possibly have bad memory/too highly clocked for your cpu memory? <GoogulatorMobile>Shouldn't be RAM, as this is a Lenovo laptop that comes stock with DDR5-5600 memory <GoogulatorMobile>On affected systems, starting a VirtualBox VM inside a Hyper-V VM would kill the host <fossy>ahhh, ok, yes nested virtualization could do it <matrix_bridge><Andrius Štikonas> Hmm, a thought for bootstrapping on newer systems with UEFI. Rather than porting meslibc to it, maybe we can write efi loader in M2 C that reads all the sources from disk (ESP) using UEFI calls, creates builder-hex0 type disk image and then create some builder-hex0 style Posix kernel that does I/O in memory rather than via BIOS calls. It could also be written in M2 and be either 32 or 64 bits... <mihi>Googulator, I wanted to ask what -accel mode you were using? Qemu with -accel hax reproducibly bluescreens Windows 11 hosts when running certain workload inside 32-bit Hurd in the VM. Running the same workload inside a qemu with -accel kvm inside a 64-bit Linux vm running in hyper-v on the same Windows 11 host (using nested virtualization), nothing crashes. Too bad there is no Windows 11 configuration officially <mihi>supported by HAXM. On a supported Win10 configuration (I don't remember which Windows build, but it was with QEMU 7), no bluescreen. But since HAXM is no longer actively developed and getting a supported configuration was always tricky (you need different HAXM version and matching QEMU version depending on Windows build number), I stopped using that configuration for anything. <mihi>With nested virtualization inside Hyper-V, so far I never crashed either host or intermediate host. Guest crashes do happen though, more often if Windows Defender Core Isolation is enabled on the (outer) host. <mihi>Nested virtualiation does not work for me if the intermediate host is 32-bit though. And since a recent Windows update, I cannot run any 32-bit Linux VMs any more inside Hyper-V (tried kernels/live media back to 2009) if the virtual machine has more than one vCPU. <mihi>(sorry for the off-topic rant) <mihi>janneke: mes 0.26 configure --with-bootstrap (without --with-cheating) fails for me with "unhandled exception: unbound-variable: (srfi-9-struct.mes)". Running on 64-bit debian inside WSL if it matters. <mihi>by works, I mean it builds and then segfaults when running mes-mescc. <mihi>I guess I must be doing something wrong. Even with configure without arguments and make is not able to create valid mes binary... :( <mihi>(building off the v0.26 tag from Savannah Git if that matters) <janneke>ACTION is trying --with-bootstrap now <mihi>without bootstrap, make files while doing "CC crt1.c" <mihi>janneke, can you run bin/mes-mescc without arguments? <mihi>system dependencies (guile, nyacc, etc.) should not have been changed from 0.25? <janneke>i'm not there yet, just passed CC "crt1.c" <janneke>that's right, no dependency changes since 0.25 <janneke>possibly the bootstrap does yet another round where that fails <oriansj>muurkha: well the research papers on neural net branch predictors show a much lower miss rate; which on longer pipeleines means considerably less wasted executions and clock cycles. On shorter pipelines the benefit is much smaller and might not be worth the transistor costs. <mihi>now tried with cheating without bootsttrap, and the resulting mes-gcc binary fails with "unhandled exception: unbound-variable: (srfi-9-struct.mes)". Now will cross-check with 0.25 to make sure I did not break anything locally. <muurkha>I learned about another one yesterday: an SSD connector standard <mihi>muurkha, that is usually called M.2 (if you do not want to call it NGFF) <oriansj>muurkha: well that was kind of the idea, M2-Planet is supposed to be the most easy to replace and forget about C compiler ever made <muurkha>oriansj: this paper suggests that it reduces branch mispredictions by 7.6% on SPECint 2017. is that what you meant by "very large"? <mihi>janneke, sorry for the noise, I must have broken something locally, 0.25 also does not build any more for me :( <oriansj>muurkha: in terms branch prediction yes; as the last 10% is a great deal harder to get than the 90% correct branch prediction rate <muurkha>yes. but this is maybe 0.8 of the last 10% <muurkha>all this insecure overcomplicated branch prediction bullshit makes me want to do a barrel processor <mihi>janneke, entirely my fault indeed. Had some problems with symlinks in repos so did "git config -g core.symlink=false", and forgot about it. Now my git did not checkout symlinks any more (instead plain files), but srfi-9 is a symlink. Sorry again. <mihi>GoogulatorMobile, was not even aware that WSL2 supports nested virtualization... So it should be similar to KVM inside Hyper-V. Do you use Windows Defender Core Isolation? <mihi>so in case you can reproduce, Microsoft may be interested. :) (In case you can reproduce on KVM on bare metal, KVM might also be interested...) <mihi>GoogulatorMobile, Googulator: Some performance stats: On my machine, building mes 0.26 --with-bootstrap using Guile takes 45 seconds for me while --with-bootstrap using Mes takes 21 minutes (about factor 20). Mes 0.25 was about 40 seconds using Guile and 12 minutes using Mes (about factor 18). In both cases, mescc is used to build both meslib and mes-mescc, in one case mescc runs on mes-gcc and in the other it <mihi>But that does not necessarily mean mes is at fault. Mescc (or rather nyacc) is notorious for repeatedly iterating over huge alists (instead of hash-tables), and assoc-ref, assq-ref and friends are heavily optimized in Guile and not in Mes. <mihi>For 0.25 I built a mescc version where all assoc and friends inside mescc are replaced by hash tables (not so easy as mes 0.25 only supported strings and symbols as hash table keys), but it did not make any noticable difference. Nyacc has about 100 usages of assoc functions too, and I did not try to replace those so far. <mihi>(instead of factor 20 I wanted to write factor 28) <GoogulatorMobile>fossy: successful bootstrap on both baremetal and qemu using the patch I posted earlier! <GoogulatorMobile>Note that the snippet has a major weakness - it can't handle things like redirection or piping <GoogulatorMobile>But there's a workaround: "tee tmp.sh", Enter, Ctrl+D, type any command that doesn't work via the snippet shell itself, Enter, Ctrl+D, "bash tmp sh", Enter, Ctrl+D, "rm tmp.sh", Enter, Ctrl+D <GoogulatorMobile>Indeed, you can write entire complex shell scripts to file using "tee" this way, and redirection works within those <matrix_bridge><Andrius Štikonas> Where are you using tee? Before interactive bash? <Googulator>fossy: testing bootstrap from a USB drive now (not the trusted one yet, just a plain old cheap Kingston), it now does boot with the new builder-hex0 stages, but later in early Linux, I did encounter an issue: on USB, /dev/sda doesn't show up immediately <Googulator>needing a wait loop in move_disk similar to the one in get_network <Googulator>luckily, I had the Bash trap enabled, so I didn't have to restart, it just needed manual intervention <stikonas>oriansj: looks like we overdid zeroing of uefi headers... I'll have to try to find out what causes it not to be recognized on one of my machines <Googulator>well, an older version of it - waiting for a definitive "no longer draft" version before I port it again <stikonas>I was briefly looking at UEFI bootstrap now... <stikonas>first I need to fix current state stage0-uefi on the machine that I've got <Googulator>maybe it would make more sense to port builder-hex0 to uefi, and then proceed with the posix bootstrap path <stikonas>yeah, that was what I was thinking earlier today <stikonas>but even current stage0-uefi doesn't run on all machines <stikonas>it runs on my very old laptop (but that laptop also has legacy mode, so can just use builder-hex0) <stikonas>but the newer one with coreboot and edk2 payload only supports UEFI <stikonas>and doesn't even run any of the efi apps from stage0-uefi (except from Development folder) <stikonas>so must be something with the PE32 header... <Googulator>you can also run a uefi environment within your OS, compiled straight from edk2 <stikonas>after that I'm thinking that we could write a loader <stikonas>that reads all the sources and creates builder-hex0 style "initramfs", loads a port of builder-hex0 with src disk into memory <stikonas>Googulator: I could also just bisect the header... <stikonas>or maybe simply go over the fields and see which field causes the hang when it is zeroed <stikonas>we have a good working PE32 header from clang <Googulator>As for making an srcfs, wouldn't it be better to just let the port of builder-hex0 access the UEFI ESP file system? <stikonas>but how can we implement rest of the stuff? <stikonas>maybe we can, I just haven't thought though it <Googulator>Anything that deals with files or memory allocation can be proxied through to UEFI, the rest handled internally like builder-hex0 does now <stikonas>though the nice thing about starting with current stage0-uefi stages and postponing builder-hex0 to a bit later, is that we can then write it in M0 or C/M2 <Googulator>also, uefi will need a real kexec system call (or equivalent) <Googulator>or some other way to call ExecBootServices at the right time <stikonas>so we would have to fix 64-bit bootstrap steps <stikonas>which in practive is just first build of tcc <Googulator>Isn't it possible to drop back to 32-bit outside of UEFI calls, like how we now switch between real and protected mode? <stikonas>I'm not that familiar with these very low level x86_64 things <Googulator>The Linux kernel can certainly access UEFI runtime services of mismatched bitness <stikonas>I can do basic assembly or hex0 coding on x86_64 but I have not looked at switching modes <stikonas>runtime services definitely should be available <Googulator>one thing to keep in mind, you need boot services or a driver for text mode output <stikonas>oh yes, otherwise builder-hex0 can't print... <Googulator>well, in theory you could set up a text mode screen buffer, and use it later (that's how UEFI framebuffers are usually handled, except that's in graphical mode) <stikonas>yeah, I think you are right about Fiwix being good step to call ExitBootServices <Googulator>they do for graphical frame buffers, but maybe not for text mode <stikonas>anyway, there are so many things going on in bootstrappable... <stikonas>and there is risc-v things that need doing too... <stikonas>we can only bootstrap up to bootstrappable tcc only... And only on posix <Googulator>And "on posix" is a massive limitation... you're trusting a binary kernel, and potentially more <stikonas>yes, but before we can have posix bootstrap working, there isn't much point of having bootstrappable kernel <stikonas>well, maybe still good for work parallelization... <oriansj>Googulator: yes, it is but progress for the risc-v port; once completed it will be much easier for someone to port builder-hex0/fiwix to risc-v and carry it forward to completion. <oriansj>So a great deal of work remains; despite the excellent progress we have been making across the board. <oriansj>muurkha: yeah, barrel processors definitely appeal to the stop the dangerous speculation desire but their performance is 1/3 to 1/5 of non OoO processors as Sun Niagara processor demonstrated very clearly <muurkha>their performance is usually worse on single-threaded code <muurkha>but their throughput is usually higher