IRC channel logs
2023-11-24.log
back to list of logs
<notgull>Seems complicated to bootstrap up to UEFI. I feel like it would involve writing a lot of drivers in hex0 before anything more advanced can be used <vagrantc>got as far as + /<<PKGBUILDDIR>>/bin/mes --no-auto-compile -e main -L /usr/share/guile/site/3.0 -C /usr/lib/guile/3.0/site-ccache /<<PKGBUILDDIR>>/scripts/mescc.scm -- -m 64 -c -D HAVE_CONFIG_H=1 -I ../lib -I ../include -I ../include -I include -v -static -nostdinc -nostdlib -fno-builtin -o lib-mes-fdgetc.o ../lib/mes/fdgetc.c <notgull>Actually, depending on the hardware, you might be able to fit everything up to M0 in before you have to do DRAM initialization. So I dunno <samplet>There’s the “cache-as-RAM” trick that coreboot uses. <samplet>At least on some platforms, the DRAM initialization code runs in a pretty comfy environment. <notgull>Although I'm not planning on trying. That sounds even more painful than usual <stikonas>notgull: well, if you have baremetal system with no usable firmware, then you definitely don't want to use UEFI... <stikonas>something closer to builder-hex0 but with different I/O <stikonas>i.e. if you can get input sources into RAM <stikonas>then this modified "builder-hex0" is basically all you need <stikonas>you might have to hardcore addresses rather than using nicer file open operations <stikonas>but other than that it's not that different <notgull>Might dig around in my garage to see if I can find bootstrap capable hardware <notgull>I know my grandpa has an old Apple 2 computer lying around somewhere <fossy>(fwiw, bootstrap capable hardware = a possible target for a possible bootstrap, there are no truly baremetal bootstraps yet) <fossy>stikonas (pder Googulator too but they're not here), what is this bios memory map thing with fiwix, and where's googulator's fix? i suspect i might be running into it... <stikonas>I think BIOS reserves some memory for some stuff <stikonas>and it passes it to OS, so that OS does not overwrite it <stikonas>might be memory to access devices, I'm not really sure <fossy>that would check out... i added an additional disk to qemu, and getting random segfaults in fiwix stage <fossy>well not random, consistent, but completely nonsensical segfaults <fossy>does this actually have something to do with PAE, or is it coincidental that fixing PAE fixes the mem map? hm <stikonas>not sure since I haven't looked at it, but likely yes <Googulator>fossy: just saw your simplify PR, and it raises a few questions <Googulator>Most importantly, does this now use disk access in Fiwix? <Googulator>I'm asking because Fiwix's disk support is extremely limited - the ony way I could get Fiwix to see my SATA drives on the bare metal test system was to set SATA mode to "compatible IDE" <Googulator>While builder-hex0 only manages to boot if SATA mode is set to AHCI <Googulator>If this isn't a limitation specific to my board, then that leaves no SATA setting that can complete the bootstrap <Googulator>& I can't use a real IDE drive either, as my board, like most from that era, uses a JMicron IDE controller that's not supported in Fiwix <oriansj>that can complete the bootstrap without human interaction <oriansj>as you could do AHCI mode and maanually set the value to compatible IDE prior to booting Fiwix <Googulator>(or you have just erased the sole copy of the Fiwix binary from memory) <oriansj>Googulator: thank you for that correction. <matrix_bridge><Andrius Štikonas> Yeah, I would definitely want to run bootstrap on hardware where fiwix can't use disk directly... <GoogulatorMobile>To add to the earlier discussion about finding the ideal "true bare metal" bootstrap HW: RK3588 may be as good as it gets, if you're willing to trust ARM Cortex cores and a 64kB (IIRC) embedded mask ROM <GoogulatorMobile>The only real obstacle right now is LPDDR4x training, which isn't yet supported in u-boot SPL or any other open source project, and is currently done via a blob :( <GoogulatorMobile>Until you do DDR training, you have 1MB of onboard SRAM at your disposal <matrix_bridge><Andrius Štikonas> GoogulatorMobile: how is it better over rk3399? <matrix_bridge><Andrius Štikonas> rk3399 can run blobless (with some limitations) <GoogulatorMobile>That's even better then - assuming we have the same level of documentation <matrix_bridge><Andrius Štikonas> Normally bootrom loads u-boot TPL, then it returns back to bootrom, then it loads SPL which then loads full uboot <matrix_bridge><Andrius Štikonas> Actually there is also ATF between SPL and uboot proper <fossy>GoogulatorMobile, stikonas: yes, it does use disk access in Fiwix. there's a few tradeoffs we have to consider here <stikonas>hmm, I'm worried that will break it for most available hardware... <stikonas>so it will be mostly limitted to qemu and some old hardware <fossy>pros: allows for effectively infinite expansion of pre-Linux; allows for the use of lower-RAM systems than 4G; allows for use of repo in fiwix stage <fossy>cons: breaks a lot of widely available bare metal hardware <fossy>Linux kernel as seed does still work <fossy>so the question is do we continue to work with the constraints of losing the pros to bring the kernel bootstrap to a wider range or bare metal systems? <fossy>an alternative tradeoff could be giving the option for more distfiles to be brought into the initrd and thus allowing the use of no disk... <GoogulatorMobile>The question is which one is easier to implement in Fiwix; AHCI, or ext2 autoexpansion <fossy>how do you mean, ext2 autoexpansion? <GoogulatorMobile>Then, when Fiwix sees that the FS is smaller than the ramdisk it's on, it does the equivalent of a resize2fs before mounting it as root <fossy>i'm a bit lost. why do we need that? <GoogulatorMobile>Because the limiting factor for initrd size is builder-hex0's memory map <GoogulatorMobile>Specifically, beyond 1152MB (on qemu - exact limit is BIOS-dependent), builder-hex0 will allocate part of the initrd in MMIO space <GoogulatorMobile>(1280MB is tolerable even in qemu with the double Linux tarball removed from srcfs, but that's only a temporary reprieve) <fossy>i am keeping in mind that expansion of the initrd also means an increased load on RAM requirement <GoogulatorMobile>There are very few motherboards out there with a true legacy IDE controller that can also support 4GB RAM <fossy>the loss of hardware support is a bit more than i am comfortable with considering a few of these issues, true <fossy>we don't need 4GB RAM with a disk though <fossy>i haven't tested exactly what it is but i suspect its around 2GB <GoogulatorMobile>The only boards that supported 4GB before the industry transitioned to AHCI + JMicron were expensive server ones, which usually have SCSI, not IDE <GoogulatorMobile>I wonder if that's the reason kernel bootstrap is failing with the refactor <fossy>hmm, could you elaborate a bit? fiwix is able to mount the disk at least in the refactor <stikonas>well, yes, older hardware might not support 4 GiB... <GoogulatorMobile>But finding a motherboard old enough to have true legacy PCI IDE, but new enough to support 4GB or even 2GB RAM <stikonas>x86 bootstrap will be quite broken in 15 years <stikonas>so we'll either have to rewind clocks for bootstrap <fossy>ideally in 15 years we'd be using riscv bootstrap or some other more open chip :P <stikonas>GoogulatorMobile: so how much space we have if we remove double linux tarball? <fossy>how did the kernel bootstrap *ever* work if the limit is 1152MB for the initrd? <fossy>was the ramdisk somehow larger? <stikonas>1350 MB is free space in 4 GiB memory, not just initrd <fossy>that would make more sense lol <stikonas>I guess in the early bootstrap mes uses a lot of memory... <fossy>it's still <1GB isn't it? i could be fairly wrong about that though <Googul8rMobile>Peak usage stays below 1152MB on the disk with the current kernel config on master <fossy>yes, because of mem map, correct? <Googul8rMobile>1350 is the total space available for initrd to grow, without hitting MMIO in builder-hex0 <stikonas>we don't have any e.g. ethernet drivers, probably not much storage either <muurkha>fossy: 1 GiB is still 16 times the size of the RAM on the machine I compiled Linux on most <fossy>muurkha: you compiled modern linux on 64MB RAM?? <stikonas>still linux kernel with reasonable amount of drivers is not that big <fossy>Googul8rMobile: thank you for looking into these bare metal issues, by the way, it has been very enlightening and helpful as to some of the challenges we face in that regard <Googul8rMobile>Modeen x86 hardware is essentially useless for baremetal already <stikonas>we just need ethernet, storage and some display drivers <Googul8rMobile>It's _locked down_ with an option to refuse recognizing external hubs enabled <fossy>that is not intentional, i will say <fossy>the config was created as just something that worked on qemu <Googul8rMobile>Also, on bare metal, I could just patch initrd to 1280MB and do nothing else, and it would work (with the right Linux config, of course) <fossy>hmmm, that maybe does make sense given that kernel variation is one of the things we already do allow.... <Googul8rMobile>I measured 70MB of space remaining between MMIO and the 1280MB initrd in qemu (with double Linux removed) <Googul8rMobile>But the latter actually included my "bring over early artifacts" tarball in initrd <fossy>how are you measuring this, out of interest <stikonas>we probably don't need to bring that many artifacts... <Googul8rMobile>Inside the initrd, I just wrote a file until NSLoD, then ls -l on it <stikonas>e.g. rather than bringing linux kernel to sysc, you can always rebuild it again <Googul8rMobile>Between top of initrd and MMIO, I just took the address printed by kexec-fiwix and added 1280MB <Googul8rMobile>srcfs size can be reduced even more by recompressing all gzs as bz2 <stikonas>though then we can't use upstream tarballs... <fossy>yeah, i am not a fan of recreating the tarballs externally to live-bootstrap... <stikonas>fossy: in your PR you have some changes that are not really related to the whole rework, maybe those can be merged first <Googul8rMobile>I was thinking of putting the recompression into the secfs generation code itself <stikonas>well, we could do that, but then secfs generation becomes non-human doable, not sure whether that matters <fossy>I'm not a fan of that. currently you can create srcfs, theoretically, without Python. I don't want to force the use of Python <stikonas>yeah, but at least people can just take upstream tarballs and not use rootfs.py <Googul8rMobile>You already need to include binaries in srcfs, not something you can type by hand <stikonas>well, it's not like you can type rest of the source by hand <fossy>the current requirements for creating srcfs are "knowing the size of all files" and "being able to write to a disk" <fossy>that is something that can be done, theoretically, by a very primitive system <stikonas>ok, but do we really need that much extra space in the future? <fossy>stikonas: i'd argue that we would hopefulyl need *less*, not *more* space in the future (but more space isn't an impossibility...) <Googul8rMobile>Largest SPI flash and largest true parallel flash part that exists <fossy>what is the significance of 256MB <Googul8rMobile>Everything beyond that size is managed flash that requires complex firmware to access <fossy>where does removing linux 5.10 tarball get us up to? <fossy>278 > 256, isn't that good enough? (if we're talking about gap?) <stikonas>hmm, not sure if that's sustainable if we thing about other arches <Googul8rMobile>Where you cut address lines A9 and beyond, to limit accessible size to 512 bytes initially (in the parallel version) <fossy>well, kernel bootstrap has nil support for riscv right now <stikonas>but that's one of the reasons I was reluctant to switch to disk in Fiwix <stikonas>it will probably make kernel bootstrap on new arches harder <stikonas>but Mukaku said he doesn't see why it can't be ported to other arches <stikonas>kernel bootstrap stuff is complicated :( <stikonas>but of course kernel bootstrap is much more lower-level bootstrap with far less blobs <stikonas>needs driver support to just pass ethernet frames... <stikonas>yeah, all of this makes non-memory bootstrap complicated :(... <Googul8rMobile>Adding those would've probably complicate the kernel so much that we could no longer use it at that stage of bootstrap