IRC channel logs
2022-09-02.log
back to list of logs
<oriansj>stikonas: well that depends on a couple things. For example if it is something that you will be doing in other pieces a function tends to be easier reuse. if it is something unique to hex0 then inline would benefit from being smaller. Also if 2 inlines is 24bytes (12bytes per) I can't imagine that a function call would be larger <stikonas[m]>Well, it's reusable function, e.g. open_protocol which I'm already using in hex2 and catm <oriansj>and yes the offset calculations do suck <stikonas[m]>But maybe larger binary with fewer offsets is easier to inspect <stikonas[m]>I think I can verify 13 bytes quicker than 2 offsets, especially since we have 2 copies of the same stuff <oriansj>If you prefer doing it that way, it is fine <stikonas[m]>Anyway, it will be maybe 2 weeks until I'm done improving hex1.hex0 <oriansj>and you may wish to add a comment to indicate the duplication <oriansj>that way if someone else wants to cut the size of the binary, they know of an easy place to do so. <stikonas[m]>Another way to cut binary sizes in uefi is not freeing any resources. But unlike posix, uefi won't clean up on exit, so at least for now I'm doing full cleanups <oriansj>well pushing size limits with UEFI is probably not a good idea yet as there seeems to be a boatload of mines <stikonas[m]>Exactly, hence I'm trying to be compliant and just get everything running <doras>stikonas: it seems that live-boostrap's final gcc doesn't support thread-local storage. From a quick search online, it may be because our assembler from binutils is missing support for it. <doras>gcc supposedly detects this at build time. <doras>Thread-local storage is required for freedesktop-sdk's elfutils. <oriansj>doras: if binutils don't support thread-local storage, which assemblers do? <aggi>doras: there is two variants of TLS i think, C11 and the GNU __thread extension <doras>aggi: in this case we need __thread <aggi>the elfutils version i fiddled with insisted on GNU __thread (which tcc didn't have) <aggi>doras: ok then, because gcc47 (if this is the live-bootstrap version) does support __thread, and failed with C11-tls <doras>aggi: interesting. I can a configure failure about it. <aggi>both __thread and C11-tls are a trouble-source, gladly i haven't got any dependency against this remaining here <doras>I suspect it may be failing because of something else. <aggi>doras: anyway, i wasn't aware elfutils contained anything which binutils didn't offer, to provide whatever freedesktop-sdk may need <doras>aggi: elfutils is required for debugedit, which is used for debug symbol manipulation. This is currently done for every package, even early in the bootstrap. <doras>But I suspect I hit a red herring. The actual cause of the failure is something else. <aggi>weird. doras: thanks for the hint. <aggi>no intention to distract off-topic, nonetheless, why would they need to mangle debug-symbols? <aggi>don't care, got no freedesktop.org/redhat stuff installed anymore <doras>aggi: I haven't looked into it much, to be honest. <doras>The issue is that elfutils insists on build-id support, and live-bootstrap's binutils is too old. <doras>I think it was first added to binutils 2.18, and live-bootstrap only has 2.14. <oriansj>well thread-local storage does enable performance via multi-threading so that is probably the reason for why elfutils might use it <aggi>i don't think elfutils contains performance-critical sections which benefit from this significantly <aggi>in any case __thread and C11 TLS create a vendor-lock and/or version-lock against particular toolchains <aggi>since gcc47 hasn't got C11 TLS to my knolwedge this wiped a recent net-dns/bind, and tcc got no __thread extension hence elfutils are gone <aggi>of more or less relevant software packages <doras>stikonas: I can try to add a newer binutils for a start, but I'll need some pointers. Are you familiar with a newer version which is acceptable for live-bootstrap and should be able to build with its existing packages? If so, when in the bootstrap would you prefer to build it? <oriansj>aggi: it is rare for software developers to know exactly what changes would produce significant performance improvements and it is a skill that can only be learned by a great many failures. <doras>stikonas: is it possible that dlopen doesn't work in live-bootstrap's final image? <doras>These are important modules, and one of them, `_posixsubprocess`, is actually used in Python's own "make install", resulting in a failure at this stage. <oriansj>doras: yeah, you'd need glibc with dynamic linking enabled <stikonas[m]>Though, I was not able to get it working but I didn't try too much <doras>Sorry for the mirror links :s <doras>So we need both support for dynamic linking in musl and not build statically as far as I understand. <doras>And this is a requirement for a newer GCC, because it needs Python. <stikonas[m]>doras: not sure. But I'm now travelling, can't check in detail <doras>stikonas: never mind then :) <oriansj>doras: bootstrapping is always just a long series of "lots of work" but it is work that needs to be done if only once <aggi>with musl-1.2.x dlopen is possible <oriansj>well dlopen and dynamic linking feel like a vestigial idea which could be replaced by the kernel checksumming pages of memory and doing deduplication on the read-only pages <oriansj>especially as one can statically link pages to static memory addresses in a way to ensure they are the only contents of the page(s) to make it even easier for the kernel <oriansj>but that might just be my bias to the belief that we always will have the source code and will be building the binaries ourselves with full knowledge of what is available. <doras>oriansj: storage on disk and size during network transfers is affected as well, not only RAM utilization. <doras>But yeah, it would have been nice to have deduplication of read-only memory regardless. <oriansj>doras: the size of the binaries wouldn't matter for network transfers if we were only distributing source code <oriansj>but I will grant you it does increase disk usage for filesystems that don't support deduplication <oriansj>but on Copy-On-Write (cow) merkle tree filesystems; deduplication is a rather cheap lookup to do (as it is cheaper than writing to disk assuming the lookup takes less than 10 microseconds) <sam_>in general if I can ever help, please let me know <sam_>if it's something you think we may know something about, etc <oriansj>sam_: honestly the most helpful bit would be to tell me about everything I did crappy and could do better <oriansj>as I'll probably be going into that a bit blind and make some really dumb mistakes <oriansj>and probably need to make some changes to make it easier to update to newer linux kernels <sam_>for line 51: there's an easier way to do kernels but it depends on personal preference <sam_>nowadays you can build the kernel as a normal package, which I love <oriansj>sam_: well my preference is a simple to follow procedure to get people to a consistent and minimal development environment <oriansj>that way, regardless of what distro they start with they can easily get setup to be productive on stage0 and above work <oriansj>so emerge -av sys-kernel/gentoo-kernel looks like it'll save me a good bit of work on making kernel config files <oriansj>and it'll still be build so it shouldn't be hard for a user to go back and customize further <oriansj>now to tweak it so that they are under 80char lines <oriansj>and probably should add a note about the different alternate architectures they could select at that point <sam_>looking good so far oriansj <sam_>we could make a note about openrc or systemd, as well as musl options but not sure if that's necessary <oriansj>sam_: I am also testing the steps as I go so I'm going to be a bit slow <oriansj>a note (or probably expanded steps) about the init seems like a reasonable thing to do <oriansj>and alternate libc seems like a bigger bit of work <oriansj>although I probably should write up something about those things to help people too <oriansj>(might be separated out notes like I did for virtualbox and vmware)