IRC channel logs
2022-08-18.log
back to list of logs
<aggi>meanwhile, bootstrapping dev-lang/perl with arm-tcc <aggi>horrendous, a bunch of virtualized memory allocation routines and macros, which trigger a segfault when compiled with tcc <aggi>in case anyone is interested: sv_inline.h #define new_body_from_arena is one such monstrosity <aggi>with some initial hacks miniperl compiled with tcc doesn't segfault anymore... doesn't work yet as it should <aggi>seems new_body_allocated / new_body_from_arena virtual memory management needs a re-implementation for tcc compiler <aggi>my current reasoning is, i wanted to remove perl entirely, yet then autotools/automake etc. are gone, and perl too is required inside Kbuild system somewhere -> horror <aggi>another idea is, to downgrade dev-lang/perl to a version which didn't implement the weird virtualized memory management <aggi>which is some version from year 2005 ... not sure how far i could get with this, if anywhere at all <aggi>conclusion: currently dev-lang/perl cannot be bootstrapped with tcc compiler, and it cannot be removed from the dependency graph without a total collapse <aggi>and even if i can hack this perl bootstrapping problem, even then i do not feel confident about any of this <aggi>linux kernel build depends on perl, autotools do <oriansj>stikonas: good point and honestly it wouldn't be much work to do things like: DEFINE mov_eax,_[eax] 8B00 <stikonas>yes, something like this is not too hard <stikonas>probably need to update it in the comments of hex apps too if we are going with this... <oriansj>just takes time and someone willing to do that work <oriansj>well changing out M1 defines does run into a problem for cases like mov 0x100, ebx <oriansj>as the binary encoding would put the 0x100 after the ebx <oriansj>and thus would have to come after for the M1 syntax <oriansj>so it couldn't map directly to nasm syntax nor at&t <oriansj>or 891d 00010000 => mov dword [0x100], ebx <oriansj>something like STORE_EBX -> [%0x100] could be done by just DEFINE -> and DEFINE [ and DEFINE ] <oriansj>or if someone wanted to make it super clean, it could be redone in octal but it would just be a bunch of work for a cosmetic enchancement <stikonas>define names is mostly just find-replace-all <stikonas>and if we don't go fully nasm "mov dword [0x100], ebx" can be done mov_dword_[###],_ebx %0x100 <stikonas>which might still be clearer and easy for M0 to parse <Hagfish>aggi: "miniperl compiled with tcc doesn't segfault anymore" that's fantastic <Hagfish>miniperl seems like "real software" for the purposes of testing tcc, so that's an important milestone <aggi>Hagfish: it doesn't work yet <aggi>tested a version rollback from perl-5.36 go perl-5.24 (year 2016) ... problems prevail <aggi>it's safe to assume perl5 hasn't been bootstrapped with any alternative compiler such as tcc in 10years <stikonas>aggi: we bootstrap perl5 in live-bootstrap <stikonas>it involved rewriting some perl scripts in awk but then we can build miniperl without any pregenerated files <stikonas>didn't go further since that is mostly enough to run gnu autotools (except for very latest versions) <aggi>thanks for the hint stikonas <stikonas>I think perl 5.8.x should be buildable with tcc too, not sure about the rest... <aggi>just checked autotools-2.71 dependencies... >=dev-lang/perl-5.6 <aggi>and alot more potential trouble, when blocking at an autotools/automake version within the entire portage tree <aggi>nonetheless, i am willing to do this <stikonas>and there is more recent commit bumping it to 5.10 <aggi>i'll give perl-5.36 another night of debug and hacking... if this fails i'll rollback perl <aggi>apropos dilfridge@gentoo.org had banned me from #gentoo-toolchain <aggi>ideally the entire autotools/perl can be removed, however then almost the entire tree which i could harvest with gcc47 and a c-only toolchain, this tree of 600 ebuilds will collapse to almost nothing <sam_>iirc the patch we used originally was 5.6-compatibel still <sam_>then paul changed it because he didn't mind bumping the perl dep <sam_>i suspect that's why we forgot to then change the ebuild <sam_>autoconf-2.71 has :$PERL -e 'require 5.006;' <stikonas>well, it might be automake that needs newer perl <stikonas>or maybe we just didn't have enough perl modules... <sam_>i forgot the patch wasn't backported to 2.70 <stikonas>in live-bootstrap we build miniperl and some modules manually <aggi>automake DEPEND=dev-lang/perl without any version tag at all <stikonas>so it's not full perl until much later where we have gcc <aggi>well, gcc47 can be compiled/bootstrapped with tcc, however i don't want to anymore <sam_>fixed autoconf, looking at automake now <sam_>or I can let you tell me the answer :) <aggi>so, regardless of any success with hacking perl-5.36 for example, a perl-5.6 deployment may be preferrable anyway <aggi>up until recently, i could keep the gcc47-toolchain package set and tcc-toolchain one almost identical <aggi>i'll minify the dependency graph against dev-lang/perl next <aggi>i am aware of at least one ebuild which might hurt: sys-boot/syslinux which is necessary to create bootable amd64.iso images <aggi>sorry for the noise, need to think about this. <aggi>not at least the gentoo archives contain perl-5.6/5.8 anymore ... <aggi>that's the last one in the gentoo archives: perl-5.10.1.ebuild <aggi>ups, nope ... perl-5.8.8-r8.ebuild <sam_>everything back to original 1st gentoo should be in the cvs archive <aggi>anongit.gentoo.org/archive/repo/gentoo-2.git <sam_>that's an archived version of the repo just before migration to git <sam_>if you look in the history, you will find older stuff <aggi>FYI perl-5.8.8-r8 compiled with arm-tcc: ./miniperl -Ilib configpm --heavy=lib/Config_heavy.pl lib/Config.pm -> Segmentation fault <Hagfish>aggi: "sorry for the noise", for what it's worth, i appreciate reading the thought process that goes into this important work <Hagfish>certainly it seems better to me than a silent channel, but i suppose other people might prefer channels to only be used for specific help requests, or whatever <oriansj>well this channel is for everyone interested in bootstrapping to learn, share and engage as a community in a positive, healthy and informative manner. So there is a good deal of freedom to share <oriansj>unless anyone has a better plan for a managed git repo solution with support for custom domains <oriansj>and the gemini support looks interesting. <aggi>side-note: perl-5.8.6-r8.ebuild compiled with arm-tcc -> miniperl does NOT segfault anymore, which seems consistent with another gentoo bug report mentioned <aggi>./perl --version "This is perl, v5.8.6 built for armv6j-linux" ... bootstrapped with tcc :) <aggi>and it's noteworthy, perl bootstrapping with tcc required a version-rollback of 17 years(!) <aggi>anyway, blocking at gcc47 c-only toolchain (year 2013) and perl-5.8.6 (year 2005) for bootstrapping toolchain and userspace, this is a gigantic trouble source to synchronize against upstream <aggi>currently, perl-5.8.6 did resolve against the dependency graph, with some minor exclusions (sys-boot/syslinux which requires backtracking still) <aggi>with syslinux i already blocked to an old version _without_ UEFI support, i'll never follow along with this <aggi>so, chances are, any syslinux version _without_ UEFI support can be retained with perl-5.8.6 <aggi>i don't feel confident with the perl version block to year 2005 <aggi>currently, with the tcc-toolchain (no binutils nor gcc nor llvm no elfutils required anymore at all), there is only two blockers remaining <aggi>musl-libs and linux kernel... <aggi>once i got those two repaired, an entire linux distribution will be available, with a tcc-toolchain only, fully separated from GNU and LLVM <Hagfish>aggi: wow, that'll be a huge achievement <Hagfish>(i must admit that i worry a bit about toybox's understated or inadvertent goal of making an ecosystem without copyleft software, for the benefit of large corporations) *blockhead scrolls back ... likes "separated from GNU" <Hagfish>it's good to avoid monoculture and single points of failure, but i think that the free software ecosystem we currently have is potentially quite fragile given all the commercial interests against it, so i'm reluctant to undermine the incentive structure that has got us this far <aggi>Hagfish: toybox project is unrelated to the toolchain chosen <aggi>although, given the situation surrounding coreutils/util-linux/iputils etc. an alternative to those is relevant <aggi>i am not concerned about copyleft nor GPL compliance at all, i am concerned about the _technical_ qualities of GNU toolchain and buildsystem <aggi>this is the reason why i want to remove GNU components, in particular gcc,binutils,autotools,bash etc.; and llvm isn't altnernative either <bauen1>aggi: iirc musl can be built with tinycc, except for a few tiny assembly snippets, that could probably be hand-assembled. Or are there other bigger issues ? <aggi>yep, this is what's hitting with musl: crt/arm/crti.s:1: error: known instruction expected <bauen1>aggi: but it's only a few (iirc 1-10) assembly instructions that tinycc doesn't support, right ? <bauen1>like the failure to assembly those instructions isn't masking an issue further down ? <aggi>yes arm-tcc assembler doesn't digest this with musl-1.2.2 arch/arm/crt_arch.h <aggi>bauen1: i think it's a syntax problem, some of which are trivial, with arch/arm/crt_arch.h i got no clue yet what the problem is <bauen1>aggi: not really, I once tried to build musl for i386 with tinycc, but I didn't actually need the multithread etc. support / fenv /... it had problems with, so I just deleted those lines :) <aggi>so far it seems tcc assembler could digest those after a minimal tweak: crti.s crtn.s <aggi>the inline assembly one in arm/crt_arch.h remains cryptic <bauen1>aggi: I would start a binary search for the line that causes it to fail, i.e. comment out 50%, then 25% etc... until you have the exact line (or rather instruction) it chokes on <bauen1>aggi: it might even just be that it doesn't like a register alias used there <bauen1>aggi: or a label, those instructions seem rather "common" <bauen1>aggi: And it's very nice to see that a (quite usable) userspace using just tinycc as toolchain is possible :) <bauen1>aggi: I am wondering, how much did you already have to patch tinycc ? (Or what exact commit of tinycc are you using) <aggi>bauen1: i don't see any problem with tcc toolchain on git HEAD <aggi>in total i had to pull in 300 ebuilds into my overlay, didn't count how much of this was related to tcc <aggi>another one, seems _Complex extension required for musl-libc... ouch <bauen1>aggi: yeah, and tinycc really doesn't support that, so you need to (somehow) build without it (probably drop to an earlier C-Standard) or add support for it to tinycc <bauen1>as in, doesn't support it at all <aggi>the _Complex one isn't that critical, not sure if it should be a compiler/language extension after all <aggi>i mean, it's important, but feasible <bauen1>It's one of the (few) missing features for tinycc to be (iirc) C11 compliant, so it would be a nice to have <aggi>reading about it, was mentioned as C99 <oriansj>well C has many great standards, none of which are perfectly followed by any implementation in existance; Thank you undefined behavior <theruran>aggi: I am wondering to what GNU toolchain technical qualities you are referring? I am really curious. <aggi>theruran: first and foremost, total lines of code .... too much <theruran>I just built libunistring and was somewhat shocked to find it's a mess of macros and it pulls in gnulib for its actual functionality (?) <sam_>libunistring + gettext are particularly bad for this because the author is one of the maintainers of gnulib <sam_>and he freely moves functionality in/out all the time <theruran>I just wanted to find the algorithms for Unicode normalizations <theruran>downloaded the source -> didn't get the source :P <aggi>libunistring was one reason to wipe GNU gettext/intltool from the tree... which was possible btw. <aggi>neither bdep nor rdep, it's wiped from my tree <aggi>i am well aware of the disadvantages, yet enough is enough, and i've had enough of GNU <aggi>for fun i dumped libc/sysdeps/linux/arm/crt1.S from uclibc-ng into arm-tcc assembler ... similar problems <nimaje>oriansj: hey, UB makes it easier to implement a standard as you can do whatever you want for those cases <aggi>strangely, this is too much already, a simple load instruction: ldr a2, 1f <aggi>ok, 1f jumps to a label, got that, seems to work <aggi>however the identifier a2, no clue, what register this shall on aarch32 <aggi>arm-tcc doesn't recognize it <aggi>then, i'm trying to navigate developer.arm.com, with w3m browser... fun <aggi>so, for bootstrapping, at the ASM level: i got a GNU assembler which i couldn't study, it's too much, aarch32 syntax, which i don't understand (yet), and an assembler which doesn't digest a simple ldr a2 1f <aggi>syntax doesn't match regardless, the jump-label parsing fails too <aggi>and this isn't the best way to practice asm <aggi>i'll do this for now, hook into binutils-as, and hack the missing _Complex extension instead, for musl-libc to pass with arm-tcc at least <aggi>aarch32 asm requires extensive studies first, before i can rewrite this <aggi>ok, checked dietlibc as an alternative.... this too contains aarch32 ASM syntax which arm-tcc cannot digest <aggi>which, raises a principle question, a fundemental one, how to approach ASM level of bootstrapping <aggi>because, it is irritating, if some aarch32 "RISC" assembly requires a toolchain with a dozen millions of LoC <aggi>and in practice, although aarch32 may not have been the right choice to begin with, arm-tcc got just that, an aarch32 assembler <aggi>yet, this assembler, couldn't yet digest aarch32 asm from musl-libc, uclibc-ng, and dietlibc <aggi>and vendor-locking into any ASM syntax for any ISA is the problem here, to invest years of work to learn just that <aggi>i did finish 99.9 of builds with a tcc-only toolchain already, 598 passed compile-time, runtime only random tests seemed ok too <aggi>however, those 0.01% of builds relying on aarch32 (or whatever) asm which is NOT supported by tcc... <aggi>and, if i wanted to dig down deeply into ASM, i think a gigatron ttl or Z80 or whatever is a better choice <aggi>given, one idea of mine, was maths, something which doesn't get into my way when programming <aggi>in this regard, Ti89 basic was rather intruiging, however, this was never open-sourced to my knowledge <aggi>hence, this doesn't pass the supply-chain verification, bootstrapping <aggi>and what remains then at the low-level realm, z80 got the appeal of 40years stability with its ASM <aggi>and gigatron ttl got some offering too <aggi>and then, i'll disconnect internet, fully, for the rest of my live, never again <aggi>and grab my maths book, some asm book from the library, and play with this <aggi>terminating my ISP contract, will save me another 30Euros/month <aggi>sadly, the east-german semiconductor manufacturing is defunct since 1990, add to this <aggi>i mean, the aarch32 ASM hacks, this can be solved, what can't be is some very fundemental problems <aggi>and the moment when, i cannot read developer.arm.com with w3m browser anymore.... discuss <nimaje>oriansj: hm, for a propper malloc for M2libc maybe steal the one from scc libc, seems to be a small freelist malloc backed by brk (http://www.simple-cc.org/) <aggi>src/libc/arch/arm/crt-posix.s doesn't pass with arm-tcc <aggi>seems binutils/gas specific, or any other standard i am not aware of <theruran>how is the tcc project at accepting patches? can it be patched to accept the ASM that aggi needs? <stikonas>theruran: it's a mess, I think anybody can push code to tcc without any review <stikonas>and there hasn't been any stabilization release for a long time <rkeene>There's the mob branch, at least <stikonas>well, that's what I'm mentioning, mob branch is anybody pushes whatever they want 