<stikonas>fossy: I might have another hacky solution... <stikonas>replace all weak symbols with strong symbols <stikonas>and patch tcc to patch duplicate symbols instead of erroring out <stikonas>at least tcc compiles and is self-hosting <stikonas>it's at least less hacky than appending .o files <stikonas>I'll probabably finish tcc before I make PR... <stikonas>although, I think sourceforge is down now <stikonas>so heirloom tools might fail to download <stikonas[m]>Maybe I can host those tarballs and get_file can use it as mirror <OriansJ>well anyone have a better solution than this for removing the need for fixup in M2-Planet? <Hagfish>"1698 out of 1701 (99.82%) paths in the minimal installation image are reproducible!" <Hagfish>i don't think it was that good last time i looked, a week or so ago <Hagfish>those are the non-reproducible packages <Hagfish>i know gcc has been one of the packages the debian repro builds team has been stuck on for a while <Hagfish>asciidoc hasn't been a problem for debian <Hagfish>oh, maybe Debian are packaging a slightly older version (seems to be release candidates) <OriansJ>Hagfish: Ironic that the programs easiest to make reproducible are those they are having the hardest time doing so. <theruran>so is Guile 3.0 now the most trustworthy Scheme because it is bootstrapped from a small hex seed? making it more preferred as a compiler implementation language? <OriansJ>theruran: not entirely bootstrapped until we solve psyntax.pp <theruran>OriansJ: but once that's finished? btw, what is up with psyntax.pp? <OriansJ>theruran: it is guile's implementation of syntax-case written using syntax-case; so you need guile to bootstrap guile. <OriansJ>I thinkn this might be more M2-Planet friendly in the end if(0 != (0x80000000 & result)) result = (0xFFFFFFFF << 31) | result; <theruran>I guess the nanopass framework has migrated to Racket stewardship. Is Racket CS more or less bootstrappable than Guile 3? I would assume less.. <OriansJ>theruran: every scheme we have looked at that supports syntax-case depends on a implementation specific version of syntax-case written in syntax-case. <OriansJ>In effect no scheme is bootstrappable without considerable work <OriansJ>as syntax-case is a freaking monster of functionality and always 100% interpreter specific to hell <theruran>that's much worse than I assumed! D: what direction now are you all thinking of how to solve that problem? <OriansJ>theruran: well mihi is currently attempting to solve that by writing it in guile primitives but it looks like it is going to be a long project. <theruran>I assumed mes had implemented macros already <OriansJ>theruran: there is a massive difference between "macros" 'macros' and syntax-case <OriansJ>mes has implemented "macros" but not 'macros' or syntax-case <theruran>oh man.. so can syntax-case be implemented with a less powerful macro system? syntax-rules? <OriansJ>theruran: we hope; if not we might have to figure out how to do syntax-case in the Haskell subset we bootstrapped in blynn-compiler <OriansJ>in short, we are working the problem and if we hit a wall, we will try another route. <OriansJ>and another and another until we finally bootstrapped *EVERYTHING* from source. <theruran>I am looking at the R6RS standard about syntax-case <fossy>OriansJ: one thing done, another small come <theruran>OriansJ: I can't find syntax-case in R7RS-small <OriansJ>fossy: always progress here even if I am just barely scrapping together time to program these days. <OriansJ>theruran: >.< yet its required to run the standard libraries. <theruran>I thought I would find syntax-case written in something else, but I'd have to read R6RS more carefully. Someone wrote it like that originally.. :/ <OriansJ>theruran: syntax-case is the sort of "uber feature" that on one hand is considered required but on the other hand so insanely complex to do correctly that no one wants to look or touch it if at all possible. <theruran>cannot find (define-syntax syntax-case ...) 😆 <OriansJ>It was literally a PhD project if I remember correctly. Hygenic macros for scheme <theruran>I found the technical report from 1992.. <theruran>sorry it pasted that way. says there's a syntax.pp expanded form that is used to bootstrap syntax.ss <theruran>ah. the last message on the mailing list gives no reason <OriansJ>possibly because syntax-case can't be portable (unless there was a way to get lazy evaluation from an eager evaluator that I don't know about) <fossy>stikonas[m]: :DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD <OriansJ>deesix: I'll have the first M2-Planet test1000 conversion to M2libc in a little bit. The rest should be done tomorrow. <gforce_de1977>gforce_de1977: sorry, micro-optimizations are bad, ignore my comment <OriansJ>gforce_de1977: fortunately M2-Planet/cc_* handles if( and if ( the same but stylewise I could be more consistent. <OriansJ>but micro-optimizations are fine if they don't increase complexity or ability to be understood. <OriansJ>yt: could you check that I didn't screw up M2libc/aarch64/Linux/bootstrap.c for test1000 in M2-Planet please? <OriansJ>and armv7l switched to M2libc and test/common_armv7l finally pulled from M2-Planet <stikonas>pder: or do you want me to fix it later... <rjek>Am I right in thinking that stage0 isn't quite to the state where it can build Mes yet, in order to complete the "stage0 -> Mes -> tinycc -> tinycc -> gcc 2.95 -> gcc 2.95 -> gcc 4.7 -> gcc4.7 -> modern gcc -> modern gcc" chain? I'm struggling to find documentation that suggests this has been achieved (yet) ***ChanServ sets mode: +o rekado
<rjek>Ah, so the tinycc that's built can't yet build gcc 2.95? <stikonas>rjek: tcc can build it, but we need to bootstrap autotools <stikonas>well, if you accept that "configure" is source than you might be fine <rjek>Ah, autotools. Even now it comes back to haunt us. <stikonas>yesterday we just built musl with tinycc (not merged yet), which would allow us to proceed further <rjek>OK. I assume one day you'll also need to build Linux and qemu for the steps described there? <stikonas>I just use qemu and Linux from my distro <stikonas>but for now you'll need some kernel, that's true <rjek>Well. Using qemu and Linux from your distro isn't really much better than using the gcc from your distro, is what I mean :) <stikonas>for now kernel is out of scope for stage0 <stikonas>there will be more work on kernel in live-bootstrap later... <stikonas>although, we use chroot more in practive <stikonas>although, we managed to build non-interactive bash <rjek>Is there a "CI" tarball of the artefacts the state-of-the-art Mes can produce that we can directly use, rather than running the chain itself? <stikonas>and interactive bash might be possible now *rjek 's interest at the moment, selfishly, is to use the outputs rather than improve the inputs or machinery. <stikonas>well, if you don't run outputs yourself, there is no reason to trust them more than your distros gcc... <rjek>I can sha256sum them against the results of somebody I trust who has more patience. <stikonas>well, there will be guix outputs later... <stikonas>i.e. they trust configure, they use pre-generated C files (bison/flex) <rjek>Hmm, doesn't sound like using their outputs yet gains anything but a bit more inconvenience :/ <stikonas>on the other hand we might be able to jump from tcc to gcc 4 and skip gcc 2.95... <stikonas>because we managed to bootstrap modern C library <civodul>stikonas: i don't think "they" vs "we" is warranted <stikonas>civodul: yeah, sorry, I do have guix here too <stikonas>and I also sent some patches to guix before <stikonas>anyway, just different strands of work going on... <civodul>right, and clearly improvements in live-bootstrap are usually beneficial to Guix, and vice versa <stikonas>one takes a bit more technical debt but is already usable <stikonas>and we already follow general direction of Guix when bootstrapping *rjek is sadly not really interested in Guix itself, just the bootstrapping of a toolchain with which to build something else. <stikonas>it's fairly new, we really started ramping live-bootstrap a month or so ago <rjek>Our current build and integration tooling basically requires a rootfs containing a basic system + gcc from which it builds everything else. <stikonas>well, live-bootstrap only needs two tiny binaries and lots of source code <stikonas>and we'll build gcc and other essential utilities <stikonas>rjek: and actually, there won't ever by place to download all artifacts <rjek>Oh? Which aren't redistributable as binaries? <stikonas>we need to compile some very early software licensed under CDDL <stikonas>in particular that's heirloom yacc and lex <rjek>is there a plan to replace those CDDL components? <stikonas>lex gets removed fairly quickly and replaced with flex <stikonas>but we compile newer tool that is redistributable <stikonas>and yacc will be replaced with bison later in the chain <stikonas>but early parts of chain will have transient non-redistributable binaries <stikonas>in principle we can recompile them now with musl <stikonas>and same program will be redistributable... <stikonas>so we need those to break cyclic dependency <rjek>Are these CDDL tools not intermediaries? I mean, they don't end up in what you ship if you're shipping lex/yacc? <stikonas>that's why they are not big deal but we can only publish final result <stikonas>but at the moment only one intermediary is replaced <rjek>I'm interested in the final result :) <rjek>I mean if I can reproduce that final result then great, but I don't feel the need to <pder>stikonas: regarding the pull request was this the issue with reordering patched tcc and patched mes-libc? <stikonas>pder: by the way, regarding your comment, I'll replace that shell script with sourced script invoked by build function <stikonas>well, I probably need to first create some basic /dev and /proc folders in another commit too <pder>So if we built patched mes-libc before patching tcc to use static linking, would the end result be the same? <bauen1>rjek: if you're interested, the feature set of the kernel necesarry for bootstrapping is probably quite minimal, so currently i'm building up myunix up to a point where it should be self-hosting, which means can run stage0 up to tinycc+musl (well, very slowly due to lack of time) <bauen1>rjek: this kernel could be written by any college student with enough motivation after completing ~3. semester <bauen1>rjek: and after that you're scratching on the issue of trusted hardware <stikonas>pder: libc is the lowest library, it has no other libraries to link to <stikonas>pder: that patch only affects behaviour of tcc linker <stikonas>and linker is not used when building libraries <pder>ok, I will clean that up <bauen1>which reminds me that live-bootstrap should probably get gcc+bintuils to run and then cross compile all parts of the final toolchain, this way they should be identical and could even be run on another system, e.g. build a toolchain to run on / build for x86 from a live-bootstrap ran on RISC-V <stikonas>probably just simple reordering (although reorder readme too) <pder>I split the building of tcc out into a separate script because I called it twice. I can probably revert that <stikonas>bauen1: yes, there will be a few rebuildings <stikonas>and next time I call compile tcc it will be from bash... <stikonas>I tried using tcc's configure, but I think we are still missing some coreutils <stikonas>should be possible to build later with musl <bauen1>stikonas: tinyccs configure script is short enough that hand creating the resulting config / config.mk files is easy and maybe even faster if you know what you need <stikonas>bauen1: well, compiling tcc is just one command anyway <stikonas>we were just supplying those values as command line arguments... <pder>I notice now that the rootfs.sh script is downloading source every time it runs <pder>It would be preferable to have the script use a local copy already downloaded if available <stikonas>first of all it causes extra strain on mirrors <stikonas>I think we should put download to sysa/*/src and not sysa/tmp/*/src <stikonas>pder: although, CI will still download every time... <stikonas>actually I just saw failure yesterday because sf was down... <bauen1>by the way, what exactly does musl use weak symbols for ? <bauen1>if you build all objects, can't you just remove weak symbols, or are they to work around limitations of archive files and how linkers lookup the symbols ? <stikonas>bauen1: we don't remove weak symbols, we replaced weak with strong <stikonas>well, it uses for a lot of functions that I guess you can override <stikonas>even stuff like memory allocation (calloc) <bauen1>stikonas: i meant removing the weak_alias calls if there the real symbol is somewhere in musl <stikonas>musl provides more than one implementation for a few things <bauen1>i never had such problems (granted i haven't made much use of musl yet) with tcc, is that because some fix is already in mob ? <stikonas>well, once we have binutils, this hack can be removed <stikonas>I tried newest revision of mob that we can use (maybe from 2019) <stikonas>anything newer does not build on mes libc <bauen1>stikonas: can a modern binutils be compiled with tinycc ? <stikonas>but with musl we might be able to go with something newer <stikonas>I think biggest limitting factor was not tcc but mes libc <stikonas>and now with modern musl, I think we can build much more <stikonas>in fact I don't even know if we need to try to build glibc any time soon... <pder>stikonas: I pushed my update mes-crt-fix branch that reorders the mes-libc-patched and tcc-patched steps and adds a header to my patch <pder>You will want to rebase your musl branch on top of that <fossy><rjek> Well. Using qemu and Linux from your distro isn't really much better than using the gcc from your distro, is what I mean :) <fossy>live bootstrap will manage kernels once it is complete <fossy>so there will be a kernel bootstrap <stikonas>fossy: I've just converted tcc-musl into bash type sourced script. But since it's two pass script, at lot of installation is done from configure... Hopefully it's fine... <fossy>yeah, that is fine, proabbyl ill have a look <stikonas>well, running the first test (might need more if something is broken) <stikonas>after musl I'll replace flex 2.4.14 with newer... (completely removing 2.4.14) <stikonas>since 2.4.11 should be able to build latest <stikonas>fossy: argh, slight problem with scripting <stikonas>overridden stages are remembered from previous packages... <stikonas>and I removed subshell because it interfered with -e... <stikonas>I guess I can unset all functions at the end of build