IRC channel logs
2026-03-06.log
back to list of logs
<stikonas>it's somewhat slow but other than that it works <stikonas>and slowliness is something that can be worked upon <aggi>good question stikonas. i do not consider mes-cc itself "wrong", but it's the combination of it's involvement together with the need to re-compile tinycc 9times total plus another 5 re-compiles of musl-libc and another 4 re-compiles of gcc in combination <aggi>in this regard, live-bootstrap itself did a great job to document this issue explicitely <stikonas>fossy and I unified libc compilation files into larger units and that made it faster. Soon we are also getting rid of pregenerated files step, should be another speedup <stikonas>well, perhaps with new mes it will be fewer compiles needed to build initial version <stikonas>as for recompiles after that, they are mostly needed to switch to new C librarires <fossy>to be honest, i don't see recompiles as a problem, other than speed <fossy>(at least of the same version, eliminating different versions is good though) <stikonas>but it's not like multiple versions are some kind of fundamental problem <aggi>i would argue it is not feasible to resolve this issue, but that doesn't mean it's resolved by live-bootstrap if noone can <aggi>i've already pointed to the root-cause, and too the symptoms of this in detail <fossy>stikonas: yeah, agreed, the moderate disadvantage to multiple versions imo is it's just increasing the amount of code we use in the base system <fossy>aggi: that doesn't help me. i don't even understand what the issue is. how can i possibly understand the root-cause of an issue when i don't even know what the issue is? <aggi>fossy: see, i'm the type of guy who's cloning HEAD of all this, maintaing a kernel fork already for tinycc, and a complete userspace fork for TinyCC/OS <aggi>so, i'm actually doing code reviews myself all day long <aggi>i was doing this for more than two decades, down to Verilog/HDL up into highest realm of JavaEE <aggi>plus a little maths, signal processing, physics and a thousand other things <aggi>and when i say, it's too much, then it is <deesix>Please, don't start again... point concrete errors and the instructions you're following. <fossy>again, you're being totally unclear <fossy>please point to explicitly, what is too much, and give the alternative you desire <fossy>there is nothing i can do if you're just going to say, in effect, "i just don't like this" <deesix>And, again... using HEAD(s) blindly does not help when there're precise combinations to be used. <aggi>for that matter i was too following HEAD of tinycc <stikonas>well, tcc doesn't really bother to make releases, so you kind of have to follow their mob branch. That is somewhat of a problem for project health, but doesn't mean we need to blindly follow head <stikonas>M2libc is not designed to be backwards or forwards compatible <stikonas>breaking changes happens all the time there <stikonas>and hence various projects pin to the specific commit of M2libc <gabif>stikonas: i guess it's not important if tcc is not the newest head if it's not a fork <stikonas>well, I just meant that generally projects should try to release stable versions <stikonas>at least more frequently than once in a decade... <stikonas>of course there are other extremes... I think Chrome is moving to biweekly releases <gabif>but we can choose some mob commit and use it for some time <lanodan>One bit I somewhat see as a (very long term) issue related to recompiles is it means more code that's involved to review, like the binary seeds are nicely small but then there's machine-hours amount of code so kind of feels hardly reviewable to me <fossy>right, that's why across *different versions* i agree, but recompiling the _same code again_ just against different libraries, that argument doesn't apply <fossy>reducing the amount of code is an improvement though (as long as we still end up at the same, or equivalent, spot) <lanodan>Yeah, different version/patches and sometimes configuration (like say pre-musl picking fallback code but not post-musl) <aggi>TinyCC has not made any release for almost a decade <aggi>i consider this blocked for at least one reason, that is a release of TinyCC/OS together with TinyCC <aggi>meanwhile, i succeeded with a complete(!) self-hosting TinyCC/OS here, with linux-tcc kernel, libc, userspace forked <aggi>this TinyCC/OS is not yet "bootstrapped" <aggi>for a release of such, i'm currently reviewing live-bootstrap with the goal to keep the dependency graph as small as possible for a development host <aggi>ideally, such a bootstrappable TinyCC/OS bundle could fit onto 3.5inch Floppy Disk shipping with BusyBox/tcc and a few other development utilities <aggi>nowadays, the size-limit of a floppy disk is not justified, but i'll try to keep it a tad bit smaller than a 1.4GiB sized distfiles folder of live-bootstrap <aggi>finally, a bootstrapped TinyCC/OS shall suffice to bootstrap all else (GNU-toolchain/gcc/g++/binutils/coreutils etc.) if desired <aggi>myself won't take this route, and i will continue to maintain the C-only/no-c++ system-profile instead <aggi>in between there is two gaps to bridge, that is the transition from live-bootstrap towards a minified TinyCC/OS release, and then to bridge the gap towards to full userspace system-profile driven by TinyCC which i already got here <aggi>long story short, since it was asked several times again, what the issue is: a bootstrappable TinyCC/OS does NOT fit onto floppy disk <aggi>for practical reasons, another issue i want to address, is compilation-times hence i put a little hope into MES-replacement instead of mes-cc <aggi>because the TinyCC/OS workflow is and always will be 100% source-driven (either JiT/tccboot or AoT) <aggi>furthermore i've had full cross-compilation confirmed with this workflow already, which means any other ARCH may have it's bootstrapping covered by X86 and then cross-compile towards another from TinyCC/OS <aggi>which is, up until when any of the stage0-posix-<aarch64|riscv64> weren't ready is a decent fallback to retain "reproducible builds" for these at least <aggi>my little junk yard for keeping notes and review of live-bootstrap: <matrix_bridge><Andrius Štikonas> gtker: I'll add missing defines to stage0-posix-aarch64 define file then <stikonas>ok, pushed, it works now with AArch master... <aggi>found an estimate for mes-cc emitting tinycc: laurenth measured 10minutes compilation-time <aggi>that's much less concerning than i thought, given rickmasters mentioned 13hours total for entire live-bootstrap <aggi>seems those 13hours acculumate at far later stages than the ones GNU mes was involved <aggi>hence i'll stick to the default route taken with GNU mes, and otherwise the verification for binary-identical output that's handled by MES-replacement and/or pnut elsewhere <aggi>since i'll drop many pieces from live-bootstrap, this will probably chop off most of the compile-time penalty from it too <aggi>once linux-tcc kernel is bootable with tinycc/busybox, that one got SMP (since yasm can process some real-mode asm pieces for this without binutils) <stikonas>once live-bootstrap boots linux, it can run multiple threads <deesix>mescc speed heavily depends on what compiler generated the mes interpreter. <deesix>... if you're using mes as interpreter at all, instead of say... guile. <deesix>So, I guess that estimate needs some extra details. <aggi>i could too cheat a little with an optionel pnut-exe integration for faster test-cycles, and do final release only with GNU mes then <aggi>in comparison to MES-replacement pnut-exe seems ready for this <stikonas>hmm, that would indeed make things slower :( <mwette>for guile, the parsers will be compiled once and reused <stikonas>well, looks like Gskartwii is willing to help with some scripting <mwette>I would have to cook makefiles to generate the scm's during "make" <stikonas>I guess Gskartwii has a very similar environment as we have in live-bootstrap <stikonas>so basically mescc-tools-extra utilities <stikonas>but anyway, as long as it is just calling some commands, we should be able to manage <stikonas>mescc-tools-extra provides cp, rm, mkdir, chmod and a few others <stikonas>no mv, but that can be emulated or aliased to cp... <stikonas>cmp can be emulated by non-standard "match" utility I guess <aleksi>Hi, I'm the person who reported the nyacc perf concern :) And yes, the environment in nixpkgs bootstrap is largely based off of stage0-posix <aleksi>With kaem as the shell at that point, and no GNU Make until mescc is available <aleksi>The performance concern was with using nyacc within Mes <ekaitz>the question is, is that nyacc has made MesCC slower, or it's just that Mes is slow? <stikonas>ekaitz: you know those pregenerated files in nyacc? <aleksi>It's that 3.03.0 removed pre-generated data which we used, and the generation is slow <stikonas>3.03.3 kind of implemented on the fly generation with mes <stikonas>but now it basically generates it on each call <stikonas>somehow we didn't realize that during earlier discussions <ekaitz>aleksi: I asked because I'm working on GNU Mes (I'm a new extra maintainer of the project) <aleksi>Right now I'm running aarch64 mescc with nyacc 3.03.0 on a small cloud VM and it has been compiled c2str from tinycc for 1.5 hours :) <aleksi>ekaitz: Very cool, I've seen your Codeberg repository and I'm excited to see where it goes <ekaitz>i rewrote the whole core of the interpreter and made it become a bytecode interpreteri <ekaitz>i have a few extra changes pending, which may make it substantially faster <ekaitz>but the first things it to make it actually run :) <aleksi>Right, always happy to see bytecode <stikonas>ekaitz: but I guess that wouldn't help with restarts <ekaitz>at least it persists from one function call to the next, which didn't happen before :) <stikonas>oh, I think I remember, it was some helper for tcc <aleksi>stikonas: c2str is the little utility within tinycc that converts a header file into a bunch of C strings <stikonas>I don't think we used it in live-bootstrap <stikonas>at least not until much later when we were running rebuilds <aleksi>Oh, fair enough, maybe nixpkgs could do without as well <stikonas>anyway, new mes might be able to build newer tcc <stikonas>so we'll have to see how it works once that is released <stikonas>and I guess if we can skip tcc fork, it would solve our issue where currently either x86 works or riscv64 works <janneke>ekaitz: yeah, we've got some work to do before we can release, but yeah, compiling upstream tcc is within reach <stikonas>hopefully it will also work for riscv64... <ekaitz>ACTION is a maintainer but has no idea of the plan LOL <ekaitz>stikonas: that nyacc issue could we just write something in scheme to solve it? <aleksi>Yes, locally I have had some success with building upstream tcc with minor patches to mes and tcc, with latest nyacc. Mainly there's that mes segfaults when compiling a macro invocation of the form MACRO() or MACRO(x,) <ekaitz>the old /me would, but the modern me is a father and very busy <matrix_bridge><Andrius Štikonas> Oh yes, I imagine that really takes away all your time <matrix_bridge><Jeremiah Orians> Basically yeah, children require lots of time and attention and energy. <ekaitz>i'm also a little bit burned out <stikonas>yeah, me too... I still try to keep things running <stikonas>but haven't looked at e.g. stage0-uefi recently <matrix_bridge><Jeremiah Orians> But in a couple more years, he might finally be able and ready to help out with the programming <stikonas>(to get baremetal bootstrap working on UEFI...) <ekaitz>i've been working in several projects at the same time for long time, sleeping very little and with 0 support <ekaitz>sometimes it's frustrating to see that your are reinventing things that are more than solved... <mwette>The message I got was precompiled files is bad so I removed. For guile not a big issue, but for mes it is, we now see. To be clear, you don't want pre-compiled files in the distribution, but during the build process is OK? Then the generated scm table files would be included via include-from-path as before. <mwette>I'm hoping to get the nyacc TODO list done so I can move to other stuff. <ekaitz>mwette: yes, I'd say just building them is fine <mwette>I have something working. The Makefile is more involved now, though, since I need to be explicit about all the dependencies. <mihi>aggi: On my not so fast machine, when I tried it ~4 years ago, running the Mes step of live-bootstrap took between 20 and 30 minutes. To be fair, there have been optimizations (like unified libc) since then and machines may also gotten faster for some. Yet, most of the time of Live-Bootstrap is still spent on compiling more "modern" software (which is also the reason Guis tend to use the older version when they <mihi>have a choice between versions used in the middle of the bootstrapping chain) <mihi>cosinusoidally's tcc-alt-bootstrap only takes a few seconds, but it ends on ancient 0.9.2 tcc so you need to do quite a few more than 10 rebuilds to arrive at a recent one <mihi>Never tried pnut-exe, but it should be able to build slightly-patched 0.9.26, and I guess similar speed <mihi>About Frans' MES-replacement: I do not know if it aims to be M2-Planet compatible (initially Mes did not either, it was part of mes-m2 project to add that support), and the source does not look like it would be. stack_c is about 1k LOC and tcc_cc is about 5K LOC. So porting them is not trivial (quite a bit bigger than mes). Unless you accept tcc_cc.sl as "generated source" and only port stack_c.c. <mihi>stikonas, fair point. But there is one in the "search engine friendly" URL :)