IRC channel logs

<aggi>siraben: this seems a conceptual issue with several existing variants, the necessity of intermediate C-compilers in between M2 -> pnut|mescc|... -> tinycc|gcc|...

<aggi>instead of M2 -> tinycc|gcc|...

<aggi>which is easier much easier said than done, hence i think live-bootstrap chose a pragmatic approach with various intermediate compilers and re-compilation steps (there's many more related to musl-libc)

<siraben>aggi: yes, hence why I just want to conceptually shorten the path from m2 to tcc as much as possible

<siraben>I think a typed bootstrap has a lot of great auditability (I love Scheme but its macro system is a bit much for bootstrapping)

<siraben>I'm having a bit of an issue getting all of the logic to be contained in hcc though, it's possible that I have to just do Standard ML/OCaml-like compiler instead

<xentrac>a typed bootstrap is interesting

<xentrac>congratulations on riscv and arm64

<roconnor>what's missing from hcc?

<siraben>roconnor: in what sense? it's enough of a C compiler to compile TCC

<roconnor>er I asked the wrong question

<roconnor>I mean to say what's missing from the blynn compiler that you want for hcc

<siraben>blynn's compiler doesn't make for very performant code, many haskell optimizations are missing

<siraben>Especially around string handling, it's quite inefficient

<siraben>If the string handling were more efficient then we wouldn't need this IR step to be lowered to M1

<roconnor>Hmm

<siraben>So far I've been avoiding having too many upstream patches https://github.com/siraben/blynn-bootstrap/tree/040db25387fb53c00507bed92425d0ea57d61ac8/patches/upstreams

<siraben>I did already change blynn's semispace copying GC with a generational GC, since most objects are short-lived

<siraben>I think the combinator approach is fine for bootstrapping fwiw, keeps things small and modular

<roconnor> /* Trying to get vm.c and rts.c syncronized */.c

<roconnor>M2-Mesoplanet appends a trailing .c to this comment for some reason

<roconnor>Ugh, the vm segfaults when built with a more recent M2-Planet

<roconnor>Why is C always like this?

<siraben>roconnor: can you give me more details?

<roconnor>Submodule path 'stage0/M2-Mesoplanet': checked out '4b011a85da73a7c97212468d41f17e806ba99547'

<roconnor>Submodule path 'stage0/M2-Planet': checked out 'bd2fe4b0659fd0ad3f476a5ad0ef801bd134665d'

<roconnor>Submodule path 'stage0/M2libc': checked out '68a23cfd05d5a355ba7a30c770d684cbe86fcc4e'

<roconnor>Submodule path 'stage0/mescc-tools': checked out '5adfbf3364261a77109878a56b100aeeb6ef9ac4'

<roconnor>Submodule path 'stage0/mescc-tools-extra': checked out 'a151c245e512076971a3c85bb1502cf92cfa83b6'

<roconnor>Submodule path 'stage0/stage0-posix-x86': checked out '3b9c2bb6d4155e4f2e5f642b5e0f59255dfc5934'

<roconnor>These are the versions I'm trying to use

<roconnor>I've built the vm with M2-Planet.

<roconnor>And now running the first vm command on paranthetically causes a segfault in the run function after 14 or so iterations through the main loop.

<roconnor>I was able to build and run the vm with an earlier version of M2-Planet, but I was trying to update to a more recent one.

<roconnor>I need to take a break

<roconnor>I can push my nix branch if you think that would be helpful.

<siraben>roconnor: sure, but also doesn't the existing flake work for you if you're using nix?

<siraben>I see looks like I'm using m2-planet 1.9.1, going to update to 1.31.1

<roconnor>Is there a flake?

<roconnor>yeah the changelog suggest I'm using 1.31.1

<siraben>Oops I mean 1.13.1

<siraben>Yes there's a flake

<siraben>er, which repo are you looking at? oriansj's blynn-compiler or my blynn-bootstrap?

<roconnor>yeah I copied your typo

<roconnor>My stage0 I grabbed out of a recent live-bootstrap

<roconnor>For the blynn-compiler I'm using oriansj's, but I'm building it myself.

<siraben>Ah ok, I don't think oriansj's goes to the latest stage of blynn's compilers because he added more since

<siraben>I would first start by increasing the stack limit to unlimited and increase TOP

<siraben>see if it's memory pressure/OOM

<roconnor>with the m2-planet 1.9 ish version I was using before I got upto marginally before I hit a compiler bug.

<roconnor>I figure once fi get oriansj's version working, I can see about updating it.

<roconnor>fair warning, it took me two days to migrate to 1.13.1

<roconnor>I guess I just have a broken version of M2-Planet

<roconnor>If I replace put(num(1)); on line 550 of vm.c with

<roconnor>unsigned foo;

<roconnor>foo = num(1);

<roconnor>put(foo);

<roconnor>Then it seems to work.

<roconnor>well at least until patty.

<roconnor>Probably a similar error.

<roconnor>I wonder if there is a bug where a function pointer calls on the result of a regular function call.

<xentrac>roconnor: if I were debugging this problem, I would try to start by getting a stack trace, but I don't know how much of a hassle that is with code built with M2-Planet

<roconnor>Clearly there is a regression in M2-Planet if put(num(1)); segfaults and unsigned foo; foo = num(1); put(foo); succeeds.

<roconnor>I don't know if I feel like putting in the effort to build a issue. I'd have to try master first anyways.

<xentrac>not necessarily, no

<xentrac>I mean it's common for incorrectly compiled C programs to run successfully without segfaulting sort of by accident

<xentrac>the bug might still be present in the compiled version of the second program, or the previously compiled version of the first program, and just fail to be detected by the operating system because the incorrect address it was accessing happened to be in a mapped page

<roconnor>These two programs are semantically the same.

<xentrac>so the usual folklore for debugging C and C++ programs is that you should take advantage of the great good fortune of a segfault to fix the bug, because it's possible that the bug has been silently corrupting data for years :-)

<xentrac>yes, of course they are

<roconnor>The bug is almost certainly in M2-Planet.

<xentrac>Yes, definitely.

<roconnor>I wouldn't even know where to look.

<xentrac>But that doesn't mean that it's only in the new version of M2-Planet!

<xentrac>well, start by getting a stack trace

<xentrac>maybe also run the program under GDB and disassemble the subroutine it's segfaulting in

<xentrac>and dump the registers

<roconnor>GDB works with M2-Planet?

<xentrac>it's surely executing an instruction that accesses memory using some kind of address that's computed from values in the registers

<xentrac>GDB works with anything, but if your compiler doesn't give you stabs or something it isn't going to show you the source code

<roconnor>This is all above my paygrade. There is a reason I want to boostrap a Haskell compiler from combinators. :)

<xentrac>and it may have a hard time generating a stacktrace

<xentrac>yes, but if you capture the register values and disassemble the subroutine it's segfaulting in, maybe someone who knows assembly can figure it out

<xentrac>evidently the two programs are compiled differently, which is what you would expect for a simple compiler like M2-Planet which doesn't attempt to reduce everything to SSA or something

<xentrac>which is great information and should help narrow the bug down a lot

<siraben>roconnor: if you could give some more context I can debug it, can you push what you have?

<xentrac>assembly is much, much, much simpler than Haskell!

<xentrac>nothing wrong with combinators or Haskell, of course

<siraben>Ok I see you gave me the submodules above

<xentrac>just saying, assembly isn't as hard as you think it is

<roconnor>siraben: https://github.com/roconnor/nixpkgs-bootstrap/tree/blynn-compiler

<siraben>thanks, looking

<roconnor>You have to clone with --recurse-submodules, then checkout the blynn-compiler branch then git submodule update then you can nix-build . -A blynn-compiler.parenthetically

<roconnor>I traced the segfault to put(num(1)); on line 550 of vm.c

<xentrac>roconnor: if you want to spend another 60 seconds on it, run gdb programname

<xentrac>then:

<xentrac>set args whatever the command line args should be

<xentrac>r

<xentrac>bt

<xentrac>disassemble

<xentrac>info registers

<xentrac>quit

<xentrac>and put the output somewhere that people can see it

<siraben>I have a patch for M2-Planet

<roconnor>You fixed it?

<siraben>yeah function_call() spills FP into EDX, and EDX is also already used by mul whenever you index into an array

<siraben>so unsigned num(unsigned n) { return mem[arg(n)+1] ; } fails

<siraben>in the call put(num(1))

<xentrac>nice

<roconnor>Really? That seems like it would be more widely problematic.

<xentrac>maybe anything that does a function call inside an array index?

<siraben>It was introduced in this commit https://github.com/oriansj/M2-Planet/commit/be4951f671fb6630a53c8019b750cc6ef5ccaac7

<siraben>and the CI only ever tests function points with zero args test/run-pass/function_pointers.c

<siraben> https://gist.github.com/siraben/53a6ad67cbf20c021dff8f1044c75353

<janus>this is fun to read, congrats on finding an issue

<siraben>I guess we need some sort of fuzzing harness on m2-planet? XD

<siraben> https://github.com/oriansj/M2-Planet/pull/167

<xentrac>congratulations!

<FransFaase>I have been trying to compile the Tiny C Compiler (the version used in live-bootstrap), but I am getting a segmentation error when I execute tcc-boot0.

<FransFaase>I just read issue 470 of live-bootstrap. I just pulled the latest version and get the same error as mentioned in the issue, when trying to compile unified-libc.c with tcc-mes.

<FransFaase>In the past months, I have been working on a replacement for stage0, not using the GNU Mes compiler, that also works for x86_64 (or also called amd64) and I thought I was doing something wrong.

<FransFaase>But now it seems that the Tiny C Compiler version that is being used, does not work for that target.

<matrix_bridge><Jeremiah Orians> siraben: well we only had the manual fuzzing that I did when I had time but yes proper fuzzing should be setup if anyone is interested in doing that work

<FransFaase>Is anybody working on this? Or is there e version of the Tiny C Compiler where this does work?

<roconnor>FransFaase: FWIW, I just updated my partial nix bootstrap based on live-bootstrap this weekend and it works fine for me.

<roconnor>But I'm using x8l

<roconnor>x86

<roconnor>siraben: does your hcc work on amd64?

<siraben>roconnor: yes

<siraben>and i386, aarch64, riscv

<siraben>riscv64

<siraben>Fuzzing ran overnight, going to have a bunch of fixes up for M2-Planet

<roconnor>So we are saying that the tcc boostrap is broken on amd64 due to mes and your hcc bypasses mes and thus has solved the amd64 bootstrap problem?

<roconnor>That sounds like a pretty big deal

<siraben>tcc bootstrap is broken on amd64? it has been working fine for me

<roconnor>FransFaase: was pointing to this open issue: https://github.com/fosslinux/live-bootstrap/issues/470

<roconnor>I was just taking that at face value.

<roconnor>Maybe this is specific to live-bootstrap.

<FransFaase>When I compile the TCC version now included in live-bootstrap with gcc and my own compiler for the amd64 target it returns the exact same tcc-boot0, which crashes on tcc_new@plt when executed. The tcc-boot0 is compiled with the standard library taken from GNU Mes.

<siraben>I'll take a look

<roconnor>lol siraben fixes everything

<siraben>fuzzer output be like

<siraben>struct 0 0(){signed 0=(0)0;if(0)void(*)()0

<roconnor>Trying to crash the M2-Planet parser I see.

<roconnor>Do you think a fuzzer could actually catch this function pointer issue?

<siraben>It did

<siraben> https://github.com/oriansj/M2-Planet/pull/168

<siraben>Have a patch for issue #470 in live-bootstrap

<FransFaase>siraben: I am not waiting on #470 to be solved. When I execute the steps in tcc-0.9.26/pass1.kaem (either using gcc or my own compiler) it works until I call tcc-boot0. It crashes on the code at tcc_new@plt. There seems to be something wrong with the relocation tables.

<siraben>looks like Mes compiles c & (ll ? 63 : 31) incorrectly on x86_64

<siraben>FransFaase: checking

<FransFaase>I presume that this is a bug in the version ot TCC that is used in live-bootstrap. I know that there are newer versions of TCC and I wonder if it has been solved in those versions.

<siraben>damn he disconnected before I could send him the fix

<siraben> https://paste.siraben.dev/fuBg4D+zL3k.diff

<roconnor>siraben: If you run out of important things to fix, there is some bug in M2-Mesoplanet where it fails to compile vm.c

<roconnor><roconnor> /* Trying to get vm.c and rts.c syncronized */.c

<roconnor><roconnor> M2-Mesoplanet appends a trailing .c to this comment for some reason

<siraben>is it in your repo?

<roconnor>I can push it.

<siraben>yes, please do, a separate branch is fine

<siraben>or the same one

<roconnor>siraben: https://github.com/roconnor/nixpkgs-bootstrap/tree/mesoplanet-vm

<roconnor>same deal as before but building blynn-compiler.vm

<roconnor># starting M2-Planet build

<roconnor>M2-Planet --file /build/M2-Mesoplanet-000000 --output /build/M2-Planet-000000 --architecture x86 --debug

<roconnor>Unknown type .

<roconnor>/build/M2-Mesoplanet-000000:2383:

<siraben>thanks

<siraben>have a fix

<roconnor>super

<FransFaase>siraben: I did find your patch for TCC. I accidently disconnected, but I did find it in the logs. I am going to test it.

<siraben>roconnor: https://github.com/siraben/M2-Mesoplanet/tree/siraben/fix-vm-comment-token

<siraben>hold_string wasn't being NUL-terminated

<stikonas>siraben: in https://github.com/oriansj/M2-Planet/pull/168/changes#diff-de4cba9b04579b197d2d8e2d169c8dc7ac6f8b98982fecc853eba7a0c9050c30 those first files are missing end of file new lines. Intentional?

<stikonas>oh I guess they are

<stikonas>anyway, thanks for fixes

<siraben>Let me check

<siraben>I think just the fact that the syntax is incomplete not so much the EOF

<siraben>but regardless should be fine as reproducers

<siraben>oops was looking at the wrong thing, yeah it was fine

IRC channel logs

2026-05-12.log