***alMalsamo is now known as littlebobeep
<littlebobeep>efine tcc-boot0 ;; Pristine tcc cannot be built by MesCC, we are keeping a delta of 11 ;; patches. In a very early and rough form they were presented to the <littlebobeep>;; TinyCC developers, who at the time showed no interest in supporting the ;; bootstrappable effort; we will try again later. These patches have been <littlebobeep>;; ported to 0.9.27, alas the resulting tcc is buggy. Once MesCC is more ;; mature, this package should use the 0.9.27 sources (or later). <unmatched-paren>littlebobeep: ah, looks like i got a bit confused regarding which C compiler didn't work on 64-bit... apologies. (The reason I thought it was TCC was that https://bellard.org/tcc doesn't mention 64-bit) <stikonas>unmatched-paren: you probably didn't setup environmental variables <unmatched-paren>stikonas: which environment variables do i need to change? I can't find any information in the mes manual <stikonas>I think you need to set MES_ARENA, MES_MAX_ARENA, MES_STACK, MES_PREFIX and GUILE_LOAD_PATH <stikonas>those we do set in live-bootstrap scripts <unmatched-paren>GUILE_LOAD_PATH seems to already be set, but when i look in the directory it points to, NYACC isn't there <littlebobeep>unmatched-paren: Are you using Mescc outside the context of live-bootstrap? <stikonas>mescc should work just fine outside live-bootstrap. Guix used it even before live-bootstrap even existed <stikonas>it's just a bit fiddly to set up required environment <unmatched-paren>it still happens after export GUILE_LOAD_PATH=/gnu/store/pvx5s14f7cm1j412vxnd933mz79z5fk1-mes-0.23/share/mes/module/ <unmatched-paren>but it doesn't after export GUILE_LOAD_PATH=/gnu/store/m4k7bzbys9c5cic2chk03rdi2zfn71hs-nyacc-1.06.5/share/guile/site/3.0/ <unmatched-paren>except after ^ it still segfaults, but for a different reason: unhandled exception:unbound-variable:(export) <stikonas>after you run the 2nd command, you removed mes modules from GUILE_LOAD_PATH <unmatched-paren>export GUILE_LOAD_PATH=/gnu/store/m4k7bzbys9c5cic2chk03rdi2zfn71hs-nyacc-1.06.5/share/guile/site/3.0/:$GUILE_LOAD_PATH:/gnu/store/pvx5s14f7cm1j412vxnd933mz79z5fk1-mes-0.23/share/mes/module/ <unmatched-paren>paren@guix-aspire ~/code/chibicc [env]$ ls /gnu/store/wvr9b9543yhdz4vpn8a0aivps5rck1v4-mes-minimal-0.23/share <stikonas>live-bootstrap runs from release tarball <unmatched-paren>seems like the mes-boot0 package sets these variables, i'll try replicating what it does <fossy>oriansj: "supported fully functioning operating system" -> Debian/GUIX/etc <fossy>base live-bootstrap will only have the needed tools (latest toolchain, make, git, etc) <fossy>stikonas: i _think_ i have addressed all your comments <stikonas>fossy: yes, I got that email. I'll take a look later today or tomorrow, will run live-bootstrap again too <bauen1>by the way, how close are we to go from live-bootstrap to debian or guix or something else ? <stikonas>bauen1: ideally we want to upgrade toolchain first <stikonas>other than that, live-bootstrap is now close to starting point of Linux From Scratch <stikonas>(with quite a bit more extra stuff in live-bootstrap) <stikonas>well, we have C++ compiler there, so it's not too hard to build new stuff now <stikonas>although, we did fail to build autogen without pregenerated files <fossy>i will be wokring on that, after we restructure live-bootstrap to minimise rootfs.py <stikonas>yeah, I can try to do /after -> /sysa rename first (for minimizing rootfs.py) <fossy>yes, that would be a good first step <stikonas>but we need to first merge this big packaging PR <fossy>it kinda creeped in size once i realised that i need to change this for this to work haha <stikonas>yeah, would have been easier to have a few smaller PRs but oh well <bauen1>we should already have everything to compile dpkg, but apart from that i don't actually know what debian would take to bootstrap <stikonas>well, need to create some "bootstrap debs" <bauen1>and i don't think debian packages generally don't care about pre compield autogen files <unmatche1-paren>so the purpose of live-bootstrap is partially to significantly reduce debootstrap's binary seed? cool :) <bauen1>unmatche1-paren: debootstrap doesn't actually do anything except untar a bunch of predefined debs and then run dpkg --install on the rest <bauen1>unmatche1-paren: sort of i think ***unmatche1-paren is now known as unmatched-paren
<bauen1>unmatched-paren: so the real thing you need to do is build the "build-essential" packages and all other packages with "priority: essential" <bauen1>i think after that all other debian packages should only depend on those (direct or indirectly) <unmatched-paren>that unmatche1-paren thing keeps happening even though i'm only logged on in this session... <nimaje>(maybe the reconnect is to fast for the server to see that your client has given up on the old connection and so the nick is still in use from the servers point of view) <unmatched-paren>(in that diagram, there's a red line from M1 pointing to something called `M2-Moon`) *unmatched-paren notes that a small implementation of lua isn't a bad idea <fossy>^ this includes all the stuff needed to generate files too <fossy>in terms of the raw toolchain jumps, live-bootstrap does <fossy>stage0 (hex0 -> hex1 -> hex2 -> cc_* -> M2-Planet) -> mes + mes libc -> tcc 0.9.26 + mes libc -> tcc 0.9.27 + mes libc -> tcc 0.9.27 + musl 1.1.24 (rebuilt a couple times) -> tcc 0.9.27 + binutils 2.14 + musl 1.1.24 -> gcc 4.0.4 + binutils 2.14 + musl 1.1.24 -> gcc 4.0.4 + binutils 2.14 + musl 1.2.2 -> gcc 4.7.4 (c++) + binutils 2.14 + musl 1.2.2 <stikonas>and strictly speaking we use M2-Planet to build new cross-platform tools (hex2 and M1) before mes <stikonas>hex versions of hex2 and M0 are a bit less capable than C versions: hex2 and M1 <unmatched-paren>oriansj: in the image that littlebobeep was asking about earlier, an `M2-Moon` is mentioned. i'm curious, what was that about? <stikonas>in any case it's probably something abandonned <unmatched-paren>also, since lua is tiny, it actually sounds like an ideal bootstrapping target <unmatched-paren>either way, probably some (ultimately abandoned) early idea for stage0 like the lisp and forth interpreters, except this one was never started <stikonas>yes, lisp and forth did not have much success here <stikonas>mostly because people volunteering were working mostly on C <stikonas>and hence C compilers (cc_* and M2-Planet) got bulk of the work <stikonas>M2-Planet is now quite a bit more capable <stikonas>still a small subset of C but we do have some preprocessor and a few additional C constructs <muurkha>stikonas: because "Lua" means "Moon" <muurkha>yeah, in Portuguese, which is the native language of Lua's authors <muurkha>that's why the language's logo is a crescent moon <unmatched-paren>it's maintained by a team at the Pontifical Catholic University of Rio de Janeiro, according to the about page <muurkha>and Lua is indeed pretty small, though I think not quite as small as Lisp, which is not quite as small as the λ-calculus or the ς-calculus <muurkha>core Lua is stateful, which could be an advantage or a disadvantage depending on your point of view <stikonas>yes, but the question is can C compiler be written in Lua capable on building TCC <unmatched-paren>and if you have Lua, you're probably able to get Fennel (unless fennel has some other dependency that i'm unaware of?) <stikonas>because ultimately we have to go from M2-Planet to TCC <stikonas>right now we have one slow way that only works on x86 <unmatched-paren>and replicating the entire Lua stdlib probably isn't that hard, since it's so tiny <stikonas>but finding somebody willing to do that might be harder *unmatched-paren still can't find a decent, complete, smallish assembly tutorial/guide <stikonas>riscv assembly is quite a bit smaller and easier to learn <stikonas>unmatched-paren: yes, I just compiled risc-v binaries with riscv64-unknown-linux-gnu-as <unmatched-paren>osdev.org looks to have a lot of neat tricks, but they're not general <stikonas>it has a bit of info on arithmetic operations, branching, functions <stikonas>you can build risc-v assembler (M1) with stage0-posix <stikonas>although it uses our custom non-GNU syntax <muurkha>a PEG parsing engine is easy to build in any language, so having it in the language's standard library isn't a significant advantage <muurkha>especially if you also have to implmenet the standard library, in which case it's no advantage at all <stikonas>M2-Planet itself is occasionally akward to add new features to it. But that's partially because it arose as a C port of cc_x86 compiler <muurkha>the repo I linked above includes a fairly Spartan implementation of PEG parsing in JS <unmatched-paren>annoyingly, the main lua implements the stdlib in the core C instead of as standalone lua, so we couldn't reuse it; if we could, then we could simply reuse it, and writing a bootstrap lua would suddenly seem like a very interesting idea... <muurkha>the most useful thing I've found for learning i386 assembly is GCC <muurkha>specifically gcc -g -Wa,-adhlns=foo.lst <muurkha>it directs the assembler to generate a "listing" <muurkha>listings are a bit of an archaeological thing. they were really important in punchcard days *unmatched-paren qutebrowser `:open what is an assembler listing` <muurkha>but the idea is that they tell you exactly what the assembler output and why, and with -g GCC includes the source code as well <muurkha>so you write a program in C, compile it, and look at the listing <muurkha>maybe in 01974 this was a routine thing to do in order to verify that the binary program that had been generated did what you intended, before waiting for 48-hour batch job turnarounds in order to test it <unmatched-paren>yeah it seems to be for the raspberry pi, but it does give me an idea of what a listing looks like <muurkha>yeah. -adhlns generates a more elaborate listing than just -al <muurkha>you can find the details in the gas man page <muurkha>possibly you don't have the documentation installed <muurkha>there's a long-running dispute between Debian and the GNU project about whether GNU manuals are free enough that results in GNU documentation often not being installed on Debian-derived systems. not sure if the same thing happened with Guix <muurkha>because the invariant sections provided by the GFDL cause the manuals to not fulfill the Debian Free Software Guidelines <unmatched-paren>i remember hearing about that, but i doubt it would happen on a GNU distro <muurkha>anyway the great thing about looking at assembler output is that it focuses on specifically the instructions that are useful most of the time <unmatched-paren>muurkha: i suppose debian provides their own original manual pages for GNU stuff? <muurkha>they just put the GNU docs in non-free <unmatched-paren>anyway... i still can't find a way to install a RISC-V toolchain on guix <muurkha>zooming in further from looking at the listing, you can write a program in C, compile it, and then run it under GDB instruction by instruction so that you can see the effect of each instruction <muurkha>this is most effective when you have a specific question like "does the inc instruction affect the i386 flags" or "which of these registers is the source and which is the destination" <muurkha>yes, but sometimes the answer is only comprehensible at the assembly level <muurkha>also though stepping through instruction by instruction at least once can be very helpful in understanding what's going on. not every instruction of, like, Firefox <muurkha>but for small programs it's a reasonable thing to do. when you get to running the same loop or subroutine a second time you can skip it <muurkha>this kind of thing is a lot more helpful for understanding questions like "what is this stupid instruction good for" and "how do I make a subroutine" which the instruction set manuals are usually totally useless for <civodul>muurkha: re GFDL, a number of GNU packages dropped invariant sections back then <civodul>yeah; Emacs is a notably (and sad) exception <muurkha>to see which instructions are used in practice something like this can be helpful: <unmatched-paren>...so does that mean that emacs built-in docs are completely unavailable in debian? <muurkha>objdump -d /lib/x86_64-linux-gnu/libc.so.6 | grep '^ ' | cut -c 33-39 | sort | uniq -c | sort -nr | head -20 <muurkha>no, you just have to install them from non-free <unmatched-paren>according to debian, non-free is not an official part of debian, but we all know what that discussion lead to... <muurkha>the above pipeline reveals that "mov" instructions are more than 25% of the instructions in libc <unmatched-paren>apparently i need to setup something called 'binfmt_misc' to run non-native binaries easily? <muurkha>yeah, binfmt_misc allows you to register handlers for weirdo executable formats, so you can run them with for example qemu-user <unmatched-paren>so you could pretend that java bytecode executables were native ones? <muurkha>yeah. although it wasn't very transparent for java because it always took forever to spin up a new JVM <muurkha>with qemu-user it can be transparent enough that you don't even realize you compiled a program for the wrong architecture for quite a while <unmatched-paren>qemu is basically a JIT compiler for foreign instruction sets, right? <muurkha>cribbing instruction sequences from whatever AOT compiler it was compiled with <unmatched-paren>could you technically make an ahead-of-time QEMU...? then it wouldn't matter which targets ${YOUR_COOL_NEW_PROGRAMMING_LANGUAGE} supports, you could just compile with $(YOUR_SHINY_NEW_COMPILER} to a foreign binary and AOT-qemu it into a native binary <muurkha>sure, lots of people have done binary instruction set translation. it doesn't work for things that load or generate code at runtime but otherwise it's fine <muurkha>the HP Dynamo paper actually got speedups from compiling from one instruction set into exactly the same instruction set because they were able to do better optimization on the code than the original compiler did <muurkha>that was in the 01990s, I doubt it would work today <muurkha>unless you originally compiled with -O0 or one of my toy compilers or something <unmatched-paren>muurkha: that just sounds like a language-independent machine code optimizer, not an instruction set translator <muurkha>nowadays generally C is the UNCOL of choice for people who choose to go that route <unmatched-paren>there's a C compiler targeting it (cproc) and my Pascal compiler project will as well <muurkha>but for example in the late 01960s there were simple pseudo-machine instruction sets for BCPL (O-code), SNOBOL, and META-II <muurkha>in the SNOBOL case the pseudo-instructions were implemented as assembler macros for each of the machines SNOBOL was ported to <muurkha>right now I'm playing around with some ideas for a C-oriented virtual machine for small computers, aimed more at interpretation than compilation <muurkha>the objective is to get maximum usefulness out of minimum RAM, because microcontrollers now have respectable amounts of RAM but vastly disproportionate amounts of computational power <muurkha>like, I'm looking at the Ambiq Apollo3, which has 384KiB of RAM and 1MiB of Flash, so it's comparable to an Amiga 1000 or a Mac SE memorywise, but it's a 48 MHz Cortex-M4F, about 100 times the CPU power of those machines <unmatched-paren>just reconfiguring to add qemu-binfmt-service-type to my machine... i'll presumably need to reboot once it's done? <muurkha>not unless your kernel is compiled to not support kernel modules <muurkha>(and possibly not even then if binfmt_misc is compiled into it. but normally people load it as a module) <unmatched-paren>i have binfmt_misc and a risc-v `as`, so i guess i'll try writing some assembly now :) <unmatched-paren>muurkha: first question: why do i need the `a0` in `addi a0, x0, 1` (to put the stdout file handle `1` in `x0`)? the equivalent x64 is `mov %rdi, $1`, which obviously needs only two arguments <unmatched-paren>actually, seems like my question should be `why do i need the x0`... <unmatched-paren>since it seems like RISC-V has additional friendlier names for its registers? <unmatched-paren>so the first argument is the destination, the second argument is a register (the contents of which will be added to the destination, and can use the zero register if you don't want this), and the third is an immediate value to add (which can be zero if you only want to sum the registers) <muurkha>yeah, the background there is that fixed-width instructions are much easier to implement in hardware than variable-length instructions, especially if performance is important <muurkha>(though later RISC-V did add moderately-variable-length instructions in the "C" extension) <muurkha>and 32 bits is enough for a fixed-width instruction in almost all cases, the big exception being 32-bit literals, which obviously won't fit into a 32-bit instruction word along with some kind of opcode <muurkha>so rather than making the instruction word 40 bits or 64 bits or something, they decided to make it take a two-instruction sequence to load an arbitrary 32-bit constant <muurkha>the second instruction in the sequence is addi <muurkha>the first instruction is unnecessary for constants of 12 bits or less, such as 1 *unmatched-paren has returned <unmatched-paren>muurkha: i'll define a `mov` macro for adding constants to registers, then... <muurkha>the standard name for that macro is `li` <muurkha>well, for the version that loads a constant rather than adding one <muurkha>`addi` is the general instruction at the machine level; `li` and `la` generate it as part of their output <stikonas>li is a bit more advanced than just addi <muurkha>it does, but when one of its source operands is the zero register, adding and loading are the same <stikonas>li can deal with larger than 12 bit integers <stikonas>and translates it into a few actual instructions <muurkha>addi is also used for other things, such as indexing struct fields and incrementing counters <stikonas>often it can be lui (for higher bits) followed by addi (for lowre bits) <muurkha>right, or `auipc` for higher bits in the `la` case <muurkha>"immediate" meaning "a value in the instruction stream" (as opposed to a value fetched from memory or in a register) <stikonas>in stage0-posix early programs are often small enough that auipc is not necessary to load global variables at the end of the program <stikonas>but cc_riscv64 and M2-Planet compilers have to use auipc <muurkha>I was going to say, I don't think you can know if an address is "small" before you link, but stage0-posix doesn't have a separate linking step <muurkha>auipc makes your addresses position-independent; it allows the linker (or the OS kernel in some scenarios) to relocate your code together with the things it's pointing addresses to, without fixing up relocations in the code <muurkha>that's why there are separate `li` and `la` "pseudoinstructions" (built-in assembly macros) <stikonas>but even lui + addi can only deal up to around 31 bit numbers <stikonas>if you want to load larger constants, you need more than 2 instructions on risc-v <muurkha>I thought they could get all the way to 32! <stikonas>no, only to around 31 bit - some small number <stikonas>in any case, even 32 bits is less than 64 bit <muurkha>but I guess the negative sign extension on addi hurts you there <muurkha>yeah. amd64 added a specific 64-bit load-immediate form of mov for that <muurkha>on RISC-V you have to use constant pools like on ARM <stikonas>but it's not like we care that much about performance in stage0 <stikonas>and we don't load that many large constants <muurkha>the paucity of large constants is the reason the LUI/ADDI split is 20/12 bits instead of the more obvious 16/16 <stikonas>fossy: I've added a few more comments to your PR <stikonas>two suggestions shoud fix those "grep not available" errors <stikonas>besides those, there is also one-off "error 13" printed when we checksum grep <stikonas>something must be going wrong with that _grep function <stikonas>but for some reason it only happens when checksumming gnu grep <stikonas>but I think we are almost ready for merge <unmatched-paren>is there some way to generate a unique label in gas (a bit like gensym in lisp)? i want to create a few macros for printing, but they'll need unique labels (for loops) so that they do not collide when you use the macro twice 
