IRC channel logs
2026-09-03.log
back to list of logs
<notgull>I was brainstorming a way to do Forth in a way that assumes it’s running on a machine made after 1990 <notgull>Because first off, every asm Forth I’ve seen relies on the lodsl instruction, which is x86 specific, ARM and RISCV don’t have an equivalent <notgull>But also, threaded code is weird and is effectively murder for cache locality <notgull>I reasonably think you could have a scripting language that starts off as some kind of higher level assembly, that has the ability to modify its own parser and semantics to eventually become high level <xentrac>notgull: ARM does have an equivalent to lodsl, a couple of them actually. You can `ldmdb r4!, {r5}` or `ldr r5, [r4, #-4]!`, and similarly `ldmia`, and there are also post-incrementing/post-decrementing address modes <xentrac>RISC-V indeed generally doesn't have such a thing; you have to use two instructions to update two registers because they didn't want small RISC-V implementations to need multiple write ports to the register file to hit one instruction per clock <xentrac>I've seen some attempts at gradually adding parsing and semantics to gradually evolve an assembly language into a high-level language, and I'd be interested to see more <xentrac>I don't think `lodsl` (`lodsd`) or the like is very important. on the original 8086 `lodsw` was implemented by microcode (there was no `lodsd` yet IIRC) and took several cycles to execute. and that was still true in the 80386 days. a low-end RISC-V running at the same clock speed as an 80386 will still be able to load the next threaded-code address and increment the instruction pointer faster <xentrac>but also the RISC design helps you clock it faster <xentrac>but also even slow bootstrapping is valid bootstrapping <xentrac>this is not an area in which anything other than maximum performance is intolerable <aggi>problematic compile-time performance propagates into later stages alike <aggi>which motivated my obsession with tinycc as a full drop-in replacement for almost everthing written in c, and dropping c++ as optional dependency from any basesystem <aggi>just as said, perl and python build up a notable dependency graph rather early too <aggi>plus scheme/guile/lisp (no an expert on this) <xentrac>hmm, it sounds like you're saying that if compiler A runs slowly, and compiler B is compiled with compiler A, then compiler B will also run slowly. But that's obviously not true. What do you mean by "problematic compile-time performance propagates into later stages alike"? <aggi>no, i mean a compiler such as tinycc already is sufficiently capable (depending on any final bootstrapping target, what finally should ship with a bootable *nix and a toolchain) <xentrac>how could someone possibly interpret the sentence "problematic compile-time performance propagates into later stages alike" as meaning "a compiler such as tinycc already is sufficiently capable"? <aggi>it's already been said, the autotools dependency graph is notable, together with characteristics of gcc/binutils themselves <xentrac>I mean I certainly agree that tinycc is capable, and that autotools has a lot of dependencies <aggi>and most important, the whole dependency graph can shrink significantly, to retain a fully capable *nix with a c-toolchain <aggi>that's one compile-time performance aspect with later stages of bootstrapping <xentrac>but I don't think any of that is relevant to how many CPU cycles of overhead the interpreter loop in a hypothetical Forth implementation might need <aggi>in the earlier stages the involvement of an intermediate language such as scheme/lisp raises questions too <xentrac>I mean, once you get tinycc built, you don't run the hypothetical Forth implementation anymore, right? <aggi>which so far, i consider GNU mes/mes-cc justified, for the simple fact they're heading for support of multiple ARCH with it, including risc-v <xentrac>so the hypothetical Forth implementation's hypothetically problematic performance won't propagate into later stages <aggi>myself considers risc-v less relevant for another reason though, which i rather not repeat myself <roconnor><xentrac> I've seen some attempts at gradually adding parsing and semantics to gradually evolve an assembly language into a high-level language, and I'd be interested to see more <roconnor>Like Ben Lynn's compiler quest? Have other examples? <aggi>maybe, it's not perfect and it's not multiple ARCH, but the whole stage0/M2 -> cc_x86 -> pnut-cc (without another non-C intermediate language) seems a major breakthrough too <aggi>there's even a chance to rebase for tinycc-HEAD with this, something risc-v could benefit from some time too given most activity continued on head <aggi>the principle question remains, if such a C/system-bootstrap should rely upon an intermediate language (forth, lisp) in between <aggi>i'm not referring to a LISP/FORTH/Haskell bootstrap itself, which is necessary anyway, but instead if such should be entangled into C-bootstrap <aggi>so far the major advantage of GNU mes is it's support to arrive at multiple ARCH other than X86_32 only <aggi>with the proposed transition (by dropping tinycc) from mes-cc-[ARCH] -> gcc-[ARCH], because then this type of bootstrapping too locks into unavoidable compile-time performance degradation, which a pure C bootstrap seems not affected by <aggi>then later, from the kernel-side, various ARCH other than x86_32 too lock into GCC with all it's pros and cons, without an alternative to it <aggi>in parallel the assembler situation remains complicated (for several reasons, some related to binutils/gas), on x86_32, and any other ARCH alike i guess <aggi>i'm not yet myself convinced, the YASM proposal to digest a few kernel bootcode.S pieces and syslinux bootloader was the preferrable choice <aggi>i'll keep that option available with tiny-bootstrap fork regardless, and will try to cope with a few remaining syslinux/NASM issues <aggi>isolinux.bin has not been re-compiled/assembled (by gentoo-portage at least) for a little while, >10years i think <aggi>not sure if the NASM/YASM route is reasonable, i'll know soon once some book arrived (i'm by no means a proficient assembly hacker) <aggi>and, it's not meant as a provocation at all, but it's another question of a bootstrap for any other than X86_32 is crucial <aggi>given various hindrances remaining (kernel/posix, assembler) <aggi>and, almost forgot rebasing for tcc-head in case it persisted as dependency for whichever ARCH <aggi>and, if such a rebase was desirable or necessary before proceeding with ARCH=RISC-V|ARM in live-bootstrap <aggi>i'm not sure either if syslinux was preferrable over GRUB, all i know so far is there remains a chance to move syslinux (together with cdrecord/isolinux.bin) far earlier into the dependency chain of live-bootstrap than GRUB could be <aggi>and of cause, i do follow the reasoning GNU focuses on their domestic projects (GRUB, binutils, gcc, guile/lisp/scheme) instead of alternatives (which are mostly GPL alike) <aggi>often it seems GNU is rather ambitious, with projects growing rather tall, beyond a size limit preferrable for bootstrapping <aggi>that's why i seek to bundle a few things with tiny-bootstrap fork, with another side-effect C-toolchain portability with tcc was fully confirmed <aggi>with another significant compile-time performance boost harvested by this, just to show it is feasible to do so <aggi>almost forgot, GNU autotools is another such GNU favorite, which of cause is important (together with a perl bootstrap) <aggi>but i do know it's possible to move this out of the way during rather early bootstrapping stages (before bootloader, before linux kernel, before busybox) <aggi>i still consider this problematic, if a bootloader such as GRUB could not be compiled/assembled without a dependency graph of hundreds of millions lines of code <aggi>for that matter, i've intentionally circuumvented UEFI <aggi>of cause it is important to support this with a bootstrap and bootloader, but for myself i am at the liberty to say NO to this <aggi>hundreds of millions of lines of code to yield some firmware/bootloader support? (for whichever ARCH, there is many) <aggi>i don't know yet in detail what RISC-V vendors will come up with next (but i've had a glimpse at ARM with this) <aggi>even a fully libre type u-boot firmware/bootloader makes any such system mostly non-bootstrappable in conjunction with live-bootstrap criteria <aggi>of cause it's important such remains opensource and free-software, to at least document some design-flaws (if there's a better wording i appologize) <aggi>and i've got no better solution than this myself either, even with coreboot/seabios sources at hand <aggi>i tend to think, various vendors kept their firmware stuff propietary, to keep all those problems hidden, shielded by NDAs <aggi>then saying, as Nvidia does, our stuff, that's worth a trillion accounted for <aggi>and what a coincidence, they're invested into risc-v (according to google search) <aggi>somehow i feel, FORTH and LISP type thinking belongs into the bootstrapping chain before ASM even <aggi>in case there was some system with nothing available, no BIOS/firmware, no bootloader, no assembler <aggi>torvalds mentioned once, some could envision ASM in their brain meanwhile programming C <aggi>various assembler feel rather "high-level", in a way, when thinking how digital circuits are designed <aggi>almost as if something was missing in between digital-circuits/opcodes, an abstract way of thinking/programming, to arrive at some assembler later <aggi>or, how could anyone call a BIOS interrupt, if there was no BIOS implemented <aggi>i mean, a "real-mode" BIOS, not some high-level UEFI <aggi>and i've had to think for a while, which assembler-book i would want to read first <aggi>more than 20years ago i tried MC68000 first, but the assembler.exe for this was some win32 binary anyway <aggi>way above abstraction level that digital circuit design was lectured in, and way above assembler itself <aggi>it NEVER made any sense to me, that's how a university curriculum was arranged (in parallel to java and a thousand other things) <aggi>even when LISP or FORTH style programming wasn't the most practically relevant first <aggi>there's some educational value in it, if it's kept small and simple to understand <aggi>which is contradicted by the fact those too are plagued by a hundred dialects and incompatible variants <aggi>TU Vienna lectured Z80 still (last time i checked), but they chose a limited command set for this <aggi>for the same reason, i abstained from the german "Das Assemblerbuch", which documents a thousand Intel(R) extensions, with a focus on Win32 environments <aggi>i am a little less worried about nasm on *nix, and yasm supports both, nasm and gas syntax for x86 at least... check. <aggi>the major grievance with x86_[16,32], it's the BIOS, that's where bootstrapping starts <aggi>Telekom lectured MC68000 <aggi>and i mean, how could any system be bootstrapped if there wasn't no Vista7 UEFI windoze computer standing nearby? <aggi>on the software side, there's a good chance to keep most of it FOSS, but as soon as hardware-development is involved <aggi>in practice, you've had to re-iterate over entire computer history since 1960s some time to see when this went wrong <aggi>that's the interesting detail with LISP, it's been developed long before some BIOS was haywired into each computer <aggi>it formalized an abstract/vendor-neutral style of thinking/programming <aggi>could be i'm in error, with regards to a LISP bootstrap, that by itself depends on cc_x86 anyway? <aggi>or is there some LISP written in opcode/macro-assembly? <aggi>with the former assumption, it would be hex0 -> .. cc_x86 -> some LISP -> for mes-cc -> and back to C? <aggi>(so far i skipped all lisp/scheme/guile pieces since i didn't need gcc with it's dependency on scheme inside buildsystem) <aggi>but for the sake of LISP itself, i would be curious, because historically it predates all C-compilers <aggi>it may not predate assemblers though <aggi>interestingly, wikipedia articles (german, english) do not mention the history of assembler <aggi>it's never been standardized either <aggi>in BASIC i've seen program lines mapped to memory addresses, for goto statements <aggi>no clue how LISP handles this <aggi>but it is possible to write a (limited) C-compiler in assembly/hex0/opcodes, by the live-bootstrap definition of hand-written code <aggi>which is comprehensible by a human brain <aggi>blowing up assembler with thousands of extensions contradicts it's intended use case (not specific to CISC or RISC i think) <aggi>i mean, if it's really supposed to be hand-written code comprehensible by a human brain, not some disassembly of generated compiler-output <aggi>if a real-mode BIOS must be bootstrapped first, there's no full assembler.exe available either <aggi>to construct various language/macro and I/O primitives <aggi>if the hardware provides a reasonable set of opcodes and memory-mapped peripheral i/o <aggi>initial pieces of assembler can be done with pencil and paper <aggi>if hardware design remains sufficiently simple, that too can be done with pencil and paper <aggi>and i've yet to learn why any such design had to be any more complicated, even when it was upscaled to 64bit data types <aggi>too lazy to search myself: did Intel(R) ever document their opcode formats? <aggi>i remember having seen this for MC68000 <aggi>chances are Z80 got this documented, but it's limited to 16bit <aggi>well, of cause, this is what an assembler does, soaking mnemonics and emitting opcodes <aggi>so the assembler sources might reveal some of it, but some assembler could emit opcodes without disclosing their format still <aggi>and i'm not in the mood to reverse-engineer this (former soviet/GDR u880 clone did btw.) <aggi>from the verilog/hdl side, this does document the hardware circuitry, but not necessarily expresses the opcode format directly <aggi>not that it matters, it's been more than 20years ago <aggi>i got a suspicion in this case, why risc-v is praised as the latest greatest nowadays <aggi>to document a few pieces explicitely, with pencil and paper <aggi>if Intel(R) and Nvidia(R) can find a few who could do it, with pencil and paper <aggi>for the lols over a trillion $$$ budget <matrix_bridge><cosinusoidally> aggi: Yep the opcodes are documented. I once created an emulator for a subset of the i386 instruction set using the opcode encodings from the official intel manual. I did notice there were one or two errors in the intel manual though, eg iirc one of there conditional jump instructions was incorrectly documented, which was a pain to track down until I had a look at a later intel manual (the i486 I... <matrix_bridge>... think). I think there are also one or two "invalid instructions" that are actually in use in the wild though (I can't remember off the top of my head which ones, but I think it's similar to how the 6502 would have some invalid instructions that actually work and do useful things). <pukkamustard>hello! the recent lwm article mentions a gnu mes thing implemented in ML (I'm assuming/hoping that's Standard ML). Can somebody point me towards that project? <xentrac>Sean B. Palmer did something in that direction that wasn't public last I heard <xentrac>06:22 < aggi> somehow i feel, FORTH and LISP type thinking belongs into the bootstrapping chain before ASM even <xentrac>I think this is a reasonable thing to try although not obviously correct. you may be interested in reading StoneKnifeForth, which emits binary i386 machine code from StoneKnifeForth source <xentrac>if you want to study how to write an assembler in assembly (or machine code) you might want to read Knuth's TAOCP, which works through a lot of example programs in assembly (originally MIX, now the RISCy MMIX) <xentrac>07:31 < aggi> and i mean, how could any system be bootstrapped if there wasn't no Vista7 UEFI windoze computer standing nearby? <xentrac>well, I've seen people toggle code into RAM with the front panel switches of an Altair, and I've entered machine code in octal on the front-panel keypad talking to a minimal monitor in ROM <xentrac>there are lots of LISPs written in assembly (usually with macro abilities). basically all of the Lisps from the 01960s and 01970s were written that way, and these days you have for example SectorLisp: https://justine.lol/sectorlisp2/ <xentrac>LISP does not predate assemblers, no <xentrac>08:26 < aggi> and i've yet to learn why any such design had to be any more complicated, even when it was upscaled to 64bit data types <xentrac>incremental development with backward compatibility <xentrac>09:20 < aggi> too lazy to search myself: did Intel(R) ever document their opcode formats? <xentrac>yes, obviously, and they continue to do so fairly assiduously <xentrac>pukkamustard: that sounds awesome; I'd like to see it too. let us know if you find it <xentrac>anyway you can write a Lisp in assembly in under a week, maybe under a day <xentrac>a simple Lisp interpreter is only about 128–256 lines of C, so you'd expect 256–2048 lines of assembly <matrix_bridge><Jeremiah Orians> Aggi: if your goal is a minimalist interactive language. Checkout wozmon it is implemented in 248 bytes and can be used to read and set memory and jump to memory addresses <matrix_bridge><Jeremiah Orians> And if you set the reset vector to its reset label; it’ll be what the system returns to when anything faults <xentrac>also, though, if you want something like wozmon to read and set memory and jump to memory addresses, you don't necessarily need any firmware at all <xentrac>if I'm not mistaken, Atmel AVRs can be programmed over SPI with a couple of debounced pushbuttons (resistors and capacitors) <xentrac>and, as I mentioned above, I've seen people toggle code into RAM with the front panel switches of an Altair, <matrix_bridge><Jeremiah Orians> True; I just need to find a weekend to write an x86 bios in M0 that is good enough for builder-hex0 to run on and then we can do a more advanced bios if we need one for Fiwix. Then I will have to do a mescc-tools port to 6502 and Z80 <matrix_bridge><Jeremiah Orians> Then if we can get the schematics for the Monster6502; then we would only need to figure out the odds and ends <xentrac>a BIOS serves a few different main purposes. first, it lets you run the same binary on different hardware without recompiling it. second, it eliminates duplication from different applications (and operating systems) — they can all call the same read-sector subroutine. third, it allows you to add and remove peripherals and RAM without recompiling the applications <xentrac>I guess also it allows the hardware vendor to not document their hardware <xentrac>none of these are necessary for bootstrapping <xentrac>if you have a read-sector subroutine for your floppy drive or NAND flash or whatever, putting it in the BIOS instead of your application doesn't make your work any easier. if you're trying to conform to an existing BIOS interface it might make it harder <matrix_bridge><Jeremiah Orians> True but I do tend to do things the hard way <xentrac>that's fine if that's what motivates you :-) <xentrac>another suggestion for how to not do things the hard way: I'd suggest using ARM or RISC-V instead of 6502 or Z80 <matrix_bridge><Jeremiah Orians> Well it is true that they have access to much more memory by default but a 6502 doesn’t have a vendor specific initialization routine. It just has 4 hard coded memory addresses <xentrac>also you get 32-bit arithmetic and base-offset memory addressing out of the box <matrix_bridge><Jeremiah Orians> And ARM includes multiply but RISC-V doesn’t. <xentrac>well, RV32I doesn't, but RV32IMAC and RV64IMAC do; it's the "M" (which also implies division) <xentrac>more recently there's the Zmmul instruction <xentrac>ARM also has pre- and postincrement addressing modes and conditional instructions, which are pretty convenient when programming in assembly <xentrac>any particular ARM or RISC-V processor tends to just have a hardcoded memory address for initialization <xentrac>it isn't standardized across all of them <matrix_bridge><Jeremiah Orians> I did not find the conditional arithmetic that useful. At most it allowed me to shave 6 instructions off of my division/modulus routines but that was about it <xentrac>it's not that important, but it's convenient <xentrac>for "Zmmul instruction" please read "Zmmul extension" <xentrac>also ARM has ldm/stm instructions for loading and saving blocks of registers from/to memory, so usually a subroutine prologue or epilogue is just a single instruction <matrix_bridge><Jeremiah Orians> I don’t deny that ARM includes a bunch of features that are useful if your goal is to squeeze something in to a few bytes as possible <xentrac>or write as little code as possible. RV32C is actually better than ARM or even Thumb-2 for code density <xentrac>a lot of ARM's design decisions are questionable in the light of hindsight in that they created a lot of potential for new processors to have unintentional incompatibility with existing software that did things like increment and overwrite the same register in a single instruction <matrix_bridge><Jeremiah Orians> But in terms of making easy to reason about assembly; most of it doesn’t help <xentrac>yeah, I can see how it's sort of optimized for writing rather than reading <xentrac>I mean, I feel like it's nice to have less code to read, but some of the semantics are a little surprising <matrix_bridge><Jeremiah Orians> And you’ll notice that my assembly coding style is very simple and explicit state tracking and easy to reason about at every instruction <xentrac>yes. is RISC-V more to your taste? other than the bitfields they threw in the blender in the instruction encoding obviously <xentrac>they claim 2100 CMOS "gate equivalents", i.e., 8400 transistors, although I haven't seen the design <matrix_bridge><Jeremiah Orians> If it wasn’t for the bit fields, the backwards encoding and the compressed subset quirks; RISC-V is a very clean and intuitive ISA <xentrac>that's twice the size of the 6502 (whose 2-input NANDs only needed three transistors instead of 4) <xentrac>(because it was NMOS rather than CMOS) <xentrac>I think SeRV needs tens of cycles per instruction, so it's much slower than a 6502 <matrix_bridge><Jeremiah Orians> Well sub-10K transistors is within the range of what could be built out of individual transistors on a PCB <xentrac>yeah, when I looked at this a couple of years ago, I came to the conclusion that the most sensible approach would be to build it out of "discrete logic" like 74HC08 and 74HC14 (which for some reason is cheaper than a 74HC04 at LCSC, maybe because it's produced in larger volumes these days) <xentrac>like you might need 200 chips, but JLCPCB will sell you the 74HC08 (four AND gates) for 9.85¢ and the 74HC14 (six NOT gates with Schmitt triggers) for 7.64¢ <xentrac>74HC164, 74HC165, and 74HC595 are also in the 10¢ price range <xentrac>so the BOM for a CPU might be US$20 of discrete logic <matrix_bridge><Jeremiah Orians> Honestly yeah, I could throw a few hundred in the pot <xentrac>the design files are unfortunately in EAGLE, which is not just proprietary but no longer available <xentrac>I'm pretty sure that's where I learned that JLC offers a special price for 100mm×100mm PCBs <xentrac>and receive a stack of assembled boards in the mail the next week <xentrac>I think he may have lost interest in the project <xentrac>the current state of the Pineapple One is that if you want one you need to convert the EAGLE files to something current (which I think you can do if you sign up for Autodesk Fusion), export gerbers, extract BOMs, send them to a PCB assembly house, and then possibly solder a bunch of parts by hand <xentrac>(I'm not clear on how much design-for-manufacturing he did; it looks like he at least did use SMD parts in most cases) <xentrac>(but I'm just inferring that from eyeballing the XML from EAGLE) <xentrac>also if I'm not mistaken he broke the boards down by functional units, so each board is different. so, instead of ordering 10 instances of a single board from the PCBA house, you're ordering one instance of 10 different boards <xentrac>whereas ideally you'd have 8 or 32 identical boards, each containing a single-bit-wide slice of the processor <xentrac>I think JLC can do PCB order quantities down to 5 boards, although some of its parts have higher MOQs like 15 parts <aggi>Jeremiah Orians> True; I just need to find a weekend to write an x86 bios in M0 that is good enough for builder-hex0 to run <aggi>i'm not sure if this was possible with modern X86 systems, recalling coreboot (complicated hardware init for AMD with AGESA for example) and seabios (it's rather tall) <matrix_bridge><Jeremiah Orians> It is possible but it is going to require a bunch of processor specific files <aggi>thinking for another moment, assuming coreboot/seabios on AMD were considered, with those a libre/opensource firmware seems good enough even when it's not bootstrappable <matrix_bridge><Jeremiah Orians> So, you’ll need to know the sku and the micro-architecture before building and burning a ROM <aggi>if for the sake of educational purposes a full system bootstrap was desired, i think a specially tailored hardware is a suitable choice <aggi>because currenlty i see no practically relevant benefit of re-writing a BIOS for modern x86 if coreboot/seabios support exists for some of those <aggi>and a specially tailored hardware for educational purposes could cover just that, how to design hardware beginning with pencil and paper