IRC channel logs
2023-05-11.log
back to list of logs
<muurkha>you don't need a lot of instructions to write an assembler. call, ret, mov, load, store, add, jmp, some kind of conditional jump, some way to handle immediate constants <drakonis>i only ask due to the assembler that i'm using has some very interesting properties <muurkha>oriansj: I think it would take me longer than that, but that's pretty cool <oriansj>but the lack of macros and reader macro support really makes for a very different lis <muurkha>Lisp didn't grow macros until the 70s <drakonis>fasm-g is extremely minimal and does everything with macros, EVERYTHING <muurkha>drakonis: have you looked at GPM? or maybe m4 <muurkha>m4 shows you can do anything with macros but you might regret it <oriansj>macros that only you need to work with can be blessings. Macros written by others more often than not are a curse <drakonis>though i can see why you're bringing up gpm and/or m4 <drakonis>since i could do effectively the same thing? <drakonis>ie: actually roll my own assembler instead of relying on someone else's work <oriansj>drakonis: well yeah, that is easy if you go the M1/hex2 route <drakonis>ehhh i wanted to learn some low level programming <drakonis>i have never gotten to do anything free from the shackles of higher level languages <oriansj>well a stage0 port will definitely make you know *EVERYTHING* about the low level details of an architecture <oriansj>and armv7l still needs work (I can provide free hardware if you are interested) <drakonis>its too soon for me to even take on that yet <drakonis>and perhaps there is such a thing as too low level :v <oriansj>drakonis: you know the first step in stage0 is usally writing the bits in assembly before doing the conversion right? <oriansj>so you'd be writing bog standard assembly <oriansj>and starting from a small program and slowly working up to a C compiler in assembly <drakonis>i'll... think about it, but its still too early for me in my current level of knowledge <oriansj>well if you need a confidence boost, email me (jeremiah@pdp10.guru) with contact details and I can give you a call (signal, session, matrix, etc) <drakonis>what i need for now is just starting my assembly to forth project before taking on anything else <drakonis>and thanks oriansj, when i'm ready i'll let you know <stikonas>janneke, oriansj: that's not exactly true with how mescc is implemented <stikonas>it has some workarounds for loading immediates at the expense of fairly long instructions <stikonas>e.g. it stores immediate inline in the code and then loads from there and jumps... <stikonas>so immediates themselves might be doable <stikonas>on the other hand I've now remembered labels <stikonas>and they would definitely need riscv fixed word behaviour <stikonas>so perhaps some crazy combination of M1 with amd64 architecture and hex2 with riscv might work... <stikonas>in my opinion stage0-posix defines are better <stikonas>as they allow encoding anything from RV32I, RV32M, RV64I and RV64M <stikonas>drakonis: yes, doing stage0-posix work, e.g. porting new architecture will make you fairly good at assembly <stikonas>I new very little assembly a year ago... <stikonas>now I'm fairly familiar with riscv and x86/amd64 (well for these it's basic stuff, I guess roughly equivalent to RISC instructions, not anything to with vector stuff, etc...) <oriansj>heck, redoing an existing port and improving it is useful too (I hear x86 maps better to octal) <muurkha>I think it's maybe possible to design a less error-prone textual macro language <muurkha>yes, the idea of GPM was that it was a macro assembler like fasm, just without the assembler part <muurkha>a different approach to bootstrapping an assembler which might be easier is to write a bytecode interpreter in assembly, then write the actual assembly in bytecode <muurkha>this is maybe more appealing when the machine is pretty clumsy, like the 6502 or especially the 8080 <muurkha>uh, I mean write the actual *assembler* in bytecode <muurkha>drakonis: you said you're interested in bootstrapping a forth. a misleading thing about stoneknifeforth and jonesforth is that they're noninteractive <drakonis>hmm, that certainly would complicate things <muurkha>I think it's maybe better to think of forth as being like an interactive debugger than like a C compiler <muurkha>like, you can type 71f1 ? to display the word in memory at address 71f1 (assuming you've run the word HEX previously) <muurkha>in a C debugger you often proceed by stepping past a line of code <muurkha>in Forth you name each line of code instead of numbering it <muurkha>so you can "step past" a line of code by typing its name <drakonis>as opposed to being the mummified corpse that is most compiled languages <muurkha>and with that in mind a lot of its weird design decisions are more understandable <muurkha>like, traditionally, no local variables <drakonis>the author of nasmjf gave a talk about forth earlier this year <drakonis>the lack of local variables is usually remedied by the implementations providing the facility <muurkha>I mean if you provide the facility it means you can't use the outer interpreter to look at the variables' values <muurkha>because they're gone by the time you reach it <muurkha>the idea with Forth, as with Emacs, the ITS HACTRN, and akkartik's Mu, is that your normal UI also works as a debugger <muurkha>yeah, I think it's flawed in some ways but inspiring in others <drakonis>because it isnt some perfect machine it isnt treated as something immutable and immaculate <muurkha>the VM design is inherently inefficient in ways that make it unusable for a lot of their targeted apps, so they added peripherals for things like sprites <muurkha>yeah, I really like its wabi-sabi vibe <muurkha>also I think there's no real benefit to the limited address space, and it makes it hard to bootstrap uxn programming up to the level of something like Lua <muurkha>but what kind of matters more is that they have a bunch of roms for it <muurkha>I mean, for some purposes, that matters more <drakonis>for all purposes and intents, its supposed to be something you could build in the event that access to complex hardware becomes limited <drakonis>it is a lot like a fantasy console, in a manner of speaking <drakonis>but instead of being some custom hosted design, it is meant to do more than running games <drakonis>what is also important is that it actually did inspire others to do something better <drakonis>muurkha: you did mention that jonesforth and stoneknifeforth arent interactive, couldn't they be extended into becoming interactive? <drakonis>preforth is non interactive but is the base for seedforth, which is in fact, interactive <drakonis>rather, the step after seedforth which actually becomes interactive <drakonis>anyways, i'm thinking too ahead right now <muurkha>well, I think you could write an interactive Forth in them <muurkha>I think uxn could be improved a lot in its implementability on low-powered hardware <muurkha>like, you could implement it efficiently on an FPGA, if you were going for low power consumption rather than massive speed <muurkha>but not on a 6502 or an 8080 or a VAX or a PDP-11 or a PDP-8 or an 8088 or a 68000 or a PicoRV32 <muurkha>and its programming techniques depend a lot on self-modifying code, which means you can't put the "roms" in ROM <muurkha>or build a Harvard processor to run them <muurkha>it's written in an assembly language for a pretty generic register machine; you could recompile it for just about any processor without much difficulty <muurkha>and only a smallish amount of performance overhead, less than 2x usually <muurkha>including all those I mentioned above <emilytrau[m]>John Ericson: i did see your comment about splitting up the mes derivation for parallelism! will look into it 👀 <ericson2314>when I was working on CA derivations, I also would dream of trying to avoid bootstrapping mass rebuilding too :) <ericson2314>very excited to see this finally happen in Nixpkgs!! <emilytrau[m]>John Ericson: mmm dreaming of the day content-addressed derivations gets enabled in nixpkgs :D <drakonis>muurkha: should be a fun read after resting a bit <muurkha>I think BCPL or SNOBOL originated the technique oriansj used there, with the idea that you'd implement the virtual machine instruction set in macros for your local assembler <ericson2314>emilytrau[m]: Hopefully we can convene a Hydra team to get merged the necessary changes to it <ericson2314>and then we can test it -> enable it optionally -> enable it by default <roconnor>how bad would it be to make some of these bootstrap derivations fixed output? <muurkha>you can sort of think of the core Forth words like DUP DROP @ ! ; that way, but you generally can't implement them efficiently <drakonis>largely because they don't map very well to this technique? <muurkha>well, it's more that the Forth VM isn't very similar to the 8088 or RISC-V <muurkha>if you're writing assembly for a virtual register machine you have to do register allocation yourself <drakonis>or really because common off the shelf hardware isnt the same? <muurkha>and the implementation can map some or all of the virtual machine's registers onto the target machine's registers <muurkha>except maybe on the 6502 where you'd map them to the zero page <muurkha>whereas to compile Forth efficiently your compiler needs to do the register allocation <muurkha>and the Forth syntax isn't generally amenable to being parsed by common macro assemblers either <drakonis>you'd need an assembler with more advanced macro capabilities that isnt focused on a specific ISA, yes? <ericson2314>roconnor: if we just use this to rebuild the bootstrap tarball, we are doing something pretty close to that <oriansj>well sectorFORTH did a great job of showing the minimal turing subset of FORTH <ericson2314>(or sure, replace builtin fetch derivation with fixed output) <roconnor>If building the tarball is deterministic. Which it probably is? <roconnor>I don't know what shenanagins gcc gets up to. <muurkha>roconnor: yeah, it's important to make tarballs deterministic <muurkha>hmm, maybe Griswold 1970, 1972a is online by now <muurkha>yeah, the book goes into a lot more depth <stikonas[m]>I will be able to update stage0-posix once that is merged <stikonas[m]>rickmasters: so we need 2 binaries in total before we start building them ourselves? <rickmasters>stikonas: right, I think a readme is also needed to explain how to compile hex0 and kaem if you choose to use the binary+kernel bootstrap <stikonas[m]>Anyway I'll take a look at your pr in more details later (possibly over weekend) <pder>rickmasters: I was testing your kernel-bootstrap-phase2-v1 branch which is really amazing work by the way. Running rootfs.py -q completed successfully all the way to gcc 13 <pder>I noticed that during the builder-hex0 phase it loads in files including ones in sysc/distfiles but when make_fiwix_initrd is called all of the files in sysc/distfiles fail on copy_files because they are not present <pder>After linux boots and the remaining packages are built all of the tarballs for sysc are downloaded again <rickmasters>pder: I'm not sure I understand. Are you pre-populating files from sysc? I don't think I'm seeing copy_file errors... <rickmasters>pder: I mean, are you pre-populating files in sysc/distfiles? <pder>I used download-distfiles.sh to predownload all the tarballs if that is what you mean. <rickmasters>pder: ok, I haven't tried that. Thanks, I'll take a look. <pder>When builder-hex0 boots up and loads files into its memory filesystem, is it supposed to include all files needed to bootstrap to the end of sysc? <pder>OK, actually I was wrong- builder-hex0 does not load sysc/distfiles/* into its filesystem, but fiwix-file-list.txt contains sysc/distfiles/* <pder>hence the errors in copy_files in make_fiwix_initrd <rickmasters>pder: I suspect it will only work by downloading sysc packages after linux is launched due to memory constraints but maybe I can at least avoid errors... <pder>maybe when generating fiwix-file-list.txt you can just ignore the sysc/distfiles directory <rickmasters>stikonas: Are the sysc/distfiles from download-distfiles.sh only used for chroot/bwrap? <rickmasters>stikonas[m]: It's a bit tricky following all the options and code paths. <rickmasters>stikonas[m]: I'll try to make sure I understand what happens with kernel bootstrap if someone uses download-distfiles.sh or --external-sources ... <pder>stikonas: I don't think --external-sources would make any difference in this case because if I run rootfs.py -q --external-sources system_c.prepare() never gets called <stikonas[m]>download distfiles is bash implementation of rootfs.py downloader <pder>stikonas the else block on line 253 of rootfs.py is run <stikonas[m]>Yeah, it's probably missing sysc prepare option there <oriansj>rickmasters: I agree with stikonas, a simple readme with a link to where builder-hex0 can be found should be sufficient. <oriansj>and yes, that is the current master repo (until savannah changes their mind and lets us host it there) <probie>Assume easily supporting multiple architectures is a high priority, and we can assume linux. Do you think it would be viable to implement a simple register based vm, write a bytecode interpreter for it in $arch, write a more full featured assembler in the vm's bytecode, and then later write "compilers" from the bytecode back to $arch? <probie>My intuition is that this is going to fall over (or at least lose any advantage of the single portable bytecode) due to syscalls being different for different architectures <probie>but perhaps that can be reduced by building in all the relevant syscalls into the vm, and just having different implementation levels of the vm where more and more syscalls are supported, instead of trying to allow it to make arbitrary syscalls?