<oriansj>stikonas: is it only the hex1 version getting stuck or is it the clang/nasm version as well? <aggi>i see, you had re-implemented the GNU Guix bootstrapping (formerly done for gcc-2.7 with a giant binary seed tarball) <stikonas>I only have nasm and M1 versions for now <stikonas>it goes a bit further than guix. It does not need Guile and it does not use pre-generated files (such as configure scripts) <stikonas>oriansj: I didn't write C version though, that would probably work <stikonas>well, I'll go over the source and try to find a bug <aggi>the integration path i am planning for with tcc slightly differs, meaning i do not want any userspace (and kernel too) depend on gnu-toolchain (gcc/binutils/glibc and llvm alike) anymore at all <aggi>i only want a tcc-toolchain being capable to bootstrap the gnu-toolchain (gcc47/binutils2.22 for aarch32 is what i got) <aggi>and the bootstrapping path at the lower levels, in that realm i was thinking about the hardware itself, and the related supply chains <aggi>including the reasoning, if any opensource SoC can be deployed onto FPGA (there is some for ia32 btw., limited to 40MHz on an Altera FPGA) <aggi>however, bootstrapping in the hardware real raises far more concerns again (j-core.org documented some of it related to xilinx for example) <aggi>and then, i began to rule-out any non-desireable hardware options according to some criteria (opensource, no proprietary software-tooling) <aggi>my conclusion some time ago, we haven't got at least some opensource VT100 terminal anymore (hardware and software), which was fully documented and bootstrappable <aggi>that's why, i arrived at gigatron ttl, because it doesn't depend on excess dependencies on the hardware-development side <aggi>although i doubt a gigatron ttl could terminate vt100 at 640x480 at least <muurkha>I think the Heathkit H19 might be open source, and it can emulate a substantial part of the VT100 command set. on an 8085 <aggi>then there is some other interesting Z80 SoC... however guess what, the compiler suite for this, small device c compiler (SDCC).... <muurkha>if you're building a text terminal out of gates, a character generator might be worthwhile, because it allows the rest of the hardware to run at extremely modest speeds <muurkha>I guess I need to include admu.h, which brings it to 96 lines of C <aggi>currently i am fighting at another frontline, which is i want to retain a somewhat practical userspace including all development tooling, linux kernel etc... everything compiled/linked with tcc <aggi>no GNU toolchain for this, no llvm <muurkha>I think at this point SeRV and PicoRV32 probably make 8-bit processors obsolete <muurkha>and GCC and Clang can target them without difficulty <aggi>muurkha: are those opensource with verilog available? <aggi>hooking into tcc cc/ld as the only system toolchain components, offered some remarkable insight, what to think about GNU toolchain <aggi>my scepticism towards this... i don't know, sorry <aggi>to repair userspace on latest gentoo testing branch again, "only", to remove c++, and compile with gcc-47, was almost 1year of effort <aggi>and there is some nasty blockers remaining <aggi>such as, elfutils... was it really necessary to lock this against non-standard/C11 __thread extension? <aggi>because, to my knowledge, there isn't any compiler/toolchain (written in C), which could digest __thread extension... uhum, RedHat <aggi>oriansj: yes, i do. however, x86... <stikonas>aggi: well, builder-hex0 uses BIOS calls <aggi>BIOS, that's a whole other discussion <stikonas>but in principle you can follow the same idea but you'll have to replace bios calls with some hardware drivers <stikonas>so on arm some equivalent of builder-hex0 would only work on particular hardware (and also be maybe twice as big) <muurkha>(CP/M implemented BIOS on the 8080 but builder-hex0 won't run on an 8080) <oriansj>aggi: but it is a possible roadmap to a POSIX kernel written in assembly capable of running the steps needed to bootstrap a C compiler <aggi>there's several highly critical gaps of non-reproducible bootstrapping, vendor-locks etc... <aggi>ok, i think we share some principle ideas <aggi>and if we started at the hardware level, with an opensource ia32 SoC... <stikonas>BIOS does not have to be vendor-lockin, you can run builder-hex0 on seabios <aggi>what were the consequences: ... Xilinx or Altera IDE necessary with a giant Wintel PC, to deploy the hardware onto FPGA <stikonas>well, yes, at some point boundary between software bootstrapping and hw becomes a bit fuzzy <aggi>if you know what i mean... that's why the gigatron ttl is appealing <aggi>however, from there, and this is late 1960s, form gigatron ttl i don't see any way forward to scale this up and bridging the gap to a 16bit unix at least <oriansj>aggi: also knight is bootstrapped and is based on a computer too old to have firmware so that might be of interest <stikonas>pineapple one is also without firmware, isn't it? <stikonas>but ultimately, you have to reimplement input/output calls when you don't have standard firmware <oriansj>stikonas: well any thing we can put on an FPGA would be a solid step forward <muurkha>pineapple one is the most bootstrapped <muurkha>aggi: you can build SeRV or PicoRV32 with APIO (yosys and friends) <muurkha>you still need the giant Intel/AMD PC <aggi>however, i skipped the idea for now... as a rough estimate, 40MHz i386, i think this is too limited even with tcc compiler or similar <muurkha>maybe with our current live-bootstrap path <muurkha>however I can assure you that we compiled our own operating systems on machines that were much smaller than a 40MHz i386 <oriansj>well it is good enough to get us to M2-Planet relatively quickly, but mescc would probably have some serious performance problems building TCC <muurkha>I think the point at which it goes from impractical to practical is the borderline between the 12-bit PDP-8 and the 16-bit (address space) 8080 or 6502 <stikonas>after all it's a self-hosting C compiler <oriansj>but if one is willing to put up with multihour build times and get a few GBs or RAM, even a 50MHz 386 should be sufficient to build TCC <muurkha>but 32-bit address spaces and megabytes of RAM make life a *lot* easier <aggi>building tcc is one milestone <muurkha>BDS C (also free software now, btw) on CP/M had a lot of limitations <aggi>another idea, a clean VT100, both hardware and software, opensource etc. <oriansj>aggi: well mescc is the slowest piece as TCC is much faster and leaner than mescc <muurkha>its contemporary the Apple ][ was a far preferable future <oriansj>also we can do custom hardware to make our lives easier. <muurkha>at a hardware level the VT100 was similar in complexity to an Apple ][, but vastly less capable <oriansj>and if ASIC costs keep dropping, we can even make our own ASICs <muurkha>ASIC costs are rising, not dropping, if you're talking about entry-level ASICs <muurkha>25 years ago you could do a from-scratch chip through MOSIS for US$5000 <muurkha>well, one tapeout, which probably won't work <muurkha>25 years of inflation would change US$5000 to US$7000 or so <oriansj>last I checked for a block it was only $2K <muurkha>(also Google is doing a thing where they do it for free) <aggi>another problem with opensource ia32: peripherals such as usb1, ethernet... the SoC proposed seems inconclusive with the specs <aggi>and depends on, what's deployed into an FPGA, yet ia32 requires lots of FPGA space, and i doubt there is headroom remaining for some USB1 controller <oriansj>aggi: we don't need USB nor ethernet for bootstrapping <oriansj>heck punched paper tape is good enough for me <muurkha>it's going to take a long time to punch your compiled Linux kernel into tape ;) <aggi>however, i had something practically useful in mind, meaning with tcp/ip, vga terminal etc... <aggi>a clean and re-producible bootstrapping isn't the only criteria for a system to end up with <muurkha>hmm, maybe I'm confused: the lowest price I found when I looked with MPW/shuttle services was €650/mm² in an 0.35 μm process, minimum 3mm², which is indeed about US$2000 <muurkha>so maybe the prices for ASICs really *are* coming down. be interesting to see what current prices are <aggi>the gigatron ttl shipped for $150 <muurkha>that's US$2000 to get the chip made, I think you get 20 chips for that <muurkha>and it's fairly easy then to go into volume production <oriansj>muurkha: TSMC sells a 300mm wafer processed using its N5 technology for about $16,988 and by block I mean 10mmx10mm of the wafer. and skywater 130nm wafers go for $4K so you have a ballpark <muurkha>that seems a lot more expensive than US$2000 for ten 3mm² chips <muurkha>unless you mean "10 mm × 10 mm of each of 20 identical wafers" <muurkha>aha, 28 nm is enormously more capable for digital than 0.35 μm <muurkha>that would explain why it's more expensive <muurkha>100 times as many transistors, and faster clock speed too (0.35 μm was still within the Dennard scaling slope) <stikonas>oriansj: I've narrowed my hex2 bug a bit. It's a bug in ClearScratch <oriansj>muurkha: correction N5 is the 5nm process <muurkha>which is a lot more than US$2000 per design and US$100 per chip <muurkha>oriansj: so are they really making up a mask set for just one wafer? <aggi>almost forgot, the Altera FPGA board sold for ~$750, capable of an ia32@40MHz, give or take a few mhz <oriansj>I don't know about the mask costs but that is the wafer cost <muurkha>also I think the larger part of the price is the NDAs you have to sign <muurkha>oriansj: wait, you're saying there might be additional mask costs on top of the US$17000? <muurkha>because those would presumably be an order of magnitude larger if so <oriansj>muurkha: I was quoting only the public per wafer cost <oriansj>it didn't specify anything additional about what that includes in the post <aggi>in comparison a raspberry zero 1.3 (1GHz, 512MiB RAM) sold for $5 <muurkha>aggi: Lattice FPGAs that can run SeRV but not PicoRV32 sell for US$3 <aggi>anyway, this is the lower bar currently: the possibiity to move my entire development setup onto such a raspi zero 1.3 type hardware <aggi>and retain the 100% from source workflow, with tcc-toolchain there <oriansj>well let us assume $20K per 300mm wafer, if our chips are 1mm square; we can make 70,685 of them. and even if half are dead, that is still less than $1 per <aggi>and i had somehow given up hope for bridgin the bootstrapping gap downwards (both hardware and scaled-down software) <aggi>from gigatron ttl there is no way forward, and from such a raspi type system there is no way back downwards <muurkha>hmm, if they're charging you US$17000 for a 10mm × 10mm block, it seems unlikely that an entire wafer will cost you only US$20k <oriansj>muurkha: the $17K was for the whole 300mm wafer <aggi>in the middle in between a gigatron ttl type system and a raspi, there is some ia32 soc options... which are expensive <muurkha>thanks! I totally misunderstood that <aggi>did extensive research for a while, down to z80/gigatron etc... there's no way back up <muurkha>I don't think you can saw the wafer into 1mm² pieces in practice, which is why the lower limit was 3mm² from the CMP vendor I was looking at 12 years ago <muurkha>aggi: take a look at SeRV, pineapple one, and picorv32 <oriansj>aggi: well expensive is relative and if getting $100 of FPGA to someone willing to do development which helps move the bootstrap forward seems cheap to me <muurkha>though an AVR is a bit faster than a Z80 or probably a Gigatron <aggi>atmega are closed source <aggi>and as far as a clean vt100 was concerned, geoff's vt100 (and another fork) seemed appealing <muurkha>I mean emulating a 32-bit CPU on an 8-bit one <aggi>although those too utilize closed-source microchip or some stm32 <muurkha>you could reasonably claim a Z80 was emulating an 8-bit CPU on a 4-bit one, and that SeRV is emulating a 32-bit CPU on a 1-bit one <aggi>muurkha: i did read a DEC VAC 32bit system already could process 128bit general purpose <aggi>a z80 got 4bit ALUs iirc <aggi>and got a 16bit address bus regardless <muurkha>not sure what you mean about "process 128bit" <stikonas>oriansj: so I think the problem was that there was no guarantee that scratch area that I "malloced" would have zeroes. Can be fixed by adding an overflow check <stikonas>I think on POSIX we had it initialized to zero <aggi>muurkha: such as xorshift on int128 ... iirc DEC VAC could do it, most recent x86_64 couldn't <muurkha>I never learned enough VAX assembly language to know what it was capable of, but remember the VAX was heavily microcoded <muurkha>x86_64 can copy megabytes of data with a single rep movsb instruction <muurkha>but that doesn't tell you anything interesting about its efficiency <muurkha>stikonas: POSIX malloc isn't required to zero memory it allocates, but often it happens to do so. sbrk() does zero the newly created memory <stikonas>muurkha: well, hex2 on POSIX uses unreserved memory at the end of executable that is zeroed <oriansj>muurkha: well sbrk() doesn't ensure the memory is zero'd but kernels that don't zero leak secrets from one process to another. <stikonas>spotted another small issue, but I think I'll ignore it (on UEFI when we open existing file for writing it does not get truncated) <muurkha>stikonas: thank you also for the correction <stikonas>I think that's why sbrk usually zeroes it when first accessing that memory but if you lower sbrk and increase it again it might not be zeroed <oriansj>also libresilicon might give us a route to cheap 500nm <oriansj>like $100 per wafer with mask costs cheap <oriansj>which is more than advanced enough to do 64bit processors 1GB sticks of ECC RAM <oriansj>not to mention simpler chips for things such as controlling hard drives or something like Gigabyte's I-RAM <oriansj>but we need to find people who are interested in doing hardware and porting stage0 to those targets <aggi>j-core.org made considerable progress,... however, mentioned it already, when i see Xilinx or Altera in action with their software tooling (and who knows what else) <aggi>haven't got a full oversight, over what's possible with free software in this realm (ghdl, yosis, etc.) <aggi>and even, some weeks ago, i decided to blacklist various software which violated acceptance criteria of my c-only toolchain profile <aggi>since then, GTK is gone... just saying <aggi>that's why, the gigatron is appealing, because, it can be soldered onto a breadboard, the digital circuitry designed with pencil and paper <oriansj>aggi: Icestorm is the biggest move forward on FPGAs <muurkha>is your concern with C++ related to not being able to bootstrap GCC, aggi, or is it a question of RAM usage or compile time? <aggi>well, talking about this, got me banned in various channels recently <aggi>long story short, i consider the entire GNU toolchain wrecked (llvm isn't an alternative) <muurkha>oh, that doesn't sound so unreasonable <aggi>too i consider most of the buildsystem (makefiles, autotools etc) wrecked, and there is nasty dependency graphs build up against c++ <aggi>which ... makes it worse, much worse, that much... GTK gone <muurkha>you want something you can understand? or that isn't politically co-opted by Microsoft or something? <aggi>well, with Micro-Soft, i consider the Altair BASIC intersting, that's it what i wanted to say about them <aggi>meaning, this is their product portfolio from late 1970s <aggi>muurkha: wrong, it does... unicode, harfbuzz/pango/... this <muurkha>I mean that's why they wrote their own object system in C, right? to avoid C++ <oriansj>what about a GUI toolkit like FLTK or QT; how do you feel about them? <aggi>GTK/GNOME are heavily locked against harfbuzz/c++... another one written in c++ was gperf... <aggi>it's a complicated dependency graph <aggi>anyway, that's not the worst part, since motif still exists, if need be <aggi>when GNU GCC switched to c++, gcc-4.8, you know it, that's crossing a red line imo <oriansj>console only lifestyle isn't a bad choice all things considered <aggi>the C11/__thread dirt inside elfutils... cannot be digested by tcc either... <aggi>oriansj: even then, to cleanup the profile again, was almost 1year of work <aggi>and it still doesn't fully pass with tcc compiler <aggi>not to mention some stories surrounding linux kernel <oriansj>of course; personal preferences are just that. What you personally feel fits you best. We respect those choices here <aggi>i do respect c++ and related competence, it is me who was locked-out, technically, and poltically too <stikonas>well, personally I think that C++ is good for GUI since GUI is all about objects but then again it's my personal opinion <oriansj>but I am certain we can find a great deal of common ground on the lower level bootstrap bits <oriansj>and getting more platforms supported up to TCC really would be awesome <stikonas>yes, tcc seems to be quite essential to bootstrapping. Fairly capable compiler but reasonably easy to build <aggi>i hope tcc won't be overloaded with features and extensions, which is what contributed to the GNU toolchain and buildystem wreckage <oriansj>aggi: well, if they go insane we can always pick up the pieces and make our own version which we can bootstrap. We have done it before and we can do it again. <aggi>tcc can compile/link more than 99% of the builds from the latest gentoo tree i picked, which was a total of ~700 (with python, 600 without) <aggi>i was still using utilities from binutils, and want to test elfutils (which i need to rewrite the __thread tls implementation for to compile with tcc) <aggi>this is what remains to do, worst blockers: linux kernel + tcc, some aarch32 asm inside musl-libc, the elfutils one <aggi>a review of toybox (busybox fork) to ship some parts... iproute2 package for example approached a state close to total wreckage <aggi>currently rolled back to an old and patched version of iproute2 <aggi>the, with python gone, i'll have to re-implement the package-management for the tcc-toolchain distribution <oriansj>aggi: I'm sure some people here will find it interesting. <aggi>well, i do not intend to lock out GNU gcc/binutils... those already passed tcc too on aarch32 <aggi>however, i aboslutely do not want to depend on the GNU toolchain anymore <aggi>too i want to get rid of the autotools fun <oriansj>xbps is a C program if I remember correctly <aggi>void package manager? yes... this is one candidate <aggi>another one is #suckless style package management, slpm <aggi>which is written in posix shell <oriansj>aggi: would a package manager written in scheme be valid? <aggi>oriansj: i only skimmed quickly over the Guix one iirc... and at a first glimpse wasn't convinced <aggi>simple reason being, lisp/scheme isn't a language i programmed with <aggi>and given my experience with typical GNU software.... <stikonas>yes, xbps is pure C, we use it in live-bootstrap <stikonas>but we mostly use it as slightly smarter tarball <aggi>next on my list is linux kernel, currently v5.10, which i already patched to avoid C11 _Generic macro clutter (gcc47 doesn't eat this) <aggi>and linux kernel bumped compiler version requirements, AGAIN, since then <aggi>who knows what they did since then... <stikonas>well, fossy built 4.9.10 for similar reasons <stikonas>but then we are using even older GCC there <aggi>i did need linux-5.10, because of aarch64 systems i got here <aggi>another story, missing aarch64 support... and armv9 already announced, they would drop aarch32 compatibility for userspace <aggi>that's the short version, of the story, surrounding linux kernel <oriansj>honestly it is familiar to what we have seen <aggi>and another reason, i want to scale down, to aarch32, which kernel/compiler etc. exist for, and tcc should be fast enough, even on a raspi zero 1. <aggi>scaling down software requirements at that frontline too opens new opportunities, for opensource hardware / fpga deployments, which typically are a little more limited *aggi shuts up and hacks now <oriansj>please continue to share your work and progress; I think we can help each other ^_^ <aggi>i am not willing to nor cannot git-push to micros~1 github or anywhere else, without appropriate contracting <oriansj>you can do your work anywhere you like <aggi>at least, i am under the impression after more than 15 years of experience... it's not possible anymore <oriansj>to have a public git repo on a service you trust or run yourself on a server you control? <aggi>what would be one year of work of mine be worth to git push without any contracting anywhere? <aggi>it's not that i don't want to, i can't. <oriansj>aggi: what do you mean by contracting in this case as our work is under Free (as in freedom) Software Licenses <aggi>oriansj: this means in germany legislation exist, with regards to woring and compensation, and i doubt any typical HR department anywhere would recognise what i do <aggi>i mean, not at least telekom management had the intellectual abilities, to comprehend, who left them <stikonas>I don't think there is commercial interest in bootstrapping... <aggi>at Deutsche Telekom we started with digital circuit design btw... <stikonas>they are happy enough to build in e.g docker container <aggi>and mathematics... good luck with this on "labor market"... enough said. <aggi>currently, i only try to rescue what can be, which isn't much anymore, after 15 years <oriansj>aggi: ah, you mean work for hire contracting terms. <aggi>and i attribute some particular technical problems with bootstrapping related to this: lack of management <stikonas>quite a few people in my team graduated in maths... <aggi>oriansj: i mean i am looking back onto missing salary for 15years work already. <stikonas>so if you did maths, I don't think you wasted 15 years... <aggi>with maths i only recaptured some tiny fun project, some fast crypto cipher for linux kernel <aggi>which i did to practice little C <aggi>otherwise, the type of maths i did, the place i am currently, how should i say, i am not in the mood to fight with differential equations systems anymore <aggi>they don't need this anymore, in germany <oriansj>aggi: so you view your work as a sunk cost to be monetized <aggi>oriansj: i consider the entire industry close to total wreckage, including the related scientific aspects <aggi>since there is some missing pieces of "bootstrapping" down below in the realm of, mathematics too for example <aggi>remembering, TI89 series calculator s <aggi>however, i can't repair this anymore, even if i wanted to <aggi>and i doubt, even if, TI wanted to publish schematics and sources, for this old calculator, they couldn't, i think <aggi>it's bitrotting somewhere, gone forever <stikonas>well, you'll always have some missing piecies in maths <stikonas>you can't just start with some axioms and prove that what you have is consistent and complete theory <stikonas>all those incompleteness theorems, etc... <aggi>another reason why, gigatron ttl is appealing <aggi>i am considering to fully disconnect internet <oriansj>aggi: if you ignore the huge Flash inside, yes <aggi>gigatron got an emulator for 6502 too <aggi>however, i couldn't study yet it's native assembly language <aggi>and then, they got their gigatron control language <aggi>anyway, thanks for your time <aggi>once the tcc-toolchain distro is in shape, i am willing to publish of cause <oriansj>aggi: well helping people learn assembly language is certainly something we help people do here. ***genr8eofl__ is now known as genr8eofl
<stikonas>the nice thing about position independent code is that hex1 can do its job quite well, there is no need to manually encode absolute addresses like it was done in stage0-posix <stikonas>perhaps we should have used them in stage-posix too <oriansj>stikonas: well x86 makes position independent code a pain but yeah AMD64 is better for bootstrapping than x86 in terms of instructions and registers <stikonas>well yeah, I only meant stage0-posix-amd64