IRC channel logs

<oriansj>although a strange thought hit me. What would the psychological effect if we bootstrapped say Firefox from live-bootstrap. A literal, hex0 to a fully modern web browser.

<muurkha>that's the main goal, isn't it?

<pabs3>wouldn't you bootstrap to a distro and build ff from there?

<muurkha>yeah

<oriansj>muurkha: well the goal for me always was to have fun while bootstrapping software; I just ended up finding a lot of friends along the way and ended up seeing much more success than I ever expected. ^_^

<muurkha>:)

<oriansj>pabs3: well we are already in the Linux from Scratch territory; so everything after GCC 4.7.4 is just extra awesomeness

<pabs3>interesting

<oriansj>with Guile and autotools properly bootstrapped (for the first time EVER) and the builder-hex0 work; we are well within a new era of software bootstrapping.

<oriansj>In the next year or so, we should have a bootstrap path between builder-hex0 and Linux.

<muurkha>what's missing?

<oriansj>then we can go after firmware bootstrapping, or custom bootstrapping hardware, sorting out the issues with various Linux Distros, making various pieces more robust or work on more platforms, not to mention the countless languages and "special" tools that need love to be properly bootstrapped.

<stikonas[m]>We need another more powerful kernel

<sam_>"love"

<sam_>:)

<stikonas[m]>Or maybe stage0-uefi would produce some results

<oriansj>stikonas: well that would definitely be awesome

<oriansj>not to mention hobby kernels people do for fun to learn

<oriansj>ports to the various *BSDs, DOS, CPM, etc

<muurkha>oberon ;)

<oriansj>muurkha: if someone wants, why not get that extra path.

<muurkha>yeah!

<oriansj>roll your own custom hardware and I might take time to port stage0 to it

<oriansj>like the Pineapple ONE; a custom computer made out of 7400/5400 chips

<muurkha>I think the Pineapple ONE is all CMOS, no 7400/5400 TTL

<oriansj>but the idea of making one's own hardware, toggling in just a little bit of hex and bootstrapping the world.

<muurkha> https://hackaday.io/project/178826/components is the parts list

<muurkha>as far as I can tell the only bipolar parts in it are the LEDs and 12 discrete NPN transistors :)

<oriansj>hopefully one can remove the EEPROM and it'll still be functional

<oriansj>nope, ALU EEPROM Logic

<muurkha>that's pretty central, yeah

<muurkha>you could use an old-fashioned EPROM if you prefer

<oriansj>I heard of a SERV RISC-V design using 7400 in SERV - RISC-V for a Fistful of Gates - Olof Kindgren, Qamcom Research & Technology-Nh-9xfK8Q1g that it could be done in 153 gates but no details

<muurkha>actually the 71321 (7132L?) might be bipolar, the datasheet says it uses 5 mW in standby

<pabs3>is there a place to see the current status of bootstrap from hex0 up?

<muurkha>I'd like to see that too!

<oriansj>pabs3: that would require someone to write up a full and proper bootstrapping tech tree

<oriansj>and I don't think anyone knows a great many steps past what we have in live-bootstrap

<oriansj>a great deal thus far has been running into bootstrapping problems we didn't even know existed and then finding a way to overcome them. (Like the Bison bootstrap gap)

<muurkha>it would be good to have a full and proper tech tree of what we have so far

<oriansj>muurkha: well we have a partial listing already (it just needs some updating) https://github.com/oriansj/talk-notes/blob/master/live-bootstrap.dot [generated https://github.com/oriansj/talk-notes/blob/master/live-bootstrap.pdf ]

<pabs3> https://github.com/jzimmerman/langcc https://news.ycombinator.com/item?id=32949019

<pabs3>"langcc is a tool that takes the formal description of a language, in a standard BNF-style format, and automatically generates a compiler front-end, including data structure definitions for the language's abstract syntax trees (AST) and traversals, a lexer, a parser, and a pretty-printer."

<muurkha>oriansj: yeah! I refer to it often but I feel like it needs some detail

<muurkha>pabs3: langcc looks pretty neat

<oriansj>pabs3: someone please get a formal description of Haskell for this tool so that we can solve that bootstrapping path already.

<stikonas>another way is to tell mihi that haskell needs bootstrapping and wait a bit

<aggi>oriansj: quickly checking the dependency graph against bison... there isn't that many crucial packages depending on it

<stikonas>aggi: that's only because distros don't rebuild pregenerated bison files

<aggi>i keep some separated package-sets, of potential trouble makers, as candidates for removal or replacement

<aggi>stikonas: my reasoning is, to consider removal of builds which haven't got a clean bootstrapping path

<stikonas>a lot of packages have bison files inside them but when tarball is created, maintainers prebuild .y in .c/.h files and then distros don't add bison as dependency

<aggi>i will be tracking this with gentoo-portage, and blacklist bison (virtual/yacc), and then move all builds into a package set named GNU

<stikonas>e.g. even coreutils use bison

<oriansj>stikonas: true but having mihi work on anything feels like cheating because mihi is so good at solving these bootstrapping mountains.

<stikonas>though you wouldn't see that in coreutils dependency graph in Gentoo

<aggi>yes... reminder, hence i prepare for a seccond system profile: tcc-toolchain-only/toybox-userspace/kernel-2.4 etc...

<stikonas>anyway, we'll hopefully have autogen in live-bootstrap soon

<aggi>hopefully i can fully remove autotools/automake etc soon, just noted bison (virtual/yacc) too on the hitlist

<stikonas>I've cleaned up some issues (musl, some scripting variables, etc) in mihi's gnu-autogen-bootstrap and got stage1 autogen built. Trying to rebuild stage2 now

<aggi>there's still some problems remaining, with lua and tcc-toolchain btw. (oasis linux package manager requires lua, and vis-editor integration with lua seems desireable too)

<aggi>and it will be interesting to see, what will be removed from the tcc-toolchain devdrop build, once i blacklisted bison/yacc

<oriansj>aggi: not much unless you pull out the pregenerated files

<stikonas>you mean keep pregenerated files in?

<aggi>some direct dependencies against bison/yacc i see already: iproute2, gawk, some noteworthy wreckage surrounding net-libs/libpcap

<aggi>and the following indrect dependencies

<stikonas>it's a fairly narrow path if you want to avoid pre-generated files (as seen in live-bootstrap), right now we even had to combined GPL and CDDL which is not ideal...

<oriansj>stikonas: well gentoo doesn't currently strip pregenerated files from tarballs as far as I can tell.

<stikonas>oriansj: no, it doesn't, as far as I know nobody does

<stikonas>guix doesn't do that either

<stikonas>live-bootstrap was the first project to do that

<oriansj>stikonas: you do as much as possible in live-bootstrap and that is amazing to see.

<aggi>i am aware of the fact, occasionally some BDEPS aren't completely tracked (noticed when fully removing gettext and libtool)

<stikonas>well, this is more about what is BDEP

<stikonas>you can define them weakly, i.e. minimal set of packages needed for build to complete which is what most distros use

<aggi>ok, there's much work to do, concerned with bison it seems; i'll not remove that from the gcc47 profile of cause

<aggi>however, i'll track bison for removal with the tcc-toolchain profile

<stikonas>or more strong definition: set of packages needed to rebuild everything from real source (i.e. don't use files that are not human-written)

<aggi>if you remember, the proposed strategy, to freeze/archive what i got with gcc47/c-only, and rebuild tcc-toolchain profile from scratch

<aggi>together with a roll-back strategy, including linux-2.4, mes-libc tcc-assembler issues etc.

<aggi>the linux-2.4 is another noteworthy problem

<aggi>ok, thanks for listening

<stikonas>yes, building linux-2.4 with tcc will be non-trivial

<stikonas>though perhaps using other kernel makes more sense

<stikonas>given how far builder-hex0 could take us...

<aggi>i am following this reasing with great interest, of cause

<oriansj>well that depends if Linux built by TCC is easier than writing/finding a kernel that TCC can build that is good enough to run GCC to do the Linux build.

<aggi>and with this i propose another approach: GNU-toolchain removal

<aggi>together with GNU-buildsystem removal

<aggi>because, as soon as any of this is required, the entire dependency graph blows up, beyond good and evil, rather early during bootstrapping and toolchain-maintenance

<aggi>hence, instead of considering GNU mandatory, i consider it optional, and tcc-toolchain support with _everything_ being mandatory, including any kernel

<aggi>and libc

<oriansj>aggi: well finding an alternate kernel that TCC can build that is capable or running GCC would certainly give you that option.

<aggi>oriansj: i think, with tcc and linux-2.4, we got that

<aggi>i hope

<aggi>for x86/64 at least

<stikonas>yes, but linux-2.4 is no-go for live-bootstrap because of blobs

<stikonas>we can't run deblob on it to get linux-libre

<aggi>ok, this is what i currently do with linux-5.10, deblob

<aggi>i don't know yet, how much dirt is hiding inside linux-2.4

<stikonas>probably more

<oriansj>aggi: well we don't know until someone seriously looks into it and as stikonas pointed out, there are also known issues that need addressing.

<oriansj>but with some experimenting, research and a little luck we might figure out what we need to go forward; otherwise we might just end up having to do a kernel too...

<oriansj>In C this time, of course

<aggi>kernel is a huge task, sufficient hardware-support is, almost, impossible

<aggi>because, that's why, i hit a brickwall, whenever i see closed-source hardware _only_; ao486 is still there, as a proof-of-concept at least

<oriansj>aggi: we also get to be selective in what hardware we need to support and do support which should reduce the problem space a good bit

<oriansj>and with some help we can start making more hardware options.

<oriansj>yes, things might be less than ideal now but I know we can forge a better path forward together.

<aggi>another one i considered, j-core.org, with this, tcc-toolchain is missing SH2, alot of development effort required, for hardware which is, practically, not available

<aggi>with ao486, it may not be the optimal choice, for FPGA deployment... nonetheless, currently it's probably the only practically useful option remaining

<aggi>and, x86, is relatively, cost-effective, if the FPGA realm is excluded from the reasoning

<aggi>thanks, for listening.

<mihi>stikonas, to be fair that also works the other way around (I think I was the first person who mentioned here I'd love to see M2-Planet on UEFI shell)

<mihi>but before I'd touch Haskell, I'd rather look at Ada / GNAT or at Oberon (as muurkha suggested), but for Oberon I would try to avoid C altogether (bootstrap a subset of the Oberon compiler directly from hex/assembly)

<mihi>not that I'm planning to do any of these things soon.

<oriansj>fair and thank you for being awesome ^_^

<mihi>oriansj, not to forget which awesome person have us stage0 to get this all started

<mihi>s/have/gave/

<mihi>if you want to bootstrap Oberon from C, I believe it is possible to compile norebo (an Oberon compiler that runs on top of POSIX) with obnc, and then compile Wirth's single pass compiler with it.

<oriansj>mihi: interesting, I would have started from the other direction. Getting hex0 to run on Oberon and bootstrapping up to M2-Planet and adding Oberon support to M2libc

<oriansj>hmm Isn't Oberon both a Language and a Kernel

<unmatched-paren>oriansj: I think it's also hardware, not sure

<oriansj>so what is ment by bootstrappping Oberon might need clarification. (is one bootstrapping the kernel, or the language or hardware)

<unmatched-paren>hmm, no

<unmatched-paren>there's no hardware, it seems. might be remembering something else

<oriansj>probably the Ceres (workstation)

<oriansj>or maybe the Lilith (computer)

<oriansj>although recently I saw this: https://portal.mozz.us/gemini/gemini.ctrl-c.club/~stack/gemlog/2022-08-29.oberon.gmi

<oriansj>or for those of you who prefer gemini: gemini://gemini.ctrl-c.club/~stack/gemlog/2022-08-29.oberon.gmi

<oriansj>unmatched-paren: there is also the oberon station https://pcper.com/2015/12/meet-the-oberonstation-kid-friendly-and-powered-with-the-son-of-pascal/

<stikonas>yeah, I'll go back to working on M2-Planet on UEFI once I'm done with autogen and live-bootstrap (or actually still need to fix M0)

<stikonas>right now hitting some small issues rebuilding stage2 autogen (actually autogen itself, and colums have successfully rebuilt), probably due to change of prefix from build dir in gnu-autogen-bootstrap to /usr

<mihi>oriansj, the Oberon system consists of the kernel, runtime system (garbage collector, backtrace decoder, ...), language, GUI, and compiler. Some also treat Wirth's RISC5 processor as part of it, but it has also been ported on other architectures.

<mihi>So after having bootstrapped M2-Planet on the Oberon kernel, you'd have a subset of a C compiler and a Scheme interpreter, but not much to compile/interpret with, as the whole system is written in Oberon (including the Kernel for RISC5; kernels for other platforms are written in Oberon and assembly).

<aggi>stikonas: not sure, what autogen is required for and how it is relevant to bootstrapping, with the gcc47/GNU profile here, it isn't noted anywhere as BDEP nor RDEP

<mihi>so you'd still have to port/rewrite the Oberon compiler in another language, or port obnc (the Oberon compiler written iN C) so that it can run on the Oberon system.

<stikonas>aggi: that's because again, distros don't rebuild pre-generated files

<stikonas>aggi: gcc's top-level Makefile.in is not human written

<stikonas>and it's not autotools

<stikonas>it's templated using autogen

<aggi>that's why i am asking, what is generated by autogen, and how much this would impact

<mihi>aggi: https://gcc.gnu.org/git/?p=gcc.git;a=blob;f=Makefile.in;h=1919dfee8294e8b9d7719548189788327438b87e;hb=HEAD#l2

<aggi>stikonas: meaning then, if i remove GNU buildsystem entirely (autotols, automake), then chances are no direct or indirect dependency remains against autogen

<stikonas>(there are other places where it is used but gcc is the main thing)

<stikonas>gnutls too but I guess you removed it

<aggi>long time ago, yes

<stikonas>autogen is not used that much but we need it for rebuilding Makefile.in in gcc

<stikonas>mihi: you had no issues with build-aux/run-ag.sh?

<stikonas>somehow it's exiting for me with exit code 2

<stikonas>top_builddir=".." top_srcdir=".." VERBOSE="" /bin/sh "../build-aux/run-ag.sh" -MF.deps/opts-dep.mk -MTstamp-opts -MP ./opts.def

<stikonas>in getdefs directory

<mihi>I recall it does not like relative paths in any env variables, but I don't remember actually run-ag.sh. Does it help when you do top_builddir="$(PWD)/.." etc?

<stikonas>hmm, no...

<stikonas>well, I'll dig in

<stikonas>and if I can't figure it out, we can always add DESTDIR support to make install in bootstrapping scripts...

<mihi>also, some of the autogen bootstrap scripts tend to drop logfiles in /tmp. So clear this out and look if anything got dropped there.

<stikonas>oh yes, I can see some error log

<stikonas>that might be useful

<stikonas>I can actually see the command that is failing

<stikonas>ok, the failing command was

<stikonas> /usr/src/autogen-5.18.16/build/autogen-5.18.16/agen5/autogen -L/usr/src/autogen-5.18.16/build/autogen-5.18.16/getdefs/../autoopts/tpl -MF.deps/opts-dep.mk -MTstamp-opts -MP ./opts.def

<stikonas>and to fix it I can do

<stikonas> /usr/src/autogen-5.18.16/build/autogen-5.18.16/agen5/autogen -L/usr/src/autogen-5.18.16/build/autogen-5.18.16/getdefs/../autoopts/tpl -L/usr/lib/autogen -MF.deps/opts-dep.mk -MTstamp-opts -MP ./opts.def

<stikonas>sed should be able to solve that

<mihi>hmm, not sure which file it is actually missing. the stamp-opts.tpl should be in .../autoopts/tpl?

<mihi>stage1 autogen's searchpath is rather short. it looks in LIBDATADIR (value at compile time), and in the -L paths. It does not check runtime environment variables.

<mihi>but I did not need this, either.

<mihi>setting LIBDATADIR to /usr/lib/autogen when compiling stage1 might be worth trying.

<stikonas>ok, I can try that too

<stikonas>no, LIBDATADIR does not help...

<mihi>or, where did you move the bootstrapped tpl-config.tlib?

<stikonas>stagen...

<stikonas>I moved it to /usr/lib/autogen

<stikonas>and before moving it I ran

<stikonas>sed 's#/usr/src/autogen-5.18.16/build/gnu-autogen-bootstrapping-autogen-5.18.16-v1.0/build/stage1#/usr#' build/stage1/lib/autogen/tpl-config.tli

<mihi>and what value is LIBDATADIR in https://github.com/schierlm/gnu-autogen-bootstrapping/blob/main/bootstrap_tarball.sh#L155

<stikonas>well, it's pointing to that temp build directory/stage1/...

<mihi>and tpl-config.lib is also there?

<stikonas>oh, so maybe I should adjust that one?

<stikonas>instead adding -L options...

<mihi>the tlib needs to be in the LIBDATADIR, also the license header files.

<stikonas>ok, let me try that

<stikonas>though surprisingly I hit very few issues

<stikonas>one before this one

<mihi>you did not check/compile the generated files yet :)

<mihi>missing tlib sometimes just creates uncompilable c code.

<stikonas>well, as a workaround I moved old tlib back

<stikonas> cp "${PREFIX}/lib/autogen/tpl-config.tlib" autoopts/tpl/tpl-config.tlib

<stikonas>before stage2

<stikonas>but I think setting LIBDATADIR to /usr/lib/autogen is better

*mihi is not sure if every autogen call that needs tlib includes -Lautoopts/tpl

<mihi>but yeah, set LIBDATADIR to wherever you place tlib and lic files.

<stikonas>apparently most of them do, that's why I only hit a couple of issues

<stikonas>but it's a bit hacky

<mihi>make install will overwrite them in FINALPREFIX anyway.

<stikonas>well, live-bootstrap first installs stuff into /tmp/destdir and then moves to /usr

<stikonas>so if make install does not respect that, FINALPREFIX wouldn't help

<stikonas>anyway, let me try with LIBDIR

<stikonas>it sounds like it might solve all issues

<mihi>then make sure to move the final tlib to /usr, if you already placed the "half-bootstrapped" tlib there. I'd like it not to put temporary bootstrap files into final directories.

<mihi>that's why I put the tlib in stage1 subdirectory, next to stage1 binaries.

<mihi>and make LIBDATADIR for stage1 point there.

<mihi>but ymmv :)

<stikonas[m]>Well, I was planning to make 2 ackages

<stikonas[m]>First one is half bootstrapped

<stikonas[m]>Second is full bootstrapped

<mihi>ok, then you should point LIBDATADIR to /usr, so that half bootstrapped package can find tlib and lic.

<stikonas[m]>I'll check file lists later, but we can always uninstall first package

IRC channel logs

2022-09-24.log