IRC channel logs

2024-01-10.log

back to list of logs

<muurkha>oriansj: yes, FIG-FORTH is written in assembly, but IIRC it's written in assembly in FORTH. I'm not sure what point that proves; that you can't run an interpreter in itself?
<stikonas>well, interpreter in itself would simply be slow...
<stikonas>interpreters are already much slower than compilers
<muurkha>I mean you need some other layer to run the lower-level interpreter
<muurkha>it can't be turtles all the way down the way it can be with compilers, even slow ones
<stikonas>indeed, but you could think of two layer solution, though even that I haven't seen in practive
<muurkha>sure, and that's actually useful for debugging, for example
<muurkha>Squeak or PyPy will run under itself or under an alternative implementation of Smalltalk or Python
<muurkha>or anyway they used to be able to
<muurkha>but you can't *only* run an interpreter under itself; the outermost layer of interpretation needs some connection to physical reality
<muurkha>anyway in general I think it makes sense to think of FORTH environments as shell scripting for assembly subroutines, so writing the interpreter in assembly in FORTH doesn't seem like a departure from FORTH to me
<Googulator>stikonas: they definitely work in the 2nd bash
<Googulator>("too well", almost - adding the ERR trap in interactive mode exposed a previously unnoticed bug in autogen's bootstrap)
<Googulator>not tested in the 1st
<oriansj>muurkha: I think of FORTH more of a higher level JIT'd macro assembler; which went to the logical conclusion of an ad-hoc assembly developer.
<muurkha>oriansj: ITC forth isn't JITted
<muurkha>there are some forths that are
<muurkha>well, maybe. I haven't seen one really
<muurkha>but not FIG-FORTH or most of the ones described above
<oriansj>all FORTHs have the ability to dump in some assembly and jump to it
<muurkha>well, almost all. I don't think pfe does
<muurkha>(not mentioning StoneKnifeForth because it doesn't rise to the level of being a forth)
<oriansj>muurkha: well do you have the abilty in StoneKnifeForth to write bytes to memory and jump to a memory address?
<muurkha>no
<oriansj>really?
<muurkha>yeah
<muurkha>I mean the bootstrap implementation of SKF is in Python
<oriansj>yeah, that could make it harder to support that feature of FORTH
<muurkha>yeah. it also doesn't support IMMEDIATE words
<muurkha>or interactivity
<muurkha>thus my statement that it doesn't rise to the level of being a forth
<muurkha>but pfe is a forth, and it doesn't come with an assembler
<muurkha>but you could probably write one in it, if that's what you mean
<muurkha>including a load-and-go assembler
<oriansj>but if you can write bytes to memory in FORTH, then the assembler could be written in FORTH itself
<muurkha>right
<muurkha>not sure if pfe exposes the necessary mprotect() but that's fixable
<oriansj>and if you can jump to a memory address, you effectively have a JIT
<muurkha>the thing that distinguishes a JIT compiler from a load-and-go compiler from my point of view is that the JIT compiler compiles things when you execute them, not when they are input to the compiler
<muurkha>so you have something that you could build a JIT with in that case
<oriansj>which is what you would do if you needed to speed something up in FORTH
<oriansj>akin to C's ASM() blocks
<oriansj>or if you needed to include some essential functionality the language doesn't support by default (again just like C's ASM() blocks)
<stikonas>Googulator: for gnu-autogen-bootstrap, I guess we just didn't need to upgrade...
<Googulator>could be, but the patches that did go in beyond what's used in LB are ours
<Googulator>all back in 2022
<Googulator>repository untouched since then
<stikonas>I guess we considered it done
<stikonas>well it did its job...
<Googulator>oh wait, he actually merged the PR :) nice!
<stikonas>yaeh, mihi can frequently be found here too
<stikonas>mihi did another bootstrap before autogen too... Guile
<Googulator>right, the psyntax one
<Googulator>the good news is that even without a tag, we can do https://github.com/schierlm/gnu-autogen-bootstrapping/archive/6bd36aa803c76fcfd1345b043f082c6399766325.tar.gz
<stikonas>yeah, we can
<Googulator>the bad news is that it includes all of those other fixes (mostly your ones, though) that were AFAIK never before used in LB
<stikonas>I just thought maybe wait a day or two and see if we can just pull in new version
<stikonas>hmm
<stikonas>aren't they duplicated in our scripts
<oriansj>the most important bootstrap after GCC was definitely guile
<stikonas>like making manpages reproducible
<stikonas>oriansj: also kernel bootstrapping
<stikonas>that is even bigger
<stikonas>it's probably still a bit rough
<stikonas>but 2 years ago we had almost nothign
<Googulator> https://github.com/schierlm/gnu-autogen-bootstrapping/archive/6bd36aa803c76fcfd1345b043f082c6399766325.tar.gz this one isn't
<oriansj>and then boom rickmasters for the win with builder-hex0
<Googulator>the other patches are mirrored in our custom script
<oriansj>and the Fiwix work was a gift from heaven that did in only a couple years what I thought might take decades.
<muurkha>yes!
<stikonas>Googulator: well, that's just your last commit from today
<Googulator>oh right, copied the wrong one
<Googulator> https://github.com/schierlm/gnu-autogen-bootstrapping/commit/07291ecceb08dc9ee7606a1307668b707be7ac27
<stikonas>strange, it's my patch
<stikonas>it must have come from integration into live-bootstrap...
<Googulator>oh, actually we do
<oriansj>now with Googulator's help; it is getting improvements and even nicer. ^_^
<stikonas>mihi did autogen bootstrap but I had to fix some reproducibility issues after that
<Googulator>I just didn't realize we call ./configure in src_compile
<stikonas>oh because autogen is weird
<stikonas>it has two stages
<Googulator>I assumed it would be in src_configure
<stikonas>boottstrap and rebuild
<stikonas>and bootstrap one can't be run from any other path
<Googulator>makes sense then
<stikonas>so it's all in one stage
<stikonas>ideally we would split it into pass1.sh and pass2.sh
<stikonas>but it doesn't work in this particular case
<stikonas>that live-bootstrap src_compile partially unpicks the wrapper script from gnu-autogen-bootstrap
<stikonas>and manually runs parts of it
<stikonas>to better fit into live-bootstrap structure
<stikonas>i.e. we don't wan't to download stuff in src_compile...
<Googulator>Which in turn is why we caught this issue in the first place
<Googulator>because we source the script instead of calling it
<Googulator>so "set -E" propagates to it
<stikonas>anyway, mihi is our local person here
<stikonas>so it should be fine
<stikonas>unlike autogen maintainer...
<stikonas>(who wasn't interested in this bootstrapping work at all)
<Googulator>well, at least he didn't cause early code to be lost forever, unlike Haskell folks
<stikonas>true...
<stikonas>though bootstrapping was done on modern code
<stikonas>early autogen code wasn't useful either
<stikonas>depended on old guile
<stikonas>and might have had depended on itself from very early commits
<stikonas>well, bison had a commit that already depended on itself
<stikonas>so sometimes even git history doesn't help
<stikonas>but yeah, hashell is a particularly big offender
<stikonas>and in the meantime, I've cleaned up some old hack in live-bootstrap https://github.com/fosslinux/live-bootstrap/pull/401
<Googulator>I was able to find an even earlier version of hbc/lmlc today, v0.99 (the very first one to include the Haskell frontend)
<Googulator>unfortunately it too depends on an earlier version of lmlc :(
<Googulator>makes me wonder how LazyML was originally bootstrapped
<Googulator>apparently lmlc was the only compiler to ever exist for it
<Googulator>unfortunately I spoke too soon about Yale Haskell - the version in Guix segfaults on any attempt to print to the screen
<muurkha>aw :(
<Googulator>and the locally built one was actually not building half of the prelude at all - it just printed a random error message in the midst of rapidly scrolling garbage, and then moved on, pretending all is fine
<muurkha>hmm! that's depressing
<muurkha>though probably at the time the garbage was scrolling much more slowly
<stikonas>how big are those early codebases?
<stikonas>e.g. ghc 0.26
<Googulator>Yale Haskell's port to clisp is a fairly recent one
<Googulator>it's originally intended for CMUCL and a few commercial Lisps
<Googulator>I've not had luck compiling either CMUCL or Yale Haskell using SBCL unfortunately
<Googulator>(SBCL is a bootstrappable CMUCL fork, but it has heavily diverged)
<Googulator>ghc 0.26's source tarball is 26MB uncompressed
<Googulator>and that's *without* tests
<muurkha>ooh, does the CMUCL build process depend on the Lisp system supporting CMUCL-isms that SBCL has abandoned?
<stikonas>hmm, perhaps a bit too much to port to another language
<Googulator>lmlc/hbc is much smaller, about 1.8MB source, half of which is C
<Googulator>the rest is LazyML
<muurkha>I guess I was being overoptimistc when I assumed CMUCL was written in Common Lisp rather than in CMUCL
<stikonas>ok, tha'ts reasonable to port...
<stikonas>so maybe 900 KiB of LazyML
<Googulator>CMUCL's build system assumes that CMUCL is not only installed, but installed in a specific way to a specific path, on a specific kind of host system specifically configured for the specific use case of building CMUCL
<muurkha>I guess Miranda is a dead end?
<Googulator>wasn't Miranda commercial?
<muurkha>yes
<muurkha>the build-system thing makes sense now that you say it; they probably had one specific machine they built it on
<muurkha>and if they wanted to install a new machine, why, spending a few minutes tweaking the build script every three or four years wasn't a big deal
<Googulator>oh, apparently Miranda did get released under a BSD license eventually
<muurkha>I'm guessing
<muurkha>oh, that's good! was there a Haskell written in Miranda?
<muurkha>and, if so, what was Miranda written in?
<Googulator>Not that I'm aware of.
<muurkha>oh, my memory is fooling me then, sorry
<Googulator>Miranda appears to be mostly C, with a standard library ("miralib") written in itself
<Googulator>confusingly, the Miranda sources use the same ".m" extension as LazyML
<muurkha>and Octave
<muurkha>everybody wants to be Bell Labs
<muurkha>it probably doesn't matter if nobody wrote a Haskell in it. it's not like there's a large important source base written in Miranda; their historical licensing prevented that
<Googulator>for Miranda, this is all the source history we have: https://www.cs.kent.ac.uk/people/staff/dat/miranda/src/
<Googulator>this is that earlier version of lmlc/hbc: https://rsync.nic.funet.fi/pub/languages/ml/lml/lml-0.99.src.tar.Z
<Googulator> https://research-repository.st-andrews.ac.uk/bitstream/handle/10023/22651/NormanPatersonMScThesis1994_original_C.pdf?sequence=1 this could be interesting as well - if the source code survives
<Googulator>as for CMUCL, probably no one cared about being able to set up the build system on a new machine because at the time CMU could only afford a single machine powerful enough to build it
<Googulator>oh, turns out there is actually another LML compiler: https://github.com/GunterMueller/Rio
<Googulator>only one problem: *it's written in Haskell*
<Googulator>of course, we do have Hugs to run it - if it doesn't use recursive modules, that is
<muurkha>GoogulatorMobile: hey, great find!
<fossy>i'll hold out on #400 until a new release is cut?
<Googulator>Looks like one of the recent changes has broke Python checksums when running kernel bootstrap
<Googulator>My main suspect is Fiwix upgrade
<stikonas>hmm, could be...
<Googulator>Checksums in bwrap appear to be still correct
<stikonas>though strange...
<stikonas>python is so far away from kernel bootstrap
<Googulator>- 'HAVE_BROKEN_SEM_GETVALUE': 0,
<Googulator>+ 'HAVE_BROKEN_SEM_GETVALUE': 1,
<Googulator>...
<Googulator>- 'POSIX_SEMAPHORES_NOT_ENABLED': 0,
<Googulator>+ 'POSIX_SEMAPHORES_NOT_ENABLED': 1,
<Googulator>this is what changed in all Python 3.x builds
<stikonas>hmm
<rickmasters>I'll do some testing, see if I can reproduce and isolate.
<rickmasters>Hard to see how that's possible since Linux builds python but who knows...
<fossy>not really possible for Fiwix to change Python checksums unless some intermediate checksum also changes. Python doesn't use fiwix at all
<fossy>i'd say it's more likely that its one of the config changes made for bare metal
<stikonas>yeah, that seems more likely
<stikonas>unless fiwix somehow changed linux checksum
<stikonas>and that propagated to python
<stikonas>but that's quite unlikely
<stikonas>(we don't checksum linux as far as I recall)
<rickmasters>stikonas: no Fiwix update didn't change linux checksum but many were changed as a result of new musl patch
<rickmasters>if it's consistently reproducible we'll figure it out but building to python is time consuming so could be a long time before we'll know
<matrix_bridge><Andrius Štikonas> rickmasters: but do we checksum Linux
<matrix_bridge><Andrius Štikonas> I thought that's the one package that isn't checksummed
<rickmasters>stikonas: Yes, I thought Googulator fixed that... hold on
<fossy>yes, that was fixed
<rickmasters>yeah, I can't find the fix but this was related: https://github.com/fosslinux/live-bootstrap/pull/366
<fossy>oh, no, i did it lol
<fossy> https://github.com/fosslinux/live-bootstrap/commit/e06a19f9e20dc4e3fe8084f03a00b8cf2bc9a6b4
<stikonas>well, in the worse case we'll have to compare what is different between bwrap and qemu modes in file system
<rickmasters>By the way, building with linux kernel (non bootstrap) has failed for a while now.
<rickmasters>I put in some PRs but there is another problem I'm still working on that goes way back: Cannot open /proc/iomem: No such file or directory
<fossy>hm, must say i haven't really been testing that.. too interested in kernel bootstrap :P
<stikonas>fossy: yeah, but right now that's the only option if you don't have BIOS?
<stikonas>at least until we get better support for UEFI bootstrapping
<fossy>stikonas: yes, this is true
<fossy>not saying it shouldn't be fixed
<fossy>just that i haven't been actively testing it - which i should have been
<stikonas>well, with so many different modes, hard to test everything...
<rekado>GoogulatorMobile: the Yale Haskell package in Guix is borked because clisp fails to compile the prelude.
<rekado>I don’t know why, but it used to work.
<rekado>since I haven’t touched the port since 2016 I don’t think it’s my problem. Worth checking if something in clisp changed since then.