IRC channel logs

2021-01-15.log

back to list of logs

<stikonas>probably should build perl first...
<stikonas>although, that might not be any easier
<fossy>dosen't really matter the order
<stikonas>hmm, perl has some Configure file to do build
<stikonas>let's see if it's handwritten
<stikonas>it's not autotools configure
<OriansJ>*Is confused how M1/M0/hex2/hex1/hex0 is hard to verify*
<fossy>OriansJ: hard to understand without appropriate background knowledge of the architecture
<OriansJ>stikonas: cc_* is always in M0-macro assembly (as indicated by the DEFINEs and lack of crossplatform support)
<fossy>but that is just the nature of such low level
<fossy>languages
<OriansJ>fossy: and the NASM versions probably don't help much either
<OriansJ>but I do have C code versions of all of the steps as well
<fossy>OriansJ: not if you want to verify the bootstrap from teh ground up
<OriansJ>markjenkinsznc has some python version if I remember correctly.
<OriansJ>to civodul's question about source; nothing lower than M2-Planet in the bootstrapping stack is in a preferred form for modifications. As writing Assembly always sucks; it just sucks less than trying to write those pieces in FORTH.
<OriansJ>hence why hex0, hex1, hex2, M0 and cc_* are barely touched after creation.
<stikonas>fossy: small question about m4
<fossy>stikonas: go ahead
<stikonas>it uses <sys/signal.h> but we only install it to signal.h, should I patch / sed it or do we want to install it there too?
<ericonr>stikonas: if you install something into sys/signal.h, it should be a header that just does "#include <signal.h>"
<stikonas>ericonr: yeah, but m4 includes #include <sys/signal.h>
<fossy>stikonas: patch it
<stikonas>with patch or sed?
<ericonr>but yeah, that also works :P
<fossy>i dont really care, whatever you like
<stikonas>ok, I think Perl's Configure is handwritten
<stikonas>but it's 25K lines of shell scripting...
<xentrac>what, did the "Congratulations, you're not running Eunice." tip you off?
<stikonas>no, I just ran git blame :D
<stikonas>hmm, actually maybe it's not manual...
<stikonas>there is this thing: +# Generated on Tue Sep 12 17:23:02 EET DST 2000 [metaconfig 3.0 PL70]
<stikonas>ok, so we need to build this first https://github.com/Perl/metaconfig
<stikonas>and of course it's written in Perl
<stikonas>ok, I think I'll have to work with much older perl
<stikonas>which have very few c files and can probably be built manually
<stikonas>fossy: patch format fixed and tested
<xentrac>stikonas: the Perl building process starts by building a thing called "miniperl"
<xentrac>at least last time I ran it, which I'm pretty sure was more recent than Perl 5
<xentrac>not sure if it's powerful enough to build Configure with
<stikonas>xentrac: oh do you have any links?
<stikonas>I might try to look at it tomorrow
<xentrac>no, but if you run Configure I think you'll probably observe it
<stikonas>hmm, it looks like miniperl is perl without ability to load dynamic shared modules
<OriansJ>so static perl
<stikonas>yeah...
<stikonas>maybe that's good enough to drive autotools...
<xentrac>how much of configure do you need to run before you can build minnie pearl?
<xentrac>*miniperl
<stikonas>I guess the whole...
<stikonas>hmm
<xentrac>really?
<stikonas>well, in principle we can run configure...
<stikonas>if we have bash
<stikonas>but it's not really "source"
<stikonas>well, maybe one can write makefile to build miniperl...
<stikonas>altough, perl's configure is still more "source'y
<stikonas>than autotools configure
<stikonas>but I guess not good enough for "full" bootstrapping
<stikonas>well, I guess the way to go is to write custom makefile for miniperl...
<stikonas>and see if it's useful
<stikonas>miniperl is built fairly early when you run make
<fossy>stikonas: do you have any idea why i might be getting an absolute ton of undefined symbols wehn linking bash
<fossy>my rule is $(CC) -o bash $(LDFLAGS) $(LIBRARIES) $(OBJS) -lsh -lbuiltins -lglob -ltilde
<stikonas>fossy: do you have .a library there?
<stikonas>in $LIBRARIES I guess?
<fossy>stikonas: yeah
<stikonas>hmm
<fossy>libsh.a libbuiltins.a libglob.a libtilde.a
<stikonas>I would grep for thoose symbols
<fossy>i have
<stikonas>and see where in the source they are
<fossy>the only relation i can find between all of them is that they all have externs
<stikonas>hmm
<fossy>some arent even in libraries
<stikonas>and what is in LDFLAGS?
<fossy>just -L.
<stikonas>I guess that should be sufficient...
<stikonas>well, maybe push something to work branch
<stikonas>or paste makefile...
<fossy>ya i will i guess
<stikonas>I'll try to push m4 too...
<stikonas>testing it now
<stikonas>fossy: ok, I have working m4, do you want a new PR?
<stikonas>or should I just push on top of bzip
<fossy> https://github.com/fosslinux/live-bootstrap/pull/12
<fossy>stikonas: new pr
<fossy>oh
<stikonas>can you merge bzip then?
<fossy>well
<stikonas>hmm
<fossy>ill merge bzip rn
<stikonas>ok
<stikonas>otherwise it's hard to do PR on top of PR...
<stikonas>ok, m4 is here https://github.com/fosslinux/live-bootstrap/pull/13
<stikonas>fossy: by the way you can probably delete "patch" branch now
<fossy>stikonas:oh yeah
<stikonas>fossy: I'm getting ./mksyntax -o syntax.c
<stikonas>probably something bad in Makefile withthat mksyntax object
<stikonas>even before linking
<fossy>oh whoops
<stikonas>hmm, I don't see what's wrong...
<fossy>pushed fix
<fossy>it was MKBUILTINS instead of MKSYNTAX_OBJS
<stikonas>ok, I'm getting getopt optarg and optind undefind symbols here
<stikonas>when building mksyntax
<stikonas>are these the same?
<stikonas>fossy: probably something tcc related
<stikonas>I get the same error when running normal configure
<fossy>stikonas: oh
<fossy>another whoops
<stikonas>probably more patching is needed
<fossy>pushing...
<fossy>stikonas: i know what the mksyntax problem was
<fossy>i forgot to link against getopt (did it in testing but not in the makefile)
<fossy>dere
<stikonas>oh ok
<stikonas>well, it should work in the end...
<stikonas>guix builds it
<fossy>yeah it should
<fossy>idk what im missing tho
<stikonas>tcc: error: file 'bash.c' not found
<fossy>what, that didn't happen even when i ran the makefile in the thing
<fossy>live environment
<stikonas>oh
<fossy>and this is a direct copy of it now...
<stikonas>hmm
<fossy>hmmm
<stikonas>maybe make is older there
<stikonas>I'm testing with newer make
<stikonas>but $@.c on the last line is suspicious
<stikonas>why is there .c file in linking stage?
<fossy>oh yeah
<fossy>why is that there
<fossy>i'm thinking the copy-paste was not completely successfull
<fossy>lol
<fossy>i think this is better
<stikonas>ok, got those symbols
<stikonas>not sure if I'll have time before bed though
<stikonas>but let me quickly see
<fossy>that's fine, i'll look at it later today anyway
<stikonas>fossy: looks like missing defines
<stikonas>those are all inside defines
<fossy>oh...
<fossy>fk
<fossy>ok
<stikonas>ok, I'll let you look at it later...
<fossy>yeah i will
<stikonas>I'll check tomorrow if you are still not done
<deesix>OriansJ, the 'TMP' branch of my M2-Planet repo has test0021 adapted to use M2libc. Feel free to cherry-pick bcd0f498591521ddbcb43456142c8beab964a9f0
<deesix>*AArch64 test0021
<deesix>Now that all (but test1000) are done, I'll use AArch64 as the base of script unification, during the following days.
<deesix>yt_ ^ FYI too.
<xentrac>this is an interesting bootstrapping project: https://slawekk.wordpress.com/2019/11/10/lean-and-metamath-zero/
<xentrac>metamath zero is a proof checker; you feed it proofs you have constructed by some other means (probably an interactive proof assistant) and it verifies that they prove what you say they do. the proposal there is to formalize enough of the x86 instruction set that MM0 can verify itself
<gforce_d11977>fossy: live bootstrap should work on arm too, if have seen e.g. https://github.com/oriansj/mescc-tools-seed/blob/master/armv7l/GAS/hex0_armv7l.S
<fossy>gforce_d11977: yes, but none of the required code is there
<fossy>gforce_d11977: for example, we only copy x86/ in, and idk if wip-m2 works on arm fully, or if tcc is up to the required standard
<fossy>signfiicant progress on bash
<fossy>stikonas[m]: it was not defines
<fossy>it was simply the required symbols were not there, for whatever reason
<fossy>(including: a missing object on the command line, mkbuiltins clobbering files, etc
<gforce_d11977>fossy: ok, but the progress for aarch64 is more complete, is'nt it?
<gforce_d11977>(good morning to everyone!)
<gforce_d11977>(sorry, at least in germany)
<fossy>gforce_d11977: idk.. more so than arm in M2-Planet/mescc-tools, but i think less so in tinycc, janneke can correct me on that
<fossy>but much less complete than x86 for sure
<janneke>right, mes, mes libc and bootstrappable-tinycc have support for arm, although gcc-4 isn't bootstrapped yet
<malina>morning
<malina>gforce_d11977,
<bauen1>xentrac: that's very intersting, reminds me of how some math problem was solved, a huge complicated automated proof assistant (?) that nobody really understood, and a very small well understood and verified proof checker (i think coq)
<stikonas>fossy: nice, so does bash link now?
<bauen1>would a bootstrap kernel need to support SSE or floating point operations, or can you compile linux (and its dependencies) without it ?
<bauen1>or rather does printf need to support the 'e', 'f' values
<stikonas>hmm, probably not...
<stikonas>although, I'm not sure about linux...
<bauen1>just needing to decided if i want to implement sse userspace support or just strip out anything "float" related from musl libc
<bauen1>alright, wasn't too hard to strip it out, so i'll just go that way for now
<fossy>stikonas: very nearly almost
<fossy>3 undefineds left
<stikonas>ok, that's indeed very close
<stikonas>can you also merge m4 at some point?
<fossy>ya
<fossy>after i merge bash i'll move m4 to the bash script
<stikonas>yeah, that's fine
<stikonas>hmm, maybe I should split m4 patch
<stikonas>into two
<fossy>one thought, maybe we should include a mk/ directory like we do for patches
<fossy>i see only one hunk
<stikonas>yeah, that @if hunk
<fossy>?
<stikonas>hmm, maybe we need to think how to put everything, not just make files
<stikonas>kaem too
<stikonas>fossy: @if hunk needs bash in makefile
*fossy is confused what patch you mean
<stikonas>oh, nevermind
<stikonas>I was confused
<fossy>i only see this patch
<stikonas>that was probablybzip2
<fossy> https://github.com/fosslinux/live-bootstrap/blob/54a39c2e33c8dc99b5b18099a76c681e39929553/sysa/patches/m4-1.4/m4-signal-include.patch
<stikonas>yeah, this is fine
<fossy>there merged
<stikonas>ok, thanks
<stikonas>I mixed it up with https://github.com/fosslinux/live-bootstrap/blob/master/sysa/patches/bzip2-1.0.8/bzip2-1.0.8.patch#L40
<fossy>lol
<stikonas>too much building :D
<stikonas>fossy: so I'm fine with mk directory, although, it might be nice to put all build scripts there...
<stikonas>otherwise after folder is getting messy
<stikonas>hmm, on the other hand, some kaem files are more complicated (tcc kaem files work with multiple folders)
<stikonas>ok, maybe just keep mk for makefiles...
<siraben>xentrac: Haskell is definitely powerful enough to implement a metamath checker
<siraben>heh wouldn't that be fun later, bootstrap mathematics AND compilers
<bauen1>siraben: i think that would be a very good idea, from my limited understanding of formal methods they also rely on all the underlying compilers / kernels / libcs to be "bug-free"
<siraben>yes, they also need trusted cores. Coq's core is only a few hundreds lines IIRC
<siraben>You can't prove the consistency and completeness of a sufficiently expressive theorem prover within itself, Gödel strikes again!
<stikonas>fossy: we also need bison before autotools
<stikonas>I'll see if I can workaround it with different versions...
<xentrac>siraben: you can implement a metamath checker in any programming language pretty easily
<rain1> https://github.com/nineties/planckforth
<rain1> https://github.com/nineties/planckforth/blob/main/others/planck.c#L74 yes
***ChanServ sets mode: +o rekado_
***rekado_ is now known as rekado
<siraben>xentrac: oh interesting
<xentrac>one of the smallest metamath verifiers is this one by raph levien: https://github.com/david-a-wheeler/mmverify.py/blob/master/mmverify.py
<xentrac>there's a variant at http://us.metamath.org/downloads/mmverify.py
<xentrac>there are a lot of different metamath verifiers, the idea being that we'd like to not depend on a single possibly buggy and exploitable program to check a metamath proof
<xentrac>to tackle the problem bauen1 mentioned, the current proofs of seL4 (in Coq) verify not just the source code (of the kernel!) but the executable code, although they fail to verify if you use sufficiently high optimization levels
<xentrac>and related to the blog post I linked above, Carneiro is trying to do something similar with Metamath: https://arxiv.org/abs/1907.01283
<xentrac>the idea there is that by presenting a proof of correctness of a particular Metamath verifier (in x86 machine code, again, not source code) you can use any other Metamath verifier on *one* proof to verify that the x86 executable is correct for *all* proofs
<xentrac>I don't know if there's a proof of consistency of the Metamath Zero core language that these verifiers verify, but certainly it is not intended to be complete
<xentrac>so there are presumably theorems that are true but which Metamath Zero cannot prove; we hope that those theorems don't happen include too many theorems of interest
<OriansJ>deesix: your commit has been merged
<OriansJ>gforce_d11977: although I have prototyped the steps for armv7l; I have not actually converted them into hex0, hex1, hex2 and M0.
<OriansJ>Which is not a small task and is rather tedious
<OriansJ>which at this moment isn't a top priority to get it done.
<OriansJ>bauen1: well in absolute terms. Compilers don't require floating point support from an engineering standpoint but some compilers do use it for fuzzy approximation to do some things faster.
<xentrac>or for compile-time constant folding
<xentrac>when they're compiling code that uses floating-point support
<OriansJ>xentrac: wouldn't that fall under doing some things faster?
<siraben>xentrac: thanks for that information, I'll look into it
<bauen1>OriansJ: true, in absolute terms you also don't need a compiler :)
<xentrac>OriansJ: yes, but not by way of fuzzy approximation
<gforce_d11977>OriansJ: sorry no pressure, armv7l comes later 8-) for now, there enough polishing to do for x86_64, x86, the rest comes later, and maybe *someday* i will do the lowlevel stuff for MOS6502 and m68k 8-)))
<fossy>stikonas: hm
<fossy>wrt bison, I just realised bash and patch are shipping precompiled bison parsers
<fossy>we should follow gios bootstrapping path for flex and bison
<stikonas>fossy: oh, yeah
<stikonas>I noticed that when I tried to build gawk
<stikonas>fossy: bison is a bit tricky but probably not impossible
<stikonas>it's quite a new bison
<stikonas>easy to do that with gcc and autotools
<stikonas>fossy: what file is precompiled in patch?
<stikonas>I don't see anything in patch at first glance
<fossy>stikonas: I think it is called y.tab.c
<fossy>Or parse.c
<stikonas>hmm, I don't see these files in patch
<stikonas>parse.y in bash indeed
<stikonas>maybe only bash then?
<stikonas>fossy: there is one in tar though
<stikonas>I ran in our sources dir for file in *; do lsar $file | grep '\.y'; done
<stikonas>tar-1.12/lib/getdate.y
<stikonas>and getdate.y is used...
<fossy>oh what
<fossy>well we cab disable that, surely
<stikonas>fossy: yeah
<stikonas>I was able to patch it out locally
<stikonas>I'll prepare a proper patch
<stikonas>and I think I can do it with just sed
<stikonas>so good that we didn't move sed after tar
<fossy>yeah lol
<stikonas>bash is more annoying...
<stikonas>anyway, we need to bootstrap bison 3.4.1
<fossy>bash is nearly done, I didnt get anything more done since my last status update
<stikonas>but I guess parse.c is essential there?
<fossy>yea
<stikonas>ok, I think it's not the end of the world...
<stikonas>bison should be compilable with tcc
<stikonas>although, might need some small patches
<stikonas>some quick attempt showed some problems with getprogname
<stikonas>anyway, tar first...
<fossy>1 sec im getting a link
<stikonas>it's here https://gitlab.com/giomasce/bison-bootstrap
<stikonas>oh, link for bash?
<stikonas>and I thought people in early days of GNU cared about bootstrapping more than now...
<stikonas>now I can see it's so entangled...
<fossy> https://gitlab.com/giomasce/nbs/#what-happens-then
<fossy>no not for bash
<fossy>read the last 4 dot points in that section
<fossy>thats the easiest path
<stikonas>ok, we already have m4
<stikonas>fossy: what about dash?
<stikonas>instead of bash?
<fossy>like before bash?
<fossy>we need bash for autotools I think
<fossy>but dash would work for a bit
<fossy>so the primary issue is bash...
<fossy>➜ find . -name "*.y"
<fossy>./tar-1.12/lib/getdate.y
<fossy>./bash-2.05b/parse.y
<stikonas>ok, I'm already testing tar
<fossy>hm, maybe dash is sufficient to run a configure script
<stikonas>dash might be sufficient
<fossy>it seems to be on my host system
<stikonas>but that doesn't solve autotools bootstrap issue
<stikonas>it's just that we can get some shell earlier
<stikonas>no shell is a bit annoying
<fossy>yeah
<fossy>we need a shell for a number of features
<fossy>and we need bison for autotools
<fossy>well, i guess i'll finish bash and leave the PR open
<stikonas>yeah...
<fossy>then i'll do dash instead, then we can do the flex/yacc/bison dance
<stikonas>I'll let you know once my tar testing is done (https://github.com/fosslinux/live-bootstrap/pull/14)
<stikonas>and maybe looke a bit at bison
<stikonas>since I already have submodule imported in a local branch
<stikonas>fossy: or actually any other shell would work too... but I guess dash might be simplest...
<fossy>stikonas: dash is probably simplest, yea
<stikonas>fossy: I think our libc is insufficient to compile heirloom tools lex/yacc
<stikonas>ldefs.c:46: error: include file 'wctype.h' not found
<stikonas>let me see if it depends a lot on it...
<fossy>stikonas: odd, might be able to disable it...
<stikonas>possibly...
<stikonas>there are other problems though...
<stikonas>fossy: oh, tar change succeeded
<stikonas>you can merge now
<fossy>cool
<OriansJ>gforce_d11977: well the 68K could definitely run the steps for bootstrapping GCC but the 6502 would have to do a sweet-16 to run cc_* without some serious modifications as the 6502 has a rather small hardware stack (256Bytes)
<fossy>not cool
<fossy>bash links now but segfaults on startup
<stikonas>fossy: well, we need bison for bash anyway...
<stikonas>maybe then leave it for now?
<stikonas>if we dash is enough to run autotools
<stikonas>we can later compile bash using configure