IRC channel logs

<fossy>Googulator: I'm fairly happy with the openela diff. The changes are not tooo big

<fossy>Googulator: also, OpenSSL 3 i have no qualms against in theory, i'll review the PR shortly

<stikonas>fossy: I think I can also remove this patch https://github.com/fosslinux/live-bootstrap/blob/master/steps/musl-1.1.24/patches/stdio_flush_on_exit.patch

<fossy>hm, swear i ran into something trying to do OpenSSL 3 earlier, although very high chance that has been resolve acidentially in the meantime

<stikonas>this is probably leftover from that weak symbols hack

<stikonas>fossy: no, you removed that package

<stikonas>it was xbps

<fossy>ah

<fossy>yep

<Googulator>xbps was incompatible with newer openssl?

<stikonas>at least at that time it was

<fossy>stikonas: that looks removable to me, if you could check or chuck it in a PR so it can be tested that would be good

<stikonas>yeah, I just kicked off a run, it's happily continuing

<stikonas>just need to wait for checksums

<fossy>:D

<fossy>stikonas, Googulator: #454 good to merge?

<Googulator>I'm still doing final tests on it; the download-distfiles.sh and generator.py paths are good, still waiting for it to reach the helpers.sh path

<fossy>wth, I'm reading that autoconf ML thread, there are some bad takes in there

<Googulator>one of the worst is where someone suggested maintainers should just edit config.guess instead of doing autoreconf...

<Googulator>config.guess is AFAIK a generated file

<fossy>"GNU/Linux distributions have a long history of buggy backports to the autotools"

<fossy>lol

<fossy>perhaps because autotools has so much black magic

<fossy>and is very difficult to patch witout introducing bugs -.-

<Googulator>and the fact that they suggested editing config.guess makes me wonder if any official GNU release tarballs have autotools outputs that were hand-edited by the maintainer, maliciously or not

<stikonas>yeah, #454 should be good

<fossy>the answer to that i'd say is probably "yes", certainly for some pregenerated files, if not autotools

<stikonas>one good thing out of xz thingy might be some projects moving away from autotools

<fossy>yes please

<stikonas>well, but it souldn't happens for GNU projects...

<stikonas>which is what we mostly work with in live-bootstrap

<fossy> https://lists.gnu.org/archive/html/bug-autoconf/2024-04/msg00036.html

<fossy>lol, what

<fossy>this is legally correct, but completely tone-deaf

<stikonas>yeah, strictly speaking you only need to provide source on written request

<stikonas>or something like that

<stikonas>but yes, that reply just sounds a bit obstructionist...

<fossy>then earlier, this same person said this; https://lists.gnu.org/archive/html/bug-autoconf/2024-04/msg00026.html

<fossy>that "I have chosen to give tarballs to my users"

<Googulator>"(or, well, GNU has chosen for me, even before I was present)"

<stikonas>well, back then it might have made more sense

<Googulator>or, you know, even before Git was invented, and _long before_ any GNU project adopted serious source control

<stikonas>but it doesn't mean that all is set in stone

<Googulator>tarballs are the only thing you can release if the sole authoritative copy of the upstream source resides in a directory on the maintainer's hard drive (hopefully they have backups at least...)

<stikonas>fossy, Googulator: should we also try to fix those non-redistributable files (heirloom-tools) before 1.0?

<Googulator>IMO if we can do it with plan9 sources, then yes

<fossy>stikonas: hm, probably, did we have an idea for how to fix that?

<fossy>ah plan9

<stikonas>well, we somehow need to get gash to work then...

<Googulator>but I wouldn't add more Lisp/Scheme dependency for it

<fossy>probably not worth it in that case if we need gash

<stikonas>but it's probably not very well documented how to run it (without ./configure and all that generated stuff)

<stikonas>yeah, ok, we could postpone it then

<stikonas>plan9 also has some lexer?

<Googulator>plan9 has MIT-licensed versions of lex and yacc

<stikonas>well, in fact we only need to rebuild 1 file

<stikonas>the one for bash

<stikonas>that would allow us to reach musl

<stikonas>and then we could use whatever works for flex/bison bootstrap (could even keep heirloom-tools)

<stikonas>(or test plan9 tools)

<Googulator> https://github.com/plan9foundation/plan9/blob/main/sys/src/cmd/yacc.c and https://github.com/plan9foundation/plan9/tree/main/sys/src/cmd/lex

<Googulator>we would probably need to fork these and port to meslibc

<Googulator>oddly enough, lex includes a .y file

<Googulator>luckily no .l

<Googulator>and no .l or .y in yacc's source

<stikonas>well, lex is not necessary for bash

<stikonas>but yeah, it has header dependencies...

<Googulator>helper.sh part of #454 is fine too

<stikonas>there is also this https://invisible-island.net/byacc/

<Googulator>but no blex

<stikonas>yeah, but it would solve license issue

<stikonas>byacc -> bash -> musl -> heirloom-tools -> flex -> ... -> bison

<stikonas>though that's an extra tarball

<stikonas>another 885 KiB

<stikonas>it does seem buildable by tcc/mes at first glance

<stikonas>though one would have to write custom makefile

<stikonas>as this is autotooled

<stikonas>perhaps I'll investigate this at some point later

<Googulator>there's a Makefile.old

<Googulator>which seems to be a proper ready-made makefile

<stikonas>oh indeed

<stikonas>make: *** No rule to make target 'skeleton.c', needed by 'skeleton.o'.

<stikonas>maybe needs some light patching

<stikonas>but anyway, it would be easier than writing completely from scratch

<Googulator>I wonder why we build flex at the point where we do

<Googulator>before musl

<Googulator>does musl need flex?

<stikonas>no, but it's a bit for convenience

<stikonas>flex needs the "same libc" build of heirloom tools

<stikonas>we could move flex after musl

<stikonas>but then we need to do musl -> rebuild heirloom-tools -> flex

<stikonas>with this in mind flex could be moved

<stikonas>the only blocker is bash

<Googulator> https://gist.github.com/Googulator/9eee5c0ccd10fd819bf937b4f650b52f

<Googulator>builds byacc successfully without autotools *on a modern Linux distribution*

<Googulator>(not tested in the early bootstrap env)

<stikonas>yeah, I think this would work

<stikonas>defs.h has standard headers

<stikonas>which I think mes provides

<stikonas>and this yacc seems to cope well with parser.y from bash

<Googulator>#456 created for byacc and added to roadmap

<stikonas>thanks!

<stikonas>issue #456 nicely summarizes all the information that we discussed here

<fossy>thanks for the summary Googulator :)

<fossy>the path forward seems reasonable to me

<stikonas>indeed...

<stikonas>strange that we didn't consider it before

<fossy>did we know byacc exists

<fossy>(i have never seen it before today)

<stikonas>no, I didn't know about it either

<stikonas>just found it today on Wikipedia

<stikonas> (https://en.wikipedia.org/wiki/Yacc)

<stikonas>this page had a link to https://en.wikipedia.org/wiki/Berkeley_Yacc

<stikonas>and if this is much better than heirloom yacc, it migth simplify bison's bootstrap

<stikonas>and if not, we'll just keep the current path

<oriansj>well finding improvements is likely to continue for years (potentially decades); there are a great many enhancements and ordering tweaks possible.

<Googulator>I wonder who has the ability to update bootstrappable.org

<Googulator>Bad news; fpos_t isn't supported by tcc/meslibc

<Googulator>needed by byacc

<Googulator>currently trying replacing fgetpos/fsetpos with fseek/ftell using sed

<fossy>not sure a simple find/replace will work, but fgetpos/fsetpos should be fairly easily replaceable with fseek/ftell i believe

<Googulator>there's only a single use of each, so a simple replacement is enough

<Googulator># meslibc has no fgetpos/fsetpos - emulate using fseek/ftell

<Googulator>sed -i -e "s/fpos_t/long/" -e "s/fgetpos(f, &save_area.line_fpos) != 0/(save_area.line_fpos = ftell(f)) == -1/" \

<Googulator> -e "s/fsetpos(input_file, &save_area.line_fpos)/fseek(input_file, save_area.line_fpos, SEEK_SET)/" reader.c

<Googulator>meanwhile, openela kernel PR is now in CI

<Googulator>will soon test it on bare metal too

<Googulator>another missing meslibc function: tmpfile()

<Googulator>this time, I'm emulating it by just opening a file in the current directory, so the resulting byacc will be a bit "messy", but it should work

<fossy>Googulator: could you take a look at https://github.com/fosslinux/live-bootstrap/actions/runs/8678388667/job/23795249731

<fossy>not immediately sure what's wrong

<stikonas>well, byyacc is not immediately helpful in bison bootstrap, it can't build bison parser. It can do flex parsers but we need heirloom-tools for lex files there anyway

<stikonas>(possibly we could skip building heirloom yacc...)

<stikonas>but that's not changing much

<oriansj>Googulator: rekado is the one who generally updates the site

<oriansj>also adding C functions in libraries should be relatively easy to do; so instead of replacing, improve meslibc just a little.

<Googulator>fossy: curl: (22) The requested URL returned error: 502

<Googulator>so just the usual savannah instability

<Googulator>oriansj: I'd rather not wait for mes 0.27; LB 1.0 has been dragging out for way too long already

<Googulator>& of course we can't really patch mes ourselves

<Googulator>we don't even have sed when we build mes

<Googulator>only options are simple-patch and rolling our own prepatched mes tarball

<oriansj>Googulator: we have replace.c which can be used for simple patching

<oriansj>(part of mescc-tools-extra)

<oriansj>and we could definitely do a simple sed in mescc-tools-extra if we need something more advanced.

<Googulator>turns out we do already have patch-2.5.9 when we build byacc (though not when we build meslibc)

<Googulator>but replace/sed is probably not sufficient for adding entire new functions to meslibc

<Googulator>oriansj: looks like we need to start relying on your mirrors of savannah tarballs

<Googulator>what was the address for the mirror again?

<stikonas>Googulator: you could just patch meslibc before byacc

<stikonas>by then patch and sed are both available

<stikonas>and also submit that function to mes as patch

<stikonas>so that is is there for next release

<Googulator>meslibc is compiled by then

<Googulator>or just rebuild tcc+meslibc?

<stikonas>just rebuild meslibc (libc.a)

<stikonas>no need to rebuild tcc or anything else

<Googulator>oh right, we don't have the package system in place yet at that point

<stikonas>tcc is quite fast

<stikonas>so it wouldn't add much time to the build

<stikonas>only the build of meslibc with mescc is slow

<Googulator>Pushed an updated byacc to the byacc branch; this one actually works when built against meslibc

<Googulator>looking promising - bash builds fine and everything up to and including m4, if I remove heirloom and flex-2.5.11 from the manifest, and just install byacc as yacc

<Googulator>all that's needed basically is to convert heirloom's build script from kaem to sh

<stikonas>well, that's just normal work

<stikonas>shouldn't cause any surprises

<stikonas>Googulator: so you are inserting heirloom tools and flex 2.5.11 just before flex 2.6.4?

<Googulator>yes, that's the plan

<stikonas>well, all bison bootstrap steps will be in one place...

<stikonas>not sure if you saw my testing earlier, but byacc can't help with bison bootstrap itself

<stikonas>bison's .y files are too complex

<Googulator>also, we do need to keep heirloom lex

<Googulator>because there's no blex

<stikonas>exactly

<stikonas>I did try to search online, but didn't find any other lex

<stikonas>but anyway, that's fine

<Googulator>there's reflex by the same author who maintains byacc, but of course it depends on itself :(

<stikonas>the main thing was getting rid of non-redistributable binaries

<Googulator>& if we're keeping heirloom lex, it's easier to keep it working together with heirloom yacc than trying to get some byacc + heirloom lex hybrid working

<stikonas>yeah, I agree

<stikonas>and building byacc is also easier than getting gash to work

<Googulator>heirloom is basically sysv lex/yacc, so I wouldn't expect compatibility with byacc which is BSD yacc

<Googulator>not to mention not needlessly increasing our dependency on Lisp-like languages

<Googulator>in general

<Googulator>and Mes in particular

<stikonas>well, guix people would disagree with your but yes, I find it harder to work than e.g. C

<stikonas>s/your/you/

<Googulator>"Lisp is so easy, Emacs does everything for you"...

<Googulator>another side effect of moving heirloom after bash: no more "uninstall: /usr/..." directives in the manifest

<Googulator>we can just "uninstall: heirloom-devtools-070527"

<stikonas>oh nice

<stikonas>well, uninstalling is also less important

<stikonas>before this I really wanted to rm -rf heifloom as soon as possible due to non-redist

<Googulator>parse.y:1769: error: 'yy_n' undeclared

<Googulator>is this error familiar to anyone?

<Googulator>(I get this when building parse.y in flex)

<Googulator>or rather parse.c that heirloom lex generates from parse.y

<stikonas>hmm, not that I can remember...

<Googulator>"parse.y:1769" makes no sense

<Googulator>parse.y has no line 1769

<stikonas>which flex is that?

<Googulator>parse.c does, but it has no "yy_n" anywhere near it

<Googulator>the first one, 2.5.11

<Googulator>now trying a build where I strip all #line directives from parse.c before building it - hopefully tcc will output something more meaningful then

<stikonas>and yacc is from heirloom as we discussed?

<Googulator>yeah, heirloom yacc & lex

<Googulator>or at least should be

<stikonas>yeah, those bison files do mess up error reporting a bit

<stikonas>Googulator: hmm, given that we build this after musl

<stikonas>do we need to adjust some CPPFLAGS in makefile?

<Googulator>CFLAGS = -DVERSION=\"2.5.11\"

<Googulator>LDFLAGS = -static

<Googulator>that's all we have

<Googulator>no CPPFLAGS

<stikonas>hmm, yeah

<stikonas>well, that DVERSION I guess should be CPPFLAG but oh well...

<Googulator>tcc invocation in makefile also only references CFLAGS

<Googulator>not CPPFLAGS

<stikonas>well, doesn't matter, I was too sloppy when I was writing those makefiles

<stikonas>so just used CFLAGS everywhere

<stikonas>rather than making distinction between CFLAGS and CPPFLAGS

<stikonas>anyway, that's mostly just the convention

<Googulator>especially since tcc is all-in-one

<stikonas>indeed...

<Googulator>preprocessor, compiler, assembler and linker

<stikonas>anyway, see if you can get a better error message...

<stikonas>because I don't see why moving flex after musl would break it

<Googulator>parse.c:1083: error: 'yy_n' undeclared

<Googulator>a little bit more realistic

<stikonas>well, I guess it is related to 1769 line of parse.y

<stikonas>i.e. C file was created by something there

<Googulator>there's no line 1769 in parse.y

<Googulator>parse.y is only 1013 lines long

<Googulator>but at least parse.c:1083 does indeed reference yy_n

IRC channel logs

2024-04-14.log