IRC channel logs

2021-03-27.log

back to list of logs

<stikonas>a few small problems with gcc...
<stikonas>not sure why, but fairly minor
<fossy>e.g.?
<stikonas>config.sub is not recreated
<fossy>huh
<stikonas>I just copied it from /after/share/automake-1.9/config.sub
<stikonas>oh well...
<fossy>wait do we need to put that in the gcc: Regenerate prebuilt files also
<stikonas>and sys/limits.h is missing
<stikonas>not sure why it worked before, but symlink fixes that
<stikonas>put what in the gcc?
<fossy>is config.sub issue fixed
<stikonas>yes
*fossy is reasonably suprised all these new autotools work on perl 5.6
<stikonas>oh, into that issue
<fossy>ok neat
<stikonas>yeah, I need to rerun a few times to get new checksums
<stikonas>but it should work
<stikonas>xz a bit more work though
<stikonas>so should we reinstate older bash?
<stikonas>although, if it works, maybe something like 4.2 would be better
<fossy>4.x was the newest avaliable working with tcc eyah?
<stikonas>5.1 was working
<stikonas>but some issues
<stikonas>musl configure script segfaulted
<fossy>oh right
<stikonas>and with libtool moved earlier we now get a bit fewer warnings
<fossy>we probably don't want 5.1 then
<stikonas>there are still come unitialized config warnings from perl...
<fossy>i wonder if we could go with 4.x ~50?
<fossy>~ part 50
<stikonas>probably...
<stikonas>but worth rerunning musl's configure as a "test case"
<stikonas>to see if it works well enough
<fossy>yes
<stikonas>and regarding perl, it's possible to go to 5.8 without too much trouble
<stikonas>although, that's still a bit of work
<stikonas>but I don't think there are any fundamental problems
<stikonas>it's just that I didn't need newer perl yet
<stikonas>in general it's surprising how much stuff runs on tcc
<stikonas>we'll almost have a usable system built just with tcc
<stikonas>fossy: hmm, so do I not need to run libtoolize then in gcc build?
<stikonas>if I have libtool 2.2.4?
<stikonas>oh, maybe we do
<stikonas>that thing is not autoreconf'ed...
<stikonas>nevermind then
<fossy>stikonas: it should be autoreconfed...
<fossy>oh
<fossy>unless there are no libtool-generated files
<stikonas>no, gcc top folder is special
<fossy>yes
<stikonas>it's not normal autotools
<fossy>yes
<stikonas>it needs autoconf...
<stikonas>but it does not use automake
<stikonas>so for gcc I'll keep libtool...
<fossy>yeah ok
<stikonas>oh, although, if I move libtool earlier before GCC
<stikonas>I think I don't need to rebuild 1.4 after binutils
<stikonas>hmm, there is only help2man in the middle
<stikonas>and it deos not use libtool
<nimaje>is there some graph showing how far you got? or maybe a tombstone diagram? (but I'm not sure if a tombstone diagram can show the dependencies good enought)
<stikonas[m]>nimaje: only text, no graph
*fossy sighs
<fossy>i have decided to move on from kexec-tools for now
<fossy>i'll introduce it post-binutils, that is easy - i'll push an experimental branch but not PR it
<fossy>the very first part of the bootstrap needs a re-thought
<fossy>but we can come back to that later.
<stikonas[m]>Oh, can't build them without ar?
<fossy>stikonas[m]: no i think tcc's assembler is really just too poor for it
<fossy>the qemu is exiting on kexec -e
<fossy>i think it's something to do with the purgatory binary
<fossy>which is *really* hard to debug
<stikonas>oh ok, so we need as...
<fossy>i think so
<stikonas>maybe you need to wait for OriansJ's M3
<stikonas>although, I don't remember if it includes assembler
<stikonas>hmm, almost done moving stuff before gcc...
<fossy>cool
<fossy>i looked at guile - it looks doable
<stikonas>xz is left for now
<fossy>i will work on it tomorrow
<stikonas>either we patch it, or find the patch which fixes statics...
<stikonas>(not sure if it's even just one patch)
<stikonas>(I mean tcc patch from mob branch)
***mephista is now known as spicy_icecream
<OriansJ>stikonas: yes M3 will have a linker, assembler, a Stand-alone C preprocessor and a TCC compatible compiler (if other pieces are needed, I'll do them too)
<OriansJ>The linker should be done first (it is half-way done already https://github.com/oriansj/M3-Meteoroid) then I'll be working on the assembler (Which should be a drop in compatible with both TCC's assembler and binutil's assembler) which should enable us to scratch off assembler/linker issues in the bootstrap
<stikonas>yeah, so I think that will enable fossy to build kexec-tools earlier
<OriansJ>also if it would help I could produce a .dot file for live-bootstrap so that one can generate a pretty dependency graph for the pieces in it.
<OriansJ>as they are really easy to write: https://github.com/oriansj/talk-notes/blob/master/Current%20bootstrap%20map.dot
<stikonas>yeah, I've used dot a little bit before
<stikonas>at least for early bootstrap it should be possible...
<stikonas>later once you get to autotools it's getting tricker...
<stikonas>because things depend on too many things before them
<OriansJ>stikonas: fair but being able at a glance to know what is needed for each piece to be built might help people understand what is available when they want to upgrade a piece.
<stikonas>yeah, I can try to do the bottom at least...
<stikonas>we'll see how things look like there
<OriansJ>stikonas: well the bottom would be a direct copy up until after M2-Planet after that you really made a difference ^_^
<stikonas>hmm, I wonder how should I include kaem in there...
<stikonas>in principle everything at the botton uses it...
<stikonas>or we ignore it...
<bauen1>stikonas: make kaem a dependency of everything that is build using it ?
<bauen1>that would be the true way imho
<bauen1>same deal with bash / make
<bauen1>(which would then have a dependency on things build using kaem)
<stikonas>well, I tried to rework lower part a bit: https://stikonas.eu/files/T.pdf
<stikonas>that's for posix bootstrap...
<OriansJ>stikonas: I think you got it rather correct
<stikonas>now there will be a lot of arrows going down to catm...
<stikonas>live-bootstrap uses catm as empty file creation tool
<OriansJ>stikonas: unless we just add cat to mescc-tools-extra
<OriansJ>and > to kaem
<OriansJ>also there is the C version of catm in mescc-tools
<OriansJ>(also sharing of .dot files instead of pdf files enables others to build on one's work without duplicated efforts)
<stikonas>yeah, I'll share .dot file too
<stikonas>once I'm done today
<stikonas>at the moment it will just cause edit conflicts
<stikonas>while I'm actively modifying it
<stikonas>hmm, yes, there is c version of catm...
<stikonas>we can build it...
<stikonas>although, it's not done right now
<stikonas>but maybe I can remove those arrows...
<stikonas>hmm, although, that will just move them from catm to mescc-tools...
<stikonas>after tcc mescc-tools is not used anymore
<stikonas>oh well, if we count kaem, then it is...
<OriansJ>stikonas: it'll always be a changing document as we add and remove and change pieces.
<stikonas>well, yes, but I'll do a bit more and then make PR to live-bootstrap repo with dot file
<stikonas>I'll probably get bored of adding new nodes soon
<stikonas>got to musl and tcc rebuild
<stikonas>and then somebody else can take over
<OriansJ>stikonas: sounds fair
<stikonas>well, dot file is here https://stikonas.eu/files/live-bootstrap.dot
<stikonas>it's up to bison
<OriansJ>stikonas: thank you
<stikonas>I skipped a couple of minor steps (patching mes)
<stikonas>since I think those patches are upstreamed
<stikonas>it's getting quite messy in dependency graph...
<OriansJ>stikonas: I'll attempt to make it more clear
<stikonas>probably a better idea for colours would be
<stikonas>to maka boxes colour coded
<stikonas>instead of arrows
<stikonas>when something is optional
<stikonas>oh, already spotted a typo
<stikonas>herloom->heirloom
<OriansJ>I'll fix
<stikonas>also, dependencies are mostly like in live-bootstrap, e.g. m4 is built against musl in live-bootsrap, but it's buildable with meslibc too...
<stikonas>so "required" arrow is not always a strict dependency
<OriansJ>I'll break up the different versions and will be seperating orchestration tool dependency from build
<stikonas>yeah, maybe different colour?
<stikonas>although, sometimes bash is used as a dependency to create some scripts
<stikonas>to create some files
<stikonas>so it's not always orchestration tool
<stikonas>e.g. flex and bison do that
<OriansJ>thank you for that detail
<stikonas>basically I was looking at .sh and .mk files when putting dependencies
<stikonas>for now with our custom makefiles it's often fairly obvious what is used
<OriansJ>I guess I have a bit of reading to do and information to encode
<OriansJ>I'm putting my working draft in talk-notes and using https://paste.debian.net/1191234/ as an attempt to try to make it easier to understand (Still lots more to do)
<OriansJ>I was going to drop the done and essential details and use the lines to encode what the actually relationship is (like build, library, orchestration, etc)
<stikonas>yeah, I think that's a good idea
<stikonas>probably different colours too
<stikonas>e.g. C library can be one colour
<stikonas>compiler can be another
<stikonas>archiver (tar, gz, bzip2) yet another colour
<stikonas>essential vs non-essential doesn't matter as much later
<stikonas>e.g. make is not essential...
<OriansJ>stikonas: agreed. although I would put tar and compression tools under distribution
<OriansJ>and probably comment the crap out of everything along the way
<stikonas>this is also useful https://github.com/fosslinux/live-bootstrap/blob/master/parts.rst
<stikonas>although, it does not encode dependencies
<mihi>bauen1, if you want to output weird characters, you can always use hex. And instead of amending kaem to support > and <, I'd rather put it into another binary which only redirects streams and then execs the rest of the args.
<OriansJ>mihi: or just catm to create the desired file from sub-files
<mihi>yeah, if the thing you want to redirect is a cat, just catm it. If you want to "tee" some build logs to a file, you'd better have a "tee" executable though :)
<mihi>when the thing you want to redirect is your own tool, adding an -o parameter (like in my sha3sum) might be the better idea as well.
<stikonas>well, we build tee reasonably early
<stikonas>immediately after bzip2 (which is after make)
<mihi>normal tee requires pipe support (in the kernel)
<mihi>btw if you want a guile that does not come with any version of psyntax, you probably have to use 1.0 (the next available version on ftp.gnu.org, 1.3.4 from 1999, already has it)
<OriansJ>well it certainly lakes it more easy to figure out https://github.com/oriansj/talk-notes/blob/master/live-bootstrap.dot https://github.com/oriansj/talk-notes/blob/master/live-bootstrap.pdf
<stikonas>yeah, definitely colours help
<mihi>(this commit http://git.savannah.nongnu.org/cgit/guile.git/commit/?id=230c435383087a1e6ad60d9c98d3ec75dd2c3e49 from August 1997 introduced psyntax)
<OriansJ>mescc-tools certainly is a quite busy node (I might break it into pieces) as kaem is used a bit until make and bash replace it
<fossy><OriansJ> and > to kaem
<fossy>recall this must be an optional feature
<fossy>to avoid kernel support for piping
<stikonas>what about bash where it is used to generate files (not build orchestration?)
<stikonas>I think that's both flexes and bison
<mihi>fossy, why does > require pipes in kernel? As long as you can pass file descriptors on exec, you should be fine?
<stikonas>although, maybe it's fine to simplify diagram a bit...
<fossy>mihi: hmst, you might be right
<fossy>I have not thought of it that way
<mihi>and if your OS does not support pipes or multitasking, you can fake them like DOS did. Redirect first command to a temp file, read second command from temp file.
<fossy>mihi: how does one redirect stdout without pipes rhough
<fossy>(this is not an area I know much about)
<stikonas>mihi: and regarding guile, I think we don't know which versions we need...
<stikonas>I don't even know which versions of autogen would work
<mihi>fossy, you may want to read https://man7.org/linux/man-pages/man2/dup.2.html (in particular dup2 syscall)
<stikonas>OriansJ: should we add more arrows to catm?
<stikonas>a few nodes before coreutils use catm to create empty config.h files
<OriansJ>fossy: if we need it; it'll just be a problem to deal with in the kernel as well but if we can avoid it with something simple, probably a better plan.
<stikonas>well, it's not something we need very early in the bootstrap
<OriansJ>stikonas: probably but trying to figure out subgraphs first
<stikonas>yes, we build tee really early, but we are not using it
<stikonas>basically, redirection pipes and etc are only needed after bash
<OriansJ>stikonas: This all reminds me that I probably should add M2libc and catm (C version) to stage0-posix along with incorporate live-bootstrap's mes work.
<stikonas>yes, we can build C version of catm
<stikonas>and M2libc
<stikonas>well, at some point we should update bottom of live-bootstrap to new stage0-posix commits
<stikonas>and I guess at some point newer mes too
<stikonas>(possibly when 0.24 is released)
<fossy>stikonas: we need to wsit for new mes
<fossy>cause of m2libc
<fossy>its not compatible
<stikonas>oh I see
<stikonas>so it's at the same time...
<stikonas>fossy: by the way, I'm addeing bash 4.2.53 too
<stikonas>*adding
<stikonas>after automake 1.10 (and will be before xz) but xz is not moved yet
<stikonas>bash 2 is not enough to build xz
<OriansJ>fossy: yeah, I've really been slacking on that M2libc work
<mihi>stikonas, xzdec-min should probably be buildable using kaem :) https://github.com/schierlm/xzdec-min/
<stikonas>possibly, but we don't really need it until we start building newer software from 2010 or so...
<mihi>however, I never tried if tcc+musl can build it, but would not know why not
<stikonas>also there is xz-embedded
<stikonas>hmm, it might be stuck at the same place where normal xz is stuck
<stikonas>I need to either patch xz to remove static keyword from arrays, or find tcc patch that adds support for them
<OriansJ>is get_machine used in live-bootstrap? (asking if I need to put it in the graph)
<stikonas>(tcc from mob branch does support them)
<stikonas>OriansJ: no, I don't think so
<stikonas>at least if your initial .kaem files don't use it
<stikonas>OriansJ: but we do build it
<stikonas>(it's in mescc-tools-seed subdirectory)
<stikonas>still an old name before renaming to stage0-posix
<OriansJ>stikonas: I guess architecture specific code branches haven't shown up in live-bootstrap yet
<stikonas>no, we don't have anything yet
<mihi>how can you use get_machine from kaem without pipes or output redirection anyway?
<stikonas>i386 is hardcoded...
<stikonas>mihi: I guess you can't... That's why we don't use it...
<stikonas>but build scripts probably need some templating...
<stikonas>if we want to have other arches
<OriansJ>mihi: it is easy to have get_machine to write to a file or create environmental variables
<OriansJ>but it is ultimately a question of if we need it and if so any tweaks needed to get things working.
<OriansJ>everything I write is subject to change and improvement in the hopes of enabling others to bootstrap what they need.
<mihi>I think creating envars could be tricky, but returning an appropriate exit code or outputing a template file that can be processed by e.g. M1 should be easy.
<OriansJ>mihi: we already have envars in kaem
<OriansJ>also mescc-tools-extras can have any sort of functionality you can write in C
<OriansJ>be it sed, awk, etc
<mihi>I know, but AFAIK envars are only passed from parent to child and not vice versa, so usually some hacks like backticks are used to get envars the other way.
<OriansJ>we could have a flag in get_machine like: --write-env and it'll create a file for kaem to read
<OriansJ>we have options to explore when we get closer to having to solve that problem
<OriansJ>but until then it is spitball idea generation
<OriansJ>then the person who does the work gets to decide what work they prefer to solve that problem.
<OriansJ>as it has always been.
<stikonas>fossy: hmm, tcc-build bash 4.2.53 also creashes on musl's configure script...
<stikonas>but so far that's the only thing where i observed crash
<stikonas>e.g. gcc build does not trigger it
<bauen1>stikonas: tell me more about that crash :D
<bauen1>just because i haven't ruled out silent memory corruption or compiler bugs for my problem yet (well compiler bug seems unlikely given how it works for everyone else)
<stikonas>well, running it with bash -x prints
<stikonas>+ test -z ''
<stikonas>Segmentation fault (core dumped)
<stikonas>backtrace: https://paste.debian.net/1191259/
<stikonas>well, we can try to use new bash with tcc and hope that we won't hit this crash (and for building new musl, we can just rebuild bash with gcc before that)
<bauen1>oh no, running musls configure from a manually constructed userns + chroot it exits normally
<bauen1>so much for easily reproducing this issue lol
<stikonas>bauen1: that depends on bash version that you use
<stikonas>old bash 2 works fine
<stikonas>it's the new ones that crash
<stikonas>but it's not a blocking problem
<bauen1>stikonas: i'm not even past musl 1.1.14 lmao
<stikonas>I am where musl 1.2.2 is built
<bauen1>for some reason musls configure script exists, supposedly with exit code 512 :D
<bauen1>or maybe not let me re run this (see you in 50m)
<fossy>stikonas: hm.
<fossy>stikonas: can we disable test builtin
<fossy>cause test is also in coreutils
<stikonas>maybe we can, but for now I think I'll just reporder musl and gcc-bash
<stikonas>that will solve musl configure problem
<stikonas>I'm not even sure if it's test that's broken
<stikonas>it might be the next statement
<stikonas>well, I need newer bash for xz...
<stikonas>xz was not happy with bash2