IRC channel logs

<stikonas>possibly need to adjust compile flags

<stikonas>OriansJ`: I think this fails https://github.com/fosslinux/live-bootstrap/blob/master/sysa/tcc-0.9.26/tcc-0.9.26.kaem#L48

<stikonas>so we probably still need to build libc...

<stikonas>just maybe can skip mes rebuild...

<stikonas>so in the end I think it's rougly the same time 1) rebuilding mes and building tcc a bit faster or 2) building tcc slower

<OriansJ`>stikonas: mescc probably implicitly links against the libc but now we know something we didn't earlier.

<bauen1>spolier alert: mixing up autoconf, automake, 1.6.3, 2.52 randomly does not usually yield the desired result

<bauen1>but now i know that it takes me roughly 50m to get to the stage where stow could be used to easily skip build steps

<stikonas>OriansJ`: ok, I just built libmescc and directly tcc and that works

<stikonas>although, not sure if it's worth chaning build process now...

<stikonas>given that it takes roughly the same amount of time

<stikonas>bauen1: you can use pder/overlay to speed up initial builds (it's prebuilt until tcc)

<OriansJ`>stikonas: completely fair

<pabs3>debian-java (failed) attempt at bootstrapping Kotlin from scratch: https://lists.debian.org/msgid-search/ef374c2a-f5e1-cfd3-c09d-943cca56c47c@apache.org

<pabs3>(start using an old version of kotlin that didn't build using itself)

<OriansJ`>pabs3: looks like a case where guix can help as we should have older javas available.

<fossy><stikonas> fossy: strange, is this new perl or current perl?

<fossy>further investigation confirms something I added is clobbering it

<fossy><stikonas> although, I'm not sure what fossy's plans are towards x86_64 right now

<fossy>i386 first, then x86_64

<fossy><stikonas> fossy: I'm now looking at tar (and I guess same is true for some other new GNU software). When gnulib is used, Makefile.am is basically generated by gnulib-tool https://git.savannah.gnu.org/cgit/gnulib.git/tree/gnulib-tool

<fossy>please open an issue if you havenr already

<pabs3>anyone know how to contact rain1?

<pabs3>does one need to be subscribed to the mailing list to post?

<siraben>pabs3: you can just send an email to the mailing list regardless

<pabs3>hmm, ok. my mail doesn't appear to have reach the archives

<fossy>i think it's a daily dump to the archives

<fossy>don't quote me on that

<gforce_de1977>fossy: http://intercity-vpn.de/bootstrap-dcec416d0fe1d8d6effdfa985c9bd51995490381.txt

<gforce_de1977>stikonas: what can we do about changing bash-checksums? would it be a workaround for now to make the sha256-check nonfatal aka: check "$file" || true ?

<stikonas>fossy: https://github.com/fosslinux/live-bootstrap/issues/88

<stikonas>as for checksums, I'm not sure ignoring them is helpful though...

<stikonas>ingoring checksum failure, then there isn't much point in running it

<gforce_de1977>stikonas: it is really a showblocker for me. i think it's better to just print the error and check in CI the log for 'sha256sum: WARNING:'

<gforce_de1977>(after full run)

<stikonas>fossy: what do you think? ^^

<gforce_de1977>(i will run a full testrun regarding checksum ignore and report)

<stikonas>gforce_de1977: by the way, your last log has now the same size

<stikonas>as other normal runs

<stikonas>2621878

<stikonas>maybe worth diffing this binary...

<stikonas>not sure what has changed...

<stikonas>last time I ran diff, they had different sizes and were completely different binaries

<gforce_de1977>d

<gforce_de1977>sorry, this was a "CI" runs and the image is vanished already

<gforce_de1977>stikonas: in the bash-build-script you are doing "tricks" with sleep...is command "sleep" already built?

<stikonas>gforce_de1977: yes, of course

<stikonas>otherwise it would just error out with command not found

<stikonas>sleep is built really early

<stikonas>well, I guess it's a very simple program anyway

<stikonas>it doesn't do much

<stikonas>by the way, I didn't add sleep, upstream bash build process uses sleep, I just added sync after it... Although, probably anything would work

<gforce_de1977>ok, sync is IMHO a "builtin" but 'sleep' should be missing if not build via e.g. coreutils?

<gforce_de1977>here is another run with the SAME checkout, i just implemented the above idea with: checksum || true ---- but now everything is fine: http://intercity-vpn.de/live-bootstrap-329afda94d6b5f412e43f30d6be7a210ffa1e490-ignorebadsha256.txt

<gforce_de1977>search for "/after/bin/bash: OK" (for convience i added always output the sha256)

<gforce_de1977>stikonas: sorry, i overread your explanation about "we build sleep early", thanks for clarification

<fossy>we have had this conversation before, i am still firmly against making checksums *ever* be allowed to be non-fatal

<fossy>if that is happening, there is a misconfiguration somewhere

<stikonas>both sync and sleep are built in coreutils

<stikonas>well, yeah, I also didn't want to make checksums non-fatal

<fossy>/ a hardware proble,m

<fossy>/ malicious issue

<fossy>/ non-reproducibility

<fossy>all of which are bad things

<stikonas>yeah, most likely third which is least bad but still a bit bad because you can't rule out first two

<fossy>yeah

<stikonas>fossy: any ideas how we should handle gnulib?

<stikonas>since it has not releases

<fossy>not yet. haven't looked into it

<fossy>from my experience on it in void though i think most people vendor it

<stikonas>yeah, everybody just vendors it

<stikonas>that wendoring process specifies what modules you wanta to import

<fossy>i see

<stikonas>and sometimes people embed command to reproduce it (although it's not clear which gnulib-tools checkout was used, but maybe that can be found out using other means)

<stikonas>but sometimes command is not even embedded

<stikonas>(especially in older releases)

<fossy>all of our packages, do they vendor gnulib?

<stikonas>all

<stikonas>although, before we use autotools, that doesn't really matter

<fossy>yeah ofc

<stikonas>well, gnulib simply has no releases...

<stikonas>so vendring is the only option

<fossy>would be a nightmare for distros otherwise

<stikonas>indeed...

<stikonas>although, it would be a bit of nightmare for us too if we want to reimport everything

<fossy>can you give an example of a package that uses gnulib? (e.g. bintuils dosen't seem to)

<fossy>probably gcc does

<stikonas>oh, they use libiberty

<stikonas>I think gnulib was based on libiberty

<fossy>oooh

<stikonas>findutils does have it

<fossy>yes

<fossy>ok i am in understanding of the issue now

<fossy>s/in//

<stikonas>m4 has gnulib too

<stikonas>although, we've built it manually

<fossy>import-gnulib.config seems to have gnulib_version in almost all cases which hash the hash of gnulib

<fossy>has the commit hash of the version used*

<stikonas>oh, ok, that's good

<stikonas>where is that file?

<fossy>for findutils in the root dir of the tarball

<fossy>alongside import-gnulib.sh

<stikonas>oh I see

<stikonas>ok, so findutils has it, not sure about others

<stikonas>let's checkj

<stikonas>tar 1.34 doesn't

<fossy>only findutils

<fossy>➜ find . -name gnulib

<fossy>./findutils-4.2.33/gnulib

<fossy>with all tarballs extracted

<stikonas>oh, lib name can be different

<fossy>oh

<stikonas>often it is in lib

<fossy>oh

<gforce_de1977>fossy: we alreay have base64, have we?

<stikonas>sometimes in gnu

<fossy>yes i believe so gforce_de1977

<stikonas>easy to check in coreutils makefile

<stikonas>or checksums file

<fossy>stikonas: also bison (sigh), automake?, m4, coreutils

<fossy>i think

<gforce_de1977>in case if an sha256 mismatch we can at least do: echo STARTMARKER && base64 /after/bin/myfile && echo ENDMARKER

<stikonas>no, we don't have base64

<fossy>oh

<stikonas>I think you have to inject busybox

<stikonas>for debugging

<gforce_de1977>this would make it more easy to get the file out of an CI run

<gforce_de1977>locally i have injected busybox, no prob here

<fossy>hm

<stikonas>well, then use busybox base64

<stikonas>that should work

<stikonas>fossy: automake is a red herring I think

<stikonas>coreutils definitely, but so far we only attempted manual builds

<stikonas>so I think right now findutils and sed? are affected?

<fossy>when i figure out a new approach to kernel work (idea in progress) then we can probably do a host nfs mount or smth

<fossy>later

<fossy>coreutils also

<fossy>coreutils-6.3/lib/gnulib.mk has "# Generated by gnulib-tool."

<stikonas>yes, but we don't use it

<stikonas>well, it's vendored source

<stikonas>but we don't use pregen'ed files from there

<fossy>oh duh

<stikonas>coreutils -6.10 now, not 6.3

<fossy>yea

<stikonas>but they oftne have that gnulib.mk

<stikonas>which is autogenerated part of Makefile.am

<stikonas>it's fairly readable though

<stikonas>and at least not as big as gcc's Makefile.in (which is about 1 MB)

<fossy>wow

<fossy>that's a lot

<stikonas>also m4/gnulib-comp.m4 is gnulib-tool'ed

<fossy>in what?

<stikonas>e.g. coreutils

<stikonas>others too I think

<fossy>oh right

<stikonas>yes, m4 has it too

<stikonas>it's just 10 lines

<fossy>that's not bad

<stikonas>have to go afk for a while...

<stikonas>some other m4's are larger

<fossy>ok talk later

<stikonas>coreutils gnulib.mk is 70K...

<fossy>not nice

<stikonas>fossy: although, I guess a lot of configure.ac files are initially generated using autoscan too

<stikonas>and then hand tweaked

<stikonas>although, configure.ac usually is a bit smaller

<gforce_de1977>stikonas: fossy: bauen1: i just tried to built coreutils 'base64' (just in case my private CI-run fails again and i want to automatically output the bash5-binary to logfile) but adding 'base64' to the list of utils abort with an error "no rule to make target src/base64.o"...is the magic needed?

<OriansJ`>xeh ix a program in stage0 that would dump any file into hex and it can easily be built by cc_* or better

<OriansJ`>it isn't as efficient as base64 but it could trivially be added early on and enable dumping the hex of any binary that doesn't have a matching checksum, which would enable CI to give you the binary (in hex form) that you could then diff locally

<stikonas>gforce_de1977: which version coreutils are you building?

<stikonas>if it's 5 then the answer is simply it's not there

<stikonas>which is what make says

<stikonas>no rule to make base64.o means it does not know how to compile base64.o file which normally means base64.c file is not there

<stikonas>(make has implicit rule of how to build .c -> .o)

<OriansJ`>That reminds me, I probably should update that and make it more C standard with M2libc

<OriansJ`>and the revised xeh.c is now up in stage0-posix

<OriansJ`>build with: M2-Planet --architecture amd64 -f sys/types.h -f stddef.h -f string.c -f amd64/Linux/unistd.h -f stdlib.c -f amd64/Linux/fcntl.h -f stdio.c -f bootstrappable.c -f ../stage0-posix/Legacy_pieces/xeh.c -o foo.M1 --debug && blood-elf -f foo.M1 --64 -o foo-footer.M1 && M1 --architecture amd64 --little-endian -f amd64/amd64_defs.M1 -f amd64/libc-full.M1 -f foo.M1 -f foo-footer.M1 -o foo.hex2 && hex2 --architecture amd64

<OriansJ`>--little-endian --base-address 0x10000 -f amd64/ELF-amd64-debug.hex2 -f foo.hex2 -o foo && ./foo filename

<OriansJ`>probably could drop a couple of those libraries but I'll deal with that when I include it in the standard bin output

<gforce_de1977>stikonas: you are right, i tried with coreutils5, so in version 6.x it is possible? i will just try...

<stikonas>well, yeah, coreutils 6.10 has base64.c

<stikonas>if it builds, just make a PR, so that everybody has base64, it is useful to have...

<gforce_de1977>ok, will try to build base64

<OriansJ`>I think I can get the M2-Planet built version of Mes.c up to speed by converting it to be built with m2libc

<stikonas>well, we still need to wait until mes-m2 can be built with m2libc

<stikonas>I think at the moment there was some incompatibility

<gforce_de1977>funny: another working bash5.1 with "invalid" checksum qemu: 6af92e0a1b5db87220ae2ae5d345bfba4801bb4011c3386ce87b975bf9c24637

<gforce_de1977> http://intercity-vpn.de/live-bootstrap-329afda94d6b5f412e43f30d6be7a210ffa1e490-ignorebadsha256.txt

<gforce_de1977>and

<gforce_de1977>live-bootstrap-329afda94d6b5f412e43f30d6be7a210ffa1e490-ignorebadsha256-II.txt

<gforce_de1977> http://intercity-vpn.de/live-bootstrap-329afda94d6b5f412e43f30d6be7a210ffa1e490-ignorebadsha256-II.txt

<OriansJ`>stikonas: that is my project for today (child willing)

<gforce_de1977>my project this weekend is: https://2021.revision-party.net/start

<gforce_de1977>stikonas: just adding 'base64' to the makefile list of coreutils-6.x does not work: tcc: error: undefined symbol 'base64_encode'

<gforce_de1977>(i have to look deeper)

<stikonas>well, find which while has that symbols and compile it too

<stikonas>you then might need to add custom linking rule if it's more than one .o file -> base64 binary

<gforce_de1977>ok, will do

<stikonas>an my projecet for today was building GMP https://github.com/fosslinux/live-bootstrap/pull/89

<stikonas>GMP is needed both by Guile and newer GCC

<OriansJ`>my project for today, get Mes.c building and MesCC in mes-m2 running so that when I upgrade stage0-posix, we don't build a worthless binary

<OriansJ`>there finally

<OriansJ`>>.< I was too hopeful

<OriansJ`>probably a missing scheme file (but copying them all over breaks guile's ablity to run MesCC)

<OriansJ`>Let us call it a half-step

<OriansJ`>oh god it isn't buildable by GCC again >.<

<vagrantc>first successful armhf build for mes on Debian: https://buildd.debian.org/status/logs.php?pkg=mes&ver=0.23-1&arch=armhf

<vagrantc>failed on armel, though: ++ /usr/bin/gcc -nostdlib -g lib/linux/arm-mes-gcc/exit-42.S -o exit-42

<vagrantc>lib/linux/arm-mes-gcc/exit-42.S: Assembler messages:

<vagrantc>lib/linux/arm-mes-gcc/exit-42.S:46: Error: selected processor does not support `wfi' in ARM mode

<vagrantc>not sure if that's worth fixing ...

<vagrantc>not sure how much easier bootstrapping armv5 is vs. armv7

<vagrantc> https://buildd.debian.org/status/package.php?p=mes&suite=experimental

<vagrantc>armel likely to be removed from debian ... any release now...

<vagrantc>but figured was worth mentioning at least :)

<OriansJ`>so this https://github.com/oriansj/mes-m2 is currently buggy but it just the isolated kaem.run and C sources needed to build mes.c by M2-Planet

<OriansJ`>so it is good enough for stage-posix to use but I'll definitely want to clean it up ALOT

<OriansJ`>like *ALOT*

<OriansJ`>mostly getting it into a state that GCC can actually build the C source code

<OriansJ`>which really shouldn't be a thing in bootstrapping but yet here we are

<OriansJ`>but first I am going to get M2libc and M2-Planet into a release ready state and update stage0-posix so that the bootstrap will be a hair faster

<bauen1>OriansJ`: live-bootstrap will be faster ? by how much ?

<OriansJ`>bauen1: well M2libc speeds up M2-Planet generated binaries by 10-40% depending how write intensive they are.

<OriansJ`>also we will be removing the useless build of (the broken mes-m2) and doing a build of (the working) mes-m2 instead

<OriansJ`>so only a couple seconds as M2-Planet builds fast

<OriansJ`>already

<OriansJ`>but a 10-40% speedup to mes-m2 (once M2libc compat is done) will be quite nice speedup

<OriansJ`>but lets be honest, this work is long overdue

<OriansJ`>and I just got some free time to work on it

<bauen1>that does sound quite nice :)

<fossy>OriansJ`: bauen1 that is nice, but wont speed up live-bootstrap all that much

<fossy>wait no

<fossy>it will

<stikonas>fossy: well, mes libc takes quite a long time

<stikonas>that might be quicker

<stikonas>but it will still take quite some time to build tcc

<stikonas>fossy: I've done GMP today https://github.com/fosslinux/live-bootstrap/pull/89

<stikonas>briefly checked that libunistring is buildable with what we have, but haven't done scripting yet

<stikonas>also tried running configure in guile, but got some errors...

<stikonas>checking for socklen_t equivalent... configure: error: Cannot find a type to use in place of socklen_t

<stikonas>maybe musl incompatibility

*bauen1 is meanwhile doing a rebuild for perl-5.6.2 to add more support for symlinks as required by stow to recognise a file as an actual symlink

<stikonas>more modules are needed?

<bauen1>no, just support for -l

<bauen1>so HAS_LSTAT

<stikonas>oh ok

<stikonas>that's probably simpler

<bauen1>in the last few weeks i've learned more about perl than i'd like ...

<stikonas>yeah... me too

<stikonas>although, building perl was not too bad

<stikonas>well, building perl modules was a bit messy

<bauen1>building perl is not the issue, what you can do with perl is well

<bauen1>i've recently learned on wikipedia that a common critisim of perl is that it's a "write once language" and that is so far the best summarization of perl code i've read

<fossy>yeah saw the gmp pr looking at that now

<fossy>stikonas: yes almost 100% a musl incompatibility

<stikonas>fossy: that comment you left is for kaem build

<stikonas>I think we don't care about destdir so early

<fossy>oh, right

<fossy>yes

<fossy>correct

<stikonas>fossy: then GMP should be fine I think... Not even gnulib is there

<stikonas>although, we'll git more packages with gnulib...

<stikonas>e.g. libunistring uses it

<fossy>two comments for GMP

<stikonas>well, autoreconf didn't work for me. Despite what you thought, it does not rebuild libtool files