IRC channel logs

2021-02-27.log

back to list of logs

<stikonas>pder: so I was playing a bit with binutils, have some updates...
<stikonas>I was able to regenerate 9 out of 10 configure scripts
<stikonas>and for some reason intl/configure fails
<stikonas>no idea why...
<stikonas>especially considering that we run with --disable-intl
<stikonas>I get /after/include/musl/bits/alltypes.h:148: error: identifier expected
<stikonas>(I used autoconf-2.13
<stikonas>so that leaves intl/configure and quite a few Makefile.am files that we still need to deal with)
<fossy>stikonas: prepare
<fossy>stikonas: can you review https://github.com/fosslinux/live-bootstrap/pull/51
<stikonas>a couple of minor points...
<stikonas>fossy: maybe you can review https://github.com/fosslinux/live-bootstrap/pull/53
<stikonas>this should be quite uncontroversial and wouldn't conflict with anything
<fossy>oh yeah, meant to merge that
<stikonas>ok, I think I'll fix autoconf tomorrow after you merge your readme changes..
<stikonas>will then rebase on top...
<stikonas>I might try to also try to install it in some versioned folder, so that we can hav multiple autoconfs
<fossy>yeah, good idea
<fossy>or, install the binary/script as autoconf252
<stikonas>I think by default autoconf's install does autoconf-2.52
<stikonas>well, I'll see tomorrow
<stikonas>there are also data files...
<stikonas>not just binary
<OriansJ`>since you were having trouble with sha256sum, I made a couple changes to make your life a little easier: https://paste.debian.net/1187150/
<OriansJ`>so that it matches standard sha256sum file1 file2 ... fileN behavior and standard sha256sum -c file1 file2 .. fileN behavior as well.
<OriansJ`>The only parts I skipped were exit codes and the counting of number of correct/incorrect
<OriansJ`>with some tweaking to M2-Planet+M2libc and the source code, it could possibly be built by M2-Planet+M2libc and be made available earlier in the bootstrap chain.
<marusich>Hello everyone. I wanted to mention that I have made an update to https://issues.guix.gnu.org/41669 that you might be interested in.
<marusich>In short, the bug is that cross-compiling the bootstrap binaries for powerpc64-linux-gnu (and also for powerpc64le-linux-gnu, and perhaps for other architectures, too) from x86_64-linux Guix, produces a different GCC 5.5.0 when everything is built entirely from source (i.e., without substitutes) on different machines.
<marusich>The update of interest to bootstrappable is that I think I have narrowed down the likely causes to about 21 derivations, and many of them appear to come from the "reduced bootstrap" packages, which use mes and so forth.
<marusich>For details, please check out https://issues.guix.gnu.org/41669#44
<marusich>Here is a pretty little graph of inputs showing derivations that produce a differing output when built entirely from source on different machines, starting from the derivation that cross-compiles gcc-static. To entice you: https://issues.guix.gnu.org/issue/41669/attachment/44/0/1
<marusich>I am curious to know if anyone knows of known reproducibililty issues or has any ideas, after reading my latest update there. I'm wondering if the guile 2 vs guile 3 difference is significant.
<marusich>One side effect result of my testing is that it shows clearly that some derivations like those used to build mesboot, gash-utils, etc., are definitely not reproducible, even when built on the same system.
***mephista is now known as spicy_icecream
<OriansJ`>marusich: my first thought was the guix hashing producing the difference; however guile2 vs guile3 is a big difference. And gash-utils+bootar have not been tested for reproducibility yet.
<dftxbs3e>marusich, hello!
<OriansJ`>but I know for certain that reproducible builds spent a great deal of effort on GCC reproducibility and they probably didn't backport those fixes to earlier GCC versions.
<stikonas>also early binutils...
<g_bor[m]>hello guix
<g_bor[m]>sneek: botsnack!
<g_bor[m]>oops wrong channel sorry
<OriansJ`>lol
<roptat>hi! any idea about the build time for the rust bootstrap chain (from mrustc) in Guix? and for gcc (from mes)?
<stikonas>roptat: rust depends a lot on machine
<stikonas>could be 3h or so per rust version
<stikonas>on the other hand gcc is fairly quick
<stikonas>well, maybe depends on which gcc versoin
<stikonas>so rust-1.19 to rust 1.50 or so would take a few days on slower machine
<roptat>thanks :)
<roptat>let's say gcc 7, since that's the default one in Guix
<stikonas>well, for gcc bootstrap slowest steps are scheme...
<roptat>which scheme do you build? I thought it was all from mes?
<stikonas>my guess would be maybe 2h
<stikonas>roptat: guile too
<stikonas>boottar, gash etc can't run on mes
<stikonas>they depend on guile
<roptat>ah, got it
<roptat>so, in total, from mes to gcc 7?
<stikonas>well, mes and tcc takes about 20 minutes
<stikonas>well, maybe 3h
<stikonas>again depends on machine
<roptat>yeah, sounds like a very good machine ^^
<stikonas>well, I was just guestimating
<roptat>I remember build times going on for ~5-10 hours for a single version of gcc when I built LFS systems
<stikonas>can't really remember exactly the last time I did it...
<stikonas>well, probably long time ago
<roptat>yeah, that was like 10 years ago ^^
<stikonas>well, live-bootstrap currently takes maybe 25 minutes, but that's obviously not finished, we haven't reached gcc there
<roptat>I managed to build in one week-end by building everything up to gcc on Saturday, building gcc during the night, and finishing the system on Sunday ^^
<roptat>maybe there's a way I could test on my machine, with the current guix bootstrap?
<stikonas>well, if you do guix gc and clean everything and do git pull it should rebuild everything from mes
<roptat>a bit wasteful, but I guess it would work if I ask to build gcc@7 with --no-substitutes?
<stikonas>shoudn't it?
<stikonas>a few people here build with no-substitutes
<stikonas>I rebuild janneke's wip-m2 branch from hex0 to stuff like latest rust
<roptat>wow, amazing!
<roptat>what's missing from the branch for merging in Guix? (apart from having ci build it)
<stikonas>well, with a little bit of uusual cheating that guix does
<stikonas>not sure what's missing...
<roptat>what kind of cheating? :)
<stikonas>roptat: like I said, guix reuses guile to run some of the things, then it uses pregenerated bison files, pregenerated configure scripts
<stikonas>that's why fossy, pder and I are working on live-bootstrap project which does not do these things
<stikonas>so we basically dealt with the first two problems, and halfway through the last one
<stikonas>roptat: not sure if you saw my reply, you got disconnected
<roptat>I'll read the log, thanks
<roptat>oh I see, there's guile from guix which gets injected in the build environment, to drive the build
<roptat>and some pregenerated files that are not removed
<stikonas>indeed
<roptat>actually, I got a nice idea to get an estimate of the build time: look at ci's build times :)
<stikonas>you can try https://github.com/fosslinux/live-bootstrap/ although, it's still very much WIP...
<stikonas>oh yeah, you can check that too
<roptat>there seems to be a lot of variation depending on the versions
<stikonas>roptat: so at this stage live-bootstrap diverged a bit from guix bootstrap, to bootstrap bison we had to switch to musl C library
<stikonas>then to bootstrap autotools, we had to do some partial perl bootstrapping
<stikonas>but I hope that we might be able to skip old GCC and jump to newer one
<roptat>sounds fun :D
<stikonas>i.e. instead of 2.95.3 straight to 4.x
***pgreco_ is now known as pgreco
<roptat>oh, ci doesn't show correct information for rust < 1.46
<roptat>but estimating from numbers I found, it should be between 1 and 3 days
***cbaines_ is now known as cbaines
<stikonas>fossy (cc pder), so I'm mostly done with my autotools changes, but I've rebased them on top of unmerged part numbering change PR, so I guess that has to go first
<stikonas> https://github.com/fosslinux/live-bootstrap/pull/51
<stikonas>it's much nicer now with automatic numbering
<fossy>Oh good it has a tick now
<fossy>Before it didnt and I was so confuse
<fossy>So I just let it rerun
<stikonas>fossy: yeah, but I had some problems with rst2html on your file
<stikonas>although github does render it
<stikonas>fossy: rst2html shows error like this https://git.stikonas.eu/andrius/live-bootstrap/src/branch/autoconf
<fossy>I bet I could just remove that line
<stikonas>yeah, let me try
<stikonas>fossy: yes, that helps
<stikonas>other than that I guess good to go...
<stikonas>much nicer to work with automatic numbering now...
<fossy>Ok ill remove that and merge
<fossy>and then autoconf
<stikonas>yeah...
<stikonas>I need to rebase it once you merge
<stikonas>to remove first commit
<stikonas>and after autoconf I guess pder can do bash...
<fossy>there we go, done
<fossy>plz rebase :)
<stikonas>yeah, rebased locally already
<stikonas>but maybe will wait for previous CI to pass
<fossy>:-)
<fossy>hm, dosen't really matter
<stikonas>oh, it just did
<fossy>a push cancels the previous one
<fossy>but stops waste of time i guess
<stikonas>well, it was 99% done
<fossy>omg, our first autotools created program!
<stikonas>ok, pushed, please review
<stikonas>well, is own
<stikonas>its own...
<stikonas>used autotools to rebuilt autotools
<fossy>yes
<stikonas>and different versions should be coinstallable
<fossy>do we need to rm anything before https://github.com/fosslinux/live-bootstrap/pull/52/files#diff-01a2eddc9347be392bef6a876e8378389dfb087dea7b6db8623580a40e77faa8R6
<fossy>?
<stikonas>I remove in src_install
<stikonas>oh, I see what you mean
<stikonas>hmm
<stikonas>not sure
<stikonas>we can remove
<stikonas>but running these overwrites those files
<stikonas>so not sure how important it is
<stikonas>but if you want, I can add...
<stikonas>good that automake can be basically built with just autoconf
<stikonas>just had to install manually without makefile
<stikonas>but no sed or m4 magic...
<stikonas>to replace placeholder variables
<stikonas>fossy: hmm, as long as autoconf and automake pass, I guess pre-removing configure script does not gain us anything...
<stikonas>and I think we run with set -e
<stikonas>so if they fail, everything will abort
<fossy>stikonas: i would prefer the rm for clarity; it makes it extremely clear which files are being regenerated
<stikonas>yeah, ok. I can add that
<stikonas>would globs be fine?
<fossy>yep
<stikonas>e.g. rm Makefile.in */Makefile.in
<stikonas>ok
<stikonas>ok, pushed
<stikonas>can't rebuild coreutils though
<stikonas>"./aclocal.m4:46: error: Autoconf version 2.54 or higher is required for this script"
<stikonas>well, later...
<stikonas>oh, even patch needs newer autoconf
<stikonas>hmm, a lot of our old builds need 2.54
<stikonas>but bash can be rebuilt with 2.52, so I thought it's worth doing that anyway
<stikonas>since we'll finally get interactive shell
<fossy>yeah, sounds good
<stikonas>fossy: oh, I think we also need autoconf 2.12.1
<stikonas>do you want me to add it or merge current PR first?
<stikonas>only 2.12 is on ftp.gnu.org...
<stikonas>hopefully it will work
<stikonas>good, 2.12 works on binutils
<stikonas>!
<stikonas>so only need to regenerate automake stuff
<stikonas>fossy: I've pushed one more commit
<stikonas>to autoconf PR.
<stikonas>But it's more of the same (autoconf 2.12 now )
<stikonas>pder: I'm building on top of you binutils work now https://github.com/stikonas/live-bootstrap/tree/binutils-wip
<stikonas>(not ready yet, but getting closer)
<stikonas>fossy: pder so I think I can build binutils I might need to do minimal build/install
<stikonas>i.e. enter binutils or ld dirs and run make there
<stikonas>otherwise I get some problems with documentation build (pod2man is missing)
<stikonas>hmm, maybe I should tweak makefiles do kill po and doc dirs...
<stikonas>pder: maybe let's rebuild bash after binutils then?
<stikonas>we'll need fewer hacks
<stikonas>for size and RANLIB