IRC channel logs

2019-12-10.log

back to list of logs

<oriansj>dddddd: no starting on RISC-V until aarch64 is done and in an official release
<fosslinux>oriansj: aarch64 official release of wat
<oriansj>fosslinux: M2-Planet
<oriansj>further enhancements for mes-m2 (slow_lisp branch) are up with support for primitive-load (that matches guile)
<oriansj>and (number->string 42 22) now works in a way identical to guile
<oriansj>janneke: once modules are in, we should be able to have matching code with guile and we will be able pull out most of the mes unique scheme. Hopefully you are ready
<oriansj>greetings nimaje
<efraim>i'm not at all about to start working on it now, but i noticed that mes assumes little endian, which isn't the case with my 32-bit powerpc
<oriansj>efraim: that is because Mes doesn't support powerPC yet. But mescc-tools actually assumes big endian and uses flags to dictate which way you want the bytes
<oriansj>M2-Planet + mescc-tools support knight which is big endian and thus are endian neutral by design
<dddddd>oriansj, yes captain!
<fosslinux>hehe
<janneke>oriansj: i'm not really ready for that -- see my wip-modules branch if you want; but i'm happy to see things happening that i'm not ready for :-)
<xwvvvvwx>hello bootstrappable :)
<xwvvvvwx>janneke: any plans for a mes 0.22 release? I would open a PR to nixpkgs with the double compiled mes once I have an official release with the build system fixes from last week in :) :)
<janneke>hello xwvvvvwx!
<janneke>xwvvvvwx: oh great! yes, i am planning a 0.22 release around the theme rb5, 9*fb, build fixes
<janneke>i am working on getting a fresh guix wip-bootstrap branch in place, based on that and expect some bugfixes and definately a documentation update
<janneke>xwvvvvwx: have you seen the press release draft by vagrantc?
<xwvvvvwx>yes
<xwvvvvwx>looks great :) :)
<janneke>yes, we need the nix package in there too!
*janneke is thrilled by all this
<xwvvvvwx>I would need a 0.22 release before I can submit the ddc nix package I think
<xwvvvvwx>I've started digging into the mes based nix bootstrap as well
<xwvvvvwx>Am I wrong in thinking that I should be able to bootstrap NixOS using busybox and mes only?
<janneke>okay, we may want to wait with the press release for that, or not; possibly depending on jelle/arch and myself getting 0.22 out
<janneke>having nix in the works might be OK too
<janneke>xwvvvvwx: yes, that could work
<janneke>on the imminent wip-bootstrap/scheme-only bootstrap in guix, guile+gash+gash-core-utils provide what busybox provides -- sort of
<xwvvvvwx>yes that would be even better
<xwvvvvwx>but busybox + mes is only a few mb
<xwvvvvwx>so it would already be quite nice :)
<janneke>oh wait, hmm maybe it's wise to attempt the current bootstrap first; that's similar and simpler
<janneke>hmm, i think we should just start and see; hard to see the limitations from here; after we start walking it will become evident quite soon
*janneke is thinking out loud
<xwvvvvwx>yes :)
<janneke>but i'm already feeling happy: this will not "just" be a port or annoyingly much typing, we will really do something new
<xwvvvvwx>even if it is just a port I would still learn a lot I think
<janneke>yes, i feel that way too
<oriansj>xwvvvvwx: well the scheme-only bootstrap will be only a 300KB binary and once mes-m2 is done, we will have a full path from a 357byte bootstrap binary
<xwvvvvwx>that will be *very* nice :)
*janneke is bisecting a sob (scheme only bootstrap) cleanup bug
<janneke>when do i learn not to change 4 things at once?
<janneke>"this change should be harmless" :)
<vagrantc>and this change, and this change, and this other change? :)
<janneke>hmm, it may be nice to look at output checksums of the bootstrap
<janneke>vagrantc: exactly
<janneke>also, i was just thinking of you and remembered to smile to myself instead of shaking my head
<vagrantc>hah
<janneke>it worked
*vagrantc checks out the cutting edge of "wip"
<janneke>also, nice to read that i'm not alone, you've been there too :)
<vagrantc>--with-bootstrap looks like a very nice gift :)
<janneke>that's the idea; have your efforts/pain pushed upstream so that others may benefit
<vagrantc>janneke: so does --with-bootstrap produce another bin/mes, or just overwrite the original one?
<janneke>vagrantc: a bit of both; it produces bin/mes-gcc bin/mes; then bin/mes-mescc and probably bin/mes
<vagrantc>i guess i'll just install all of them? :)
<janneke>yes! \o/
<janneke>vagrantc: just chatting to xwvvvvwx i'm planning a .22 release aka rb5 / 9*fb; i'll be updating some documentation (esp. on how to build)
<janneke>also, wip-bootstrap needs to work with it again, so i need a bit of time
<vagrantc>i'll be following right along :)
<janneke>i would really like the arch build to work, and possibly clear the road for freebsd a bit; though that is optional
<jelle>I should look at providing an env for you maybe
<janneke>jelle: you're here; great!
<jelle>I am lurking yes =)
<janneke>that would be nice, a downloadable vm + recipe would prolly do for me
<vagrantc>hmmm... current wip patchset and configure --with-bootstrap fails for me with: Missing dependencies: limits.h
<vagrantc>might just be due to the extra arguments being passed via CFLAGS and co
***ng0_ is now known as ng0
<vagrantc>yup, just forced cflags to be less optimized ...
<vagrantc>this also should pull in -fdebug-prefix-map now ... so should fix the reproducibility issues
<vagrantc>or at least, some of them
<vagrantc>janneke: i think i got a build going, but it has abundant test failures ... not sure if it's running the tests on the bootstrap mes or the mes-gcc or what
<vagrantc>the i386 build was fine ... but x86_64/amd64 build seems to have regressed
<janneke>vagrantc: ah, that's at least a stroke of luck
<janneke>i haven't looked at 64bits, will do!
<vagrantc>is --with-bootstrap default now?
<janneke>no, in guix we just build two packages
<vagrantc>without passing --with-bootstrap, it produced a mes-gcc and mes-mescc binary that i hadn't seen before
<janneke>hmm, that's a bug afaic; i'll have a look, thanks!
<janneke>prolly a "false" vs unset/"" thing somewhere
<vagrantc>is mes-mescc the bootstrap build?
<janneke>yes, it builds "mes"-<compiler>
<janneke>...but you're not finding the magic number?
<vagrantc>well, i did a build where it was stripped, doing another build now and seeing
<janneke>phew!
<vagrantc>was trying to simpify the packaging, but overdid it :)
<vagrantc>grep ^FAIL: doesn't work because of colorized control characters
<janneke>oh, pretty
<vagrantc> https://paste.debian.net/1120529/ if you're curious what failed on amd64
<janneke>thanks...odd set of fails at first sight
<vagrantc>i might be mangling the CFLAGS too much, too
<janneke>vagrantc: ("@bootstrap@" . ,(if with-courage? "true" "false")) eh...
<janneke>if only all bugs were like this
*janneke smiles
<vagrantc>:)
<janneke>vagrantc: can reproduce 64bit breakage: FAIL: lib/tests/scaffold/06-call-2.c
<janneke>vagrantc: here's what's going wrong: https://paste.debian.net/1120541/
<janneke>not sure what commit or code triggered that change; hmm
<vagrantc>pretty small set of commits ... maybe git bisect ...
<janneke>yes, i'll do that
*vagrantc keeps firing off sequential test builds
<vagrantc>a few nodoc + nocheck builds on i386 while i make sure my updated packaging still works
<jelle>vagrantc: do you have a 64 bit hash? =)
<vagrantc>jelle: pretty sure that magic number began with 8
<jelle>lol
<vagrantc>spoilers: 88dcafc8a3e2f1714e4463a78afb5c91c844df130508f69f01bb8b755eaf3803 usr/bin/mes-boot0-static
<jelle>thank you!
<jelle>I might try to go down that root
<jelle>*route
<vagrantc>not sure if anyone else reproduced that, but my builds and debian builds consistantly came up the same
<vagrantc>though, you won't want all of the patches from wip ... newer ones seem to have some issues
<vagrantc>jelle: my builds were effectively a4dbbd3e60e75f85b7efc1b1ea1b491f8358a94b Fix build without git ... with a7cb636437f0e6fc181eb9d7cd45d325ecd75026 CFLAGS ... reverted
<vagrantc>jelle: i think you might need to keep the CFLAGS patch and just explicitly set CFLAGS
<vagrantc>since you had stack-protector flags or something?
<vagrantc>janneke: still got the magic #9 with my last i386 build :)
*vagrantc is curious if there are remapped debug paths or not
<vagrantc>hmmm... i may have overwritten CFLAGS -g, though ...
<janneke>\o/
<vagrantc>the original flags were just -static ... and then one of the patches added debug -g ?
<janneke>yes, thought that was neat
<janneke>configure uses CFLAGS='-g' as a default, and CFLAGS should now be honored
<janneke>wdyt?
<vagrantc>my last build passed -fdebug-prefix-map=$(CURDIR)=. and now i'll try with -fdebug-prefix-map=$(CURDIR)=. -g
<vagrantc>that should strip the build path from the debug symbols ... managed to get identical hash with it on the bootstrap binary which presumably doesn't have debug symbols and ignores CFLAGS?
<vagrantc>or ignores unknown values
*vagrantc checks the history of -fdebug-prefix-map as to what versions support it
<janneke>vagrantc: indeed, ignores CFLAGS and does not use -g
<vagrantc> https://reproducible-builds.org/docs/build-path/ says -fdebug-prefix-map is available in "all gcc versions" but i think that's probably not really true ... but it goes back
<janneke>hehe, i got a gcc-1.40 here
<janneke>well, 1.42 does not have it
<janneke>there!
<janneke>neither does 2.95.3
<vagrantc>you're probably in a better position to bisect what does and doesn't have it than most people :)
<vagrantc>clang 3.6 fwiw
<vagrantc>oops, clang 3.8
<janneke>how many people have most minor gcc tarballs checked-out besides their gcc source git?
<vagrantc>jelle: i got the magic #8 on amd64/x86_64 with more stuff applied
<jelle>all I know is that I should use http://lilypond.org/janneke/mes/mes-0.21-11-gd8f361705.tar.gz
<vagrantc>janneke: the test suite runs against ... the mes-gcc build ... or the mes-mescc build ?
<jelle>which I did no try yet btw
<vagrantc>because i'm getting the same 64-bit binary that passed the test suite in older builds
<janneke>vagrantc: the mes mescc-built (the latest bin/mes)
<janneke>vagrantc: yeah, it must be somewhere else
<janneke>git bisect is patient...
<janneke>:)
<vagrantc>janneke: my older builds tested against the gcc built mes, i'm pretty sure
<janneke>ah, right!
<janneke>of course!
<vagrantc>since i copied the source into a subdir and didn't explicitly try to run tests on it
<janneke>that explains it
<vagrantc>so this may not be a regression, per se ... just revealing bugs
<janneke>yeah, probably
<vagrantc>and you had said 64-bit couldn't compile tcc, so presumably there are bugs
<janneke>i was wondering how come the 64bit seemed pretty ok; i thought to remember it was far worse...
<janneke>but that means that the tests suite must cater for this ... hmm
<janneke>all of a sudden it matters whether you do --with-bootstrap or not
<vagrantc>heh :)
<janneke>that's not no nice
<vagrantc>well, it seems like the test suite is doing the right thing ...
<janneke>maybe your're right
<vagrantc>hmmm... or maybe the test suite should be run on both builds?
<janneke>64bit does not support --with-bootstrap, that's what the test suite says
<vagrantc>and the build i did on 64-bit is ... sketchy
<janneke>you can probably say: make check MES=$PWD/bin/mes-gcc
<vagrantc>heh
*janneke bisected their scheme-only bootstrap bug
<vagrantc>to paper over the problem ... well ... maybe should've just kept mes in experimental ... or disabled 64-bit builds
<janneke>now to fix it
<vagrantc>actually, i should probably only do --with-bootstrap on i386 for now, and not build or ship on 64-bit
<janneke>can't you just not do `--with-bootstrap' on 64bit?
<janneke>yeah
<janneke>that is, once upstream fixes their --with-bootstrap option
<vagrantc>yeah, since we know the resulting binary, while reproducible, is reproducibly failing test suites :)
<vagrantc>oh yeah :)
<janneke>after configure: sed -i s/bootstrap:=/bootstrap:=false/ .config.make
*janneke pushed a rewrite soon-to-be-rewritten to their gitlab
<vagrantc>i had assumed you weren't using gitlab anymore
<janneke>i use it for any wip-* stuff that is still work-in-progress
*janneke doesn't do backups
<janneke>err, except for on the interwebs
<janneke>sometimes my gitlab is real stale, mes-wise...it depends
<vagrantc>wip-really-no-really-wip-wip
<vagrantc>:)
<janneke>unstable wip
<vagrantc>well, with the latest regular wip patches i've got i386 building pretty nicely :)
<janneke>hehe
<jelle>ok, make check is running for my 64 bit build
*janneke goes for some dinner
<jelle>vagrantc: for me ./bootstrap.sh segfaults
<jelle> http://dpaste.com/215G5H2
<jelle>ok, that had todo with GUILE_LOAD_PATH
<jelle>now it does.. something!
<jelle>%sha256sum src/mes
<jelle>bc5ac0c4f9a41b2a0ed78e527b42fa91bd55aa79cbaea7b2f007d8a507691d2a src/mes
<janneke>jelle: it seems configure.sh has still picked-up gcc as compiler, can you check?
<janneke>mescc gets a GNU C assembly extension to parse, which it cant
*janneke downloads the paste
<janneke>oh wait...
<jelle>janneke: sorry the paste is already outdated, compiling worked
<jelle>but it's 64 bit =)
<janneke>ah, good!
<janneke>yeah, get anything to work first is a good idea
<janneke>jelle: are you sure you didn't strip the binary?
<jelle>janneke: ./src/mes: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, not stripped
<janneke>debian did that automagically, that changes the hash
<janneke>ok
<jelle>janneke: I've compiled it from the tarball
<janneke>well, diffoscope to the rescue, then
<jelle>:D
<vagrantc>jelle: gcc or mescc build?
<janneke>comparing the .s (or if you must the .o) from mescc-lib/ is actually pretty nice
<jelle>vagrantc: mescc build
<janneke>thanks to oriansj's fantastic ascii intermediate stages
<jelle>and my packaged mes is not stripped (build as a package)
<vagrantc>janneke: testing your cater-for-bootstrap-build patch
<vagrantc>and also testing only running mescc build on i386.
<janneke>very nice
<jelle>janneke: maybe I am using a different source then vagrantc?
<vagrantc>jelle: stack protector arguments/built-ins ?
<janneke>jelle: xwvvvvwx used the tarball too: 0.21-11-gd8f361705.tar.gz
<jelle>janneke: aha!
<janneke>it includes the two patches that impact the hash: MES_PKGDATADIR and core: Throw instead of segfault on non-existing input...
<vagrantc>but am i the only one who tested x86_64?
<vagrantc>enough to have a hash to compare ...
<vagrantc>of course, now i'm building one without the hash at all :)
<vagrantc>or rather, not building the thing to compare
<janneke>vagrantc: i found the 88*03 too -- but to --with-bootstrap or not; x86_64 is nice for comparison but other than that useless for bootstrap, atm
<vagrantc>right ... and since it fails so many tests... i figured to not bother shipping it in debian
<janneke>good
<vagrantc>at least, that's what my current test is ...
<vagrantc>janneke: worked like a charm!
<janneke>vagrantc: \o/
<vagrantc>should probably wait till next release before uploading, but looking good enough to me :)
<janneke>that's great, my bootstrap starts to look okay too, xz already built
<janneke>now to update the docs and this arch multi-persona gcc thingy