IRC channel logs

2021-04-02.log

back to list of logs

<stikonas>fossy: by the way, guile only needs automake 1.12
<stikonas>I just checked
<stikonas>configure.ac:AM_INIT_AUTOMAKE([1.12 gnu no-define -Wall -Wno-override \
<stikonas>so you don't need 1.16.2
<stikonas>yes, release tarball was generated with 1.16.2 but that's fine
<stikonas>we can use 1.15.x
<stikonas>ok, so config.h is different when I run configure in subdirectory...
<stikonas>must be some configure arguments...
<fossy>stikonas: ooh, i see
<fossy>ok i'll just do that
<stikonas>I think 1.15 should be enough for almost everything
<stikonas>I have not yet seen software that needed 1.16
<stikonas>although, I didn't look too much
<stikonas>but e.g. latest tar only needs 1.15
<stikonas>well, occasionally AM_INIT is incorrect
<stikonas>but 1.15 is much newer than 1.12 anyway
<stikonas>so hopefully will work
<stikonas>oh, some progress with binutils
<stikonas>need a few more sed hacks on config.in file (we already have some)
<stikonas>fossy: but I think you'll also need to package some guile dependencies
<stikonas>namely gmp and libunistring
<stikonas>ok, binutils rebuilt...
<roptat>I looked again at scala... I now have a lexer for scala2 and 3 (it's supposed to work for both languages) that I wrote by more or less translating scala to java
<roptat>so, I tried to write a parser, but the parser in Scala uses a lot of the standard library and a lot of scala features that I can't easily translate to Java
<roptat>well, I'm not sure what I should do: either spend some time to translate the infrastructure around names, constants, numbers, the ast and the parser from scala to my project, or use the documentation to derive my own types for the ast, ignoring what scala does
<roptat>the first sounds a bit complex but maybe the result will be more correct
<roptat>the second ensures that I understand better what's going on and what each construct corresponds to, so it's maybe easier for me to produce code from it
<stikonas>fossy: https://github.com/fosslinux/live-bootstrap/pull/83
<stikonas>pder: so I solved autogen for binutils too
<stikonas>we had it in 2.14 as well
<pder>I just saw that. That's great news
<pder>I just created a PR for the perl warning fix
<stikonas>pder: any reason why you we looking at 2.18? and not e.g. latest?
<pder>I was just trying to keep gcc, glibc, and binutils from the same timeframe since there can be mismatches
<stikonas>hmm, it probaby will work with new ones...
<stikonas>especially binutils
<stikonas>probably just needs some minimum versions for certain features
<stikonas>glibc and gcc might be more tied...
<stikonas>although, we managed to build gcc C backend with musl
<stikonas>which is completely different
<pder>Yeah its possible newer binutils might work. I just had in my mind we would need to make incremental jumps with compatible versions of gcc glibc and binutils
<stikonas>well, in my opinion, let's try newest first, and if that doesn't work, we go back
<stikonas>well, with gcc we definitely need incremental jumps
<stikonas>first of all we'll have to do full build of gcc <= 4.7.4 (including C++ frontend)
<pder>I cant remember what I was looking at, but I've seen a matrix of possible gcc glibc compatible versions when building toolchains
<stikonas>for gcc https://gcc.gnu.org/install/prerequisites.html says
<stikonas>you either need gcc 4.8 or greater
<stikonas>or use --disable-stage1-checking
<stikonas>so I guess we need maybe one intermediate version
<pder> https://stackoverflow.com/questions/35873558/glibc-vs-gcc-vs-binutils-compatibility
<stikonas>hmm, it's mostly question marks
<stikonas>I guess just nobody tried
<stikonas>(of ir they tried, didn't bother updating this matrix
<pder>Something to at least keep in mind in case we run into subtle breakage
<stikonas>well, with had some subtle breakage with musl too...
<stikonas>even now with building binutils 2.14... had to apply some sed hacks
<fossy>stikonas[m]: merged
<Hagfish>"GNU Automake from 1.15 series. This is the last version that runs on Perl 5.6."
<Hagfish>it's so cool to watch all this time travel :)
<Hagfish>is Perl 5.8 needed next?
<fossy>hm, something weird is up with perl
<fossy>PRIVLIB_EXP.. dosen't exist?!
<stikonas>fossy: strange, is this new perl or current perl?
<bauen1>stikonas: (apart from being useful) is there any other motivation / idea you have in mind with https://github.com/fosslinux/live-bootstrap/pull/85 ?
<stikonas>bauen1: not really at the moment
<stikonas>that's it for now...
<stikonas>and tcc-musl-pass1 does not fully support it yet
<stikonas>maybe we should split it into two passes ...
<stikonas>bauen1: well, I thought it might eventually help you with upkg
<bauen1>yes it's very useful
<bauen1>i'm still thinking about using a chroot, at least for transitioning into /after, but after that i'll do a few test runs to test setting up a /after/upkgs directory by hand, so that will make it much easier to do
<stikonas>I sometimes find DESTDIR useful for debugging
<stikonas>although, right now we mostly use autotools which supports DESTDIR anyway
<gforce_d11977>wow: i think i found the needed KERNEL_SYMBOL, now i have: bash-5.1: checksumming installed files. sha256sum: WARNING: 1 of 1 computed checksum did NOT match
<gforce_d11977>sha256sum is: 41dd3dfa62efcd8a71392e09747aab93660b0549f39cb4e673547db47a02bc27 /after/bin/bash
<gforce_d11977>and 2621200 bytes
***mephista is now known as spicy_icecream
<janneke> https://twitter.com/orionwl/status/1378006927058296834
<janneke>OriansJ`: ^
<janneke>initial risc-v support, with a question/observation
<bauen1>in run.sh do we treat /after as the new root (in that case i would very much like to do a chroot syscall to execute it), or do we just not touch anything in / except for /after ?
<bauen1>i'm thinking of where to place the "upkgs" directory, and if to include the "after" component in the pseudo packages
<stikonas>bauen1: we don't have chroot that early...
<stikonas>when we start building in /after
<stikonas>although, there might be some other ways to make very early build nicer...
<stikonas>bauen1: but anyway, we don't need anything outside /after
<bauen1>stikonas: it'd be very easy to build chroot from coreutils in early build by implementing the chroot syscall in mes libc
<stikonas>but /after is even earlier, isn't it?
<stikonas>at around M2-PLanet
<stikonas>maybe once we have M2-libc
<stikonas>gforce_d11977: worth checking the contents of pipesize.h
<stikonas>we saw some differences there
<stikonas>but I think we've got a workaround now
<bauen1>oh right
<bauen1>i mean you could write a single purpose utility to do the chroot syscall in assembly even
<stikonas>well, that's true
<bauen1>meh, this is one of those things where no available answer is the best :/ i hate those
<gforce_d11977>stikonas: only content of '/after/bash-5.1/build/bash-5.1/builtins/pipesize.h' beside comments is: #define PIPESIZE 65536
<stikonas>yeah, that's right
<stikonas>gforce_d11977: so something else is different for you...
<stikonas>well, you can try to compare two binaries
<stikonas>and see what's different
<gforce_d11977>thats bad. i will try another run. the kernel symbol which is needed for automake seems to be 'CONFIG_FILE_LOCKING=y'
<gforce_d11977>(which adds 11k to kernel size 8-)))
<gforce_d11977>stikonas: in your case i guess it says #define PIPESIZE 512 ?
<stikonas>gforce_d11977: no, pipesize 65536 is fine
<gforce_d11977>but the binary still differs, ok understand - so there is another variation we are not ware of at the moment
<stikonas>well, anything to do with automake will be in non-bootstrap kernel
<stikonas>gforce_d11977: it might be kernel related
<stikonas>but you can try to do diff betweeen two binaries and inspect it
<stikonas>that's how we initially spotted that pipesize.h issue (which since then was fixed)
<gforce_d11977>ok, i will store the binary somehow
<stikonas>try tmux and store-buffer
<stikonas>you might need to inject busybox inside
<stikonas>to encode binary into text
<gforce_d11977>stikonas: here is my bash5.1 binary: http://intercity-vpn.de/bash5.1-sha256-41dd3dfa62efcd8a71392e09747aab93660b0549f39cb4e673547db47a02bc27.bin
<gforce_d11977>base64 and sed was my friend 8-)
<stikonas>hmm, you bash binary is also smaller
<stikonas>(last time we saw hash mismatch with bash, binaries had identical size)
<stikonas>gforce_d11977: hmm, they seem to be completely different binaries
<gforce_d11977>at least the binary works...for reference, this is the used kernel config: http://intercity-vpn.de/bash5.1-sha256-41dd3dfa62efcd8a71392e09747aab93660b0549f39cb4e673547db47a02bc27-kernel-config.txt
<stikonas>we have --build=i386-unknown-linux-gnu
<stikonas>maybe we also need to add --target and --host
<stikonas>is that 32 bit kernel?
<gforce_d11977>yes, 32bit ofcourse
<stikonas>ok, can you try to add --target and --host options to configure
<stikonas>rebuild and tell me the hash
<gforce_d11977>ok
<stikonas>I'll try here with 64 bit kernel
<stikonas>just add
<stikonas>+ --target=i386-unknown-linux-gnu \
<stikonas>+ --host=i386-unknown-linux-gnu \
<stikonas>after --build= option
<stikonas>well, without pluses, those are from diff probram
<gforce_d11977>really? i added to /after/bash-5.1/bash-5.1.sh -> --host=i386 and --target=i386-unknown-linux to the configure step
<stikonas>what is really?
<stikonas>yeah, those are right
<gforce_d11977>and the has is now: d654ccfee843598297b11eb97c6ef30822c276e6584abc3db4a149cec8d94724
<stikonas>oh, maybe build both with i386-unknown-linux-gnu
<stikonas>not some variation
<gforce_d11977>and the size is now: 2624087
<stikonas>it might affect hash
<stikonas>well, I'm rebuilding here too
<stikonas>maybe I'll push to a branch
<stikonas>to make sure we are testing the same thing
<gforce_d11977>and pipesize.h has '#define PIPESIZE 65536'
<stikonas>I'm now building this branch https://github.com/stikonas/live-bootstrap/tree/bash
<stikonas>not sure about the hashes yet...
<stikonas>they might be different
<gforce_d11977>ok...lets new, for now a throw aways my build (i have all configs to eventually reproduce) and try to limit the needed config-symbols
<gforce_d11977>can you please document, that we need the symbol: CONFIG_UID16=y for the dirglobs (please find a better name)
<gforce_d11977>and that we need the symbol: CONFIG_FILE_LOCKING=y for automake-1.8.x
<stikonas>gforce_d11977: well, maybe you can document?
<stikonas>you were doing those tests with small kernel
<stikonas>although, automake is built so late, I think requireing many symbols is not that big problem anyway
<stikonas>although, it's still good to know
<bauen1>so using DESTDIR will expose various occasions where we don't create the directories before trying to copy stuff into them (i've just hit that with automake1.6.3 stage1 / stage2)
<gforce_d11977>stikonas: my kernel is not really small, it's just a 'make tinyconfig' where still a lot of stuff is in, but you are right, a 'make defconfig' works. the point is: if we want to have our own kernel for bootstrapping (which must not be linux), than this is good to know
<gforce_d11977>also: i had to try 55 invocations of bootstrap to find this one needed symbol 8-((((
<bauen1>gforce_d11977: if it makes you feel better, soon it should be possible to skip some steps in the bootstrap easily, _soon_ (tm)
<gforce_d11977>8-)))
<gforce_d11977>it makes me feel better!
<bauen1>at least the it will be easier to skip the steps after perl 5.6.2, everything before is a bit tricky
<roptat>in the end, I went with the second solution, where I use the AST from scalameta because it's much easier to implement, while following the structure from the scala parser to create the AST. I think it's best because it's easier, and I don't loose information: scala is simply doing a lot of work in the parser that I probably will do much later
<roptat>I can parse simple literals now ^^
<roptat>working on expressions, which I need to really finish literals (because interpolated strings are considered to be literals and they can contain arbitrary expressions)
<stikonas>nice work on scala!
<stikonas>gforce_d11977: hmm, no changes here in bash checksum with --host and --target options
<stikonas>bauen1: oh, that might be true regarding DESTDIR...
<stikonas>well, I guess we'll fix them eventually
<gforce_d11977>fossy: stikonas: if i can test/help something, poke me: i will prepare a pullrequest during easter regarding the "minikernel" thing, so that is really is a 386 machine
<stikonas>well, at some point we also want to cross-compile gcc for x86_64
<stikonas>and now that we have gcc, this might be sooner rather than later...
<stikonas>although, I'm not sure what fossy's plans are towards x86_64 right now
<bauen1>stikonas: yep, as i start building packages with DESTDIR i'll iron out the bugs
<stikonas>well, automake'ed packages should be fine
<stikonas>it's the stuff before that (which automake 1.6.3 stage1/stage2 are examples of
<gforce_d11977>stikonas: ofcourse we want to cross compile all architectures and check if the sha256sums are matching with native compiling
<stikonas>well, x86_64 is more than just cross-compile
<gforce_d11977>but you are right, first step is to make x86_64 working too
<stikonas>x86_64 bootstrap is likely to be via i386 at the beginning
<stikonas>just like in guix
<stikonas>(because i386 code runs natively on those CPUs)
<gforce_d11977>a really? because the is no "native" bootstrap implemented yet?
<gforce_d11977>"there is no"
<stikonas>no, I think it would work
<stikonas>but we don't really have to do a separate chain for x86_64
<stikonas>writing full bootstrap chain is much more work than just letting 32 -bit code run on 64-bit cpu
<stikonas>especially before autotools are available, there would be a lot of manual tweaking of makefiles
<stikonas>right now all the makefiles are x86 only
<stikonas>some of them I guess would work on other arches, but some of them hardcode i386 stuff
<gforce_d11977>hmmm....is this really so much work? ok, yes - there is a reason that autotools exists 8-)))
<stikonas>well, it tooks us some time go get to autotools in live-bootstrap
<stikonas>2nd time would be quicker of course...
<stikonas>but still
***Thunderbi is now known as nckx
<bauen1>gnu stow has a `require 5.006_001;` line, does that mean it won't work with anything sooner than perl 5.6.1 ?
<stikonas>bauen1: indeed
<stikonas>bauen1: that doesn't matter too much for us
<stikonas>all perl versions are built subsequently
<stikonas>and basically replace each other
<stikonas>well, maybe the only difference is that perl 5.6.1 will not yet be in a package
<stikonas>although, some of the things that we should condider building might need newer perl anyway
<stikonas>so maybe it makes sense to try to build perl 5.8....
<stikonas>fossy: I'm now looking at tar (and I guess same is true for some other new GNU software). When gnulib is used, Makefile.am is basically generated by gnulib-tool https://git.savannah.gnu.org/cgit/gnulib.git/tree/gnulib-tool
<stikonas>basically when importing gnulib into repo...
<stikonas>I'm not sure how we want to handle those...
<bauen1>stikonas: if necessary i'll just use a bunch of `ln` calls to emulate what stow is doing until we have it, currently i've adapted the first stages of automake to use stow as a test https://github.com/bauen1/live-bootstrap/tree/add-gnu-stow
<bauen1>i'll find out if that works as intented in ~1h ...
<stikonas>ok, that should work
<bauen1>in theory stow can be used to "exchange" the running perl version by invoking it at once to unlink the old one and link the new one, but i think so far the perl executable path is hard coded to 5.6.2
<OriansJ`>janneke: it is nice to see the risc-v work being started for mescc-tools by someone other than me.
<stikonas>bauen1: well, I think if we eventually get newer perl, there is no point of keeping older perl
<stikonas>everything should just work with 5.8
<stikonas>autotools in fact require 5.6 that we have but strongly recomment 5.8
<stikonas>*recommend
<OriansJ`>but there is no way for hex2 to support non-byte aligned offsets for immediates without significant increase in complexity in hex2. to a point at which just having a standard Linker and Assembler would be a better investment of time.
<OriansJ`>Now in M3 it'll certainly be easy but minor waste of memory for implemention simplicity in bootstrapping seems like a far better trade-off.
<stikonas>indeed, memory usage by first steps is very small anyway
<stikonas>mes later needs much more
<bauen1>mes (or rather mescc) is also horribly slow
<OriansJ`>bauen1: entirely my fault for making M2-Planet non-optimizing
<stikonas>I'm not sure if mescc built mes is any faster...
<stikonas>building tcc takes 8 minutes or so
<OriansJ`>stikonas: easy to prove both ways
<stikonas>well, yeah, running 2nd pass of mes build should prove
<OriansJ`>stikonas: stupid question but is M2-Planet built mes.c able to run MesCC well enough to build TCC directly?
<stikonas>OriansJ`: I suspect the problem is libc
<stikonas>M2-Planet built mes uses a tiny libc, not full mescclibc
<stikonas>in fact, I think tcc needs even more functions than mes
<stikonas>when we build meslibc, there are two parts, one is built into libc.a and another into libc+tcc.a
<stikonas>I guess the latter are extra functions needed by tcc
<stikonas>anyway, building libc takes probably even more time than building mes
<OriansJ`>stikonas: but M2-Planet built binaries don't care about libc; the libc behavior is entirely MesCC not the binary built by M2-Planet
<stikonas>hmm, that might be true...
<stikonas>hmm
<OriansJ`>so only if the binary M2-Planet makes lacks primitives that MesCC self-hosting would include would it be any different
<stikonas>well, one can try to build it directly
<stikonas>I guess nobody have tried
<OriansJ`>M2-Planet binaries need nothing outside of a kernel supporting syscalls.
<OriansJ`>(And if the functions in M2libc are done talking to hardware directly not even that)
<stikonas>I guess one can try to comment out stuff after https://github.com/fosslinux/live-bootstrap/blob/master/sysa/mes/mes.kaem#L24
<stikonas>if that works, early bootstrap might be significantly quicker...
<stikonas>OriansJ`: building now, we'll know in a few minutes
<stikonas>or maybe more if mes built with M2-Planet is slower
<OriansJ`>stikonas: thank you for checking for me ^_^
<stikonas>well, it's not a hard check... Just comment out most of the file
<stikonas>so far no errors, 2 minutes into building tcc
<OriansJ`>I'm currently trying to find a powerpc32 linux distro that works with qemu for mescc-tools development
<stikonas>so it might just work
<OriansJ`>(just slower than MesCC built mes.c probably as no optimizations done)
<stikonas>it seems to be slower enough that I can notice, but not terribly slow
<stikonas>it as about 30% slower but now I got loads of errors...