IRC channel logs

<stikonas>probably something goes wrong far earlier...

<stikonas>it's hard to see how this can happen though

<stikonas>other than that sign extension of mode, code seems identical

<mwette>Are you just adding `mode;' (i.e., noop) in a function?

<stikonas>either that or in assembly..

<stikonas>but the only effect was that global variables and function addresses would have slightly different absolute addresses in the memory

<stikonas>anyway, mescc->tcc-mes step in our bootstrap chain is one of the hardest to debug...

<stikonas>well, M2-Planet to mes probably too but generally that now works reasonably well

<mwette>thanks; I'm guessing for tcc that wouldn't change the stack, right?

<fossy>hey stikonas, do you remember why we create our repo packages with --mode=go=rX,u+rw,a-s?

<fossy>Googulator: thank you :)

<stikonas>mwette: it shouldn't but this needs more investigation, probabably way more than i have time today

<stikonas>fossy: hmm, not really

<stikonas>fossy: probably just picked some mode

<stikonas>so that it is reproducible

<stikonas>and does not depend on some strange umask that one can set

<fossy>ohh, umasks maybe was it

<fossy>Googulator: by the way, I nearly have --repo and non --repo mode bit-for-bit reproducible under bwrap/chroot modes :D

<artemist>Is there any documentation about what the hex2 special characters are supposed to mean? I'm looking at aarch64 again which has a massive number of immediate encodings

<matrix_bridge><Andrius Štikonas> artemist: aarch64 is a bit of a mess

<matrix_bridge><Andrius Štikonas> For most arches it is here https://github.com/oriansj/mescc-tools/blob/08d5a0679fa4dc00babe02306f1bc50703083389/M1-macro.c#L547

<matrix_bridge><Andrius Štikonas> So offsets or absolute addresses of various lengths

<matrix_bridge><Andrius Štikonas> Oh, sorry that link is for M1

<matrix_bridge><Andrius Štikonas> Here https://github.com/oriansj/mescc-tools/blob/08d5a0679fa4dc00babe02306f1bc50703083389/hex2_linker.c#L366

<matrix_bridge><Andrius Štikonas> It's the same meaning though

<matrix_bridge><Andrius Štikonas> For fixed word size riscv it is used for different instruction formats

<matrix_bridge><Andrius Štikonas> https://github.com/oriansj/mescc-tools/blob/08d5a0679fa4dc00babe02306f1bc50703083389/hex2_word.c#L37

<matrix_bridge><Andrius Štikonas> Arm tries to use the former scheme but with some hacks https://github.com/oriansj/mescc-tools/blob/08d5a0679fa4dc00babe02306f1bc50703083389/hex2_linker.c#L259

<matrix_bridge><Andrius Štikonas> Even though it maps to fixed word size model better

<muurkha>makes sense; arm64 and risc-v were designed about the same time with mostly the same objectives

<muurkha>in accordance with the conventional wisdom of that time

<stikonas>yeah, but stage0-posix-aarch64 predates stage0-posix-riscv64 and this word based flow...

<oriansj>and no one has yet to go back and rework it

<oriansj>into the better fitting word pattern

<artemist>I've been working on that code, it's just that the format characters are referenced in a few places so documentation would be easier than finding all the code

<matrix_bridge><Andrius Štikonas> There is some info here https://github.com/fosslinux/live-bootstrap/blob/master/parts.rst#6hex2

<artemist>thanks

<stikonas>janneke: I've sent you a small fixup for the position independent patch

<stikonas>one define (needed to build tcc but not for mes) was missing

<janneke>stikonas: great

<GoogulatorMobile>Is perl-5.32 a prerequisite for building openssl?

<stikonas>GoogulatorMobile: hmm, hard to tell

<stikonas>at the very least 5.10 is prerequisite

<stikonas>but out 5.10 build might be missing some modules

<stikonas>we only built it as a stepping stone to newer perl

<janneke>stikonas: i'll squash it with your previous patch

<stikonas>janneke: sure that's fine

<stikonas>that's why I said fixup too

<stikonas>I've only built mes when I sent the first patch

<stikonas>but now I've built tcc-mes too

<stikonas>(which is not working well yet but unlikely to need new defines)

<stikonas>(just need to fix various segfaults)

<GoogulatorMobile>stikonas: I'm trying to reduce the amount of sources used before we get SSL support - there are some obvious ones (gmp & co, which are only needed for Guile, built much later), but it doesn't quite get us to sub-256MiB

<stikonas>yeah, I know...

<GoogulatorMobile>So I'm looking for more things that can be pushed to after SSL

<janneke>stikonas: right, that makes sense

<stikonas>GoogulatorMobile: coreutils?

<janneke>(iwbn to have a test in mes that needs it, but it's not the only instruction that isn't under test)

<stikonas>I doubt that it is needed so early

<GoogulatorMobile>coreutils is indeed low-hanging fruit

<GoogulatorMobile>Especially since we build 2 versions early on

<stikonas>janneke: I like risc-v defines now. It's basically a complete set of what we'll ever need and full lines are constructed out of small parts

<GoogulatorMobile>& the official tarballs are a lot bigger than the ones obtainable from savannah

<GoogulatorMobile>But alas, fossy vetoed my plan to at least use savannah tarballs there

<stikonas>on the other hand on x86/amd64 we can't easily combine e.g. mov + rax from two parts...

<GoogulatorMobile>(difference seems to be lots of pregen documentation in the official tarballs)

<stikonas>well, the problem is that those snapshot tarballs are not guaranteed to be reproducible

<stikonas>as they are generated on the fly

<stikonas>but yes, official ones do include more files

<stikonas>which we then remove...

<GoogulatorMobile>Also, official ones are gzipped

<GoogulatorMobile>Savannah has choice between gz/bz2/xz

<stikonas>well, official ones often predate bz2 and xz

<GoogulatorMobile>IIRC even recent ones are gzip-only

<GoogulatorMobile>Doesn't matter once we have network

<GoogulatorMobile>But before network & SSL, it's a big deal

<stikonas>a lot of stuff is xz

<stikonas>but it depends on specific project

<GoogulatorMobile>Some GNU projects insist on sticking to gzip only for some reason

<GoogulatorMobile>E.g. bash

<stikonas>"easier" to bootstrap

<GoogulatorMobile>Others ship gz and xz, or gz and lzip

<GoogulatorMobile>But not bz2

<stikonas>there is now bz3

<GoogulatorMobile>...wtf

<stikonas> https://github.com/kspalaiologos/bzip3

<janneke>stikonas: yes, the risc-v defines are nice

<stikonas>the only annoying thing is the reverse order...

<stikonas>which comes from trying to make M0 (and I guess M1 to some extent) simpler

<stikonas>well, opcode is at the end... other than that order doesn't matter

<GoogulatorMobile>I'm also leaning towards shipping our own Linux tarballs

<stikonas>GoogulatorMobile: how about trying to build some other ssl library?

<stikonas>maybe that can be done much earlier

<stikonas>something like nss

<GoogulatorMobile>Does curl support anything other than openssl?

<stikonas>oh that might need ninja

<stikonas>CURL_SSL="openssl -gnutls -mbedtls -rustls")

<GoogulatorMobile>nss is a 60MiB+ tarball, so it's bad for other reasons

<stikonas>gnutls uses autogen

<stikonas>so that's bad for pregen files...

<GoogulatorMobile>(we currently download it just to extract the certificate list)

<stikonas>rustls is bad too

<GoogulatorMobile>mbedtls sounds good

<stikonas>check build system first

<stikonas>there is makefile that has

<stikonas>PERL ?= perl

<stikonas>so it does use perl too

<stikonas>perhaps older perl could work

<stikonas>but you might end up chasing dead end...

<GoogulatorMobile>Apparently perl is used to regenerate some files

<stikonas>GoogulatorMobile: perhals that's the full list of what curl supports https://curl.se/docs/ssl-compared.html

<stikonas>doesn't have rustls here in this list though but that one is not good for bootstrapping anyway

<GoogulatorMobile>wolfssl looks interesting, even ships its own mini version of curl

<GoogulatorMobile>but its approach to licensing is weird...

<stikonas>openssl < 3 is weird too

<stikonas>what does wolfssl do?

<stikonas>GoogulatorMobile: at leastrecent versions of wolfssl

<stikonas>would need new perl

<stikonas>they depend on autoconf 2.69

<GoogulatorMobile>We do build 2.69 before Linux

<stikonas>oh, maybe it's automake that needs newer perl

<stikonas>I guess 1.15 or 1.16...

<stikonas>Makefile.am doesn't specify the version...

<stikonas>so you would have to do it by trial and error...

<fossy>GoogulatorMobile: wolfssl actually looks very promising

<fossy>apart from cmake

<fossy>oh! it still has autoconf too

<stikonas>but pregen files still need perl

<stikonas>though maybe older perl is fine

<fossy>i only see one pregen file using perl

<fossy>and that is trivally removable

<stikonas>but don't we need to ship nss for certs anyway?

<stikonas>how does nss compare to wolfssl?

<fossy>couple of alternatives to nss;

<fossy>1. insecure HTTPS connections before we can get to NSS

<fossy>(don't validate certificates)

<fossy>2. pick and choose certificates required instead of taking them from Mozilla's store

<stikonas>you probably can't do 1. without ssl library...

<stikonas>as you still need to follow ssl protocol..

<stikonas>and 2. could be prone to breakages too

<fossy>oh, for sure - 1 just gets around needing to install certificates

<fossy>so one option would be wolfssl + no certificates at some earlier stage in the bootstrap, and then later on moving to our same current setup

<Googulator>I would hope to ship something other than full-blown NSS for certificates

<Googulator>we basically just need one file from the NSS tarball

IRC channel logs

2024-01-22.log