<OriansJ>stikonas: agreed however I have a hard time understanding how an FSF project (Guix) would use a binary blob without issue. <stikonas>well, I know what you mean... Although, they also use haskell blobs <stikonas>anyway, if "Source" files are banned then even reduced binary seed bootstrap has to be improved <stikonas>even if we let configure scripts, there are bison generated files there <stikonas>well, we found one path for that, although for now it needs musl <OriansJ>as they have been giving a great many languages free passes in the name of getting a user base. <OriansJ>honestly can't blame them for the choice. <stikonas>otherwise if you have nothing, you wouldn't have any users <OriansJ>Hardline bootstrapping is always a tough sell <stikonas>but at least bison bootstrapping was once done by gio and reproduced in live-bootstrap <OriansJ>but we are coming up on 5 years of progress and paths (with cheating) that work and are making great progress of paths with less cheating. <stikonas>I also didn't like that suggestion that guix users are expected to use substitutes in your guix gnutls expiration bug... <civodul>stikonas: probably an observation rather than a suggestion <stikonas>probably, hence I didn't feel strong enough about it to reply... <civodul>i don't think everyone on this channel builds everything from source either :-) <stikonas>people should have a choice, I just don't want building from source to be completely 2nd class citizen compared to substutes <civodul>expirations happens to be a class of build failures for which we have no good solution <stikonas>yeah, I guess there is no "time" namespace <civodul>like the bug mentions there's one, but not for that <stikonas>like other namespaces used to achieve containers <OriansJ>stikonas: dongcarl is absolutely right that a new guix release with an updated gnutls would entirely solve the problem; however I find the phrase "you should fork Guix" very rude. <OriansJ>it is a handwaving way of saying that it doesn't matter if you are right, we don't care and it is very disrespectful towards the people who actually care. <OriansJ>A more approprate response would have been, this will be fixed in the next release which we plan on doing at X date in the future. <Hagfish>yeah, it is very passive aggressive when people say "it's open source, you can just fork it" <Hagfish>(that's especially true when "it" is a network service that has an established user base) <OriansJ>even a simple patch request of if you update this, in X days the guix latest will have it fixed. <OriansJ>every bug is a call to invite the party to participate and make the community better. <OriansJ>even if it means we only gain a new tester, some documentation or even a better informed user. <OriansJ>It is one thing to say "patches welcome" and quite another to suggest they "fork off" <OriansJ>fossy: I hope to upgrade mescc-tools-seed in several major ways this weekend (finally get around to renaming to stage0-posix), convert all builds to use M2libc, etc <OriansJ>So that you'll be able to leverage M2libc for the various pieces you are working with. (So you'll be able to use C standards like fputs instead of file_print) <OriansJ>Then I need to begin the long import of MesCC libc functions into M2libc <stikonas>hmm, actually, I need to remove one more file there... <dongcarl>I think we should make sure that: 1. “expiration” build failures do not place undue burden on maintainers who may need to keep cutting new releases (say release v2, time passes, then fail, v2.0.1, time passes, then fail, then v2.0.2, etc.) 2. As stikonas mentioned: building from source is not a completely 2nd class citizen compared to <dongcarl>In that sense, the package transformation options are great: they allow working around small problems _without_ the need to fork. In other words, they are solutions which seem proportional to the problem that needs solving. <dongcarl>It seems like a --without-tests-for-drv=<derivation to modify> flag would be very helpful, and perhaps a longer-term solution to the "broken test that doesn't really matter and I don't mind rebuilding" problem. Although I am not too familiar with derivation modification internals, I'd be happy to learn. <dongcarl>I do want to say that I want to continue directing members of my community to the official Guix releases (and all the other good things on Guix's website) and not my fork. I see the gnutls issue as an opportunity to demonstrate Guix’s flexibility and programmability rather than a reason to fork. <stikonas>well, guix is released approximately 2 times per year according to wikipedia, so I guess in a couple of months that issue will go away anyway <OriansJ>dongcarl: fortunately expiration build failures are a reproducible build failure as well. So distros like Debian are working to eliminate them for us. <OriansJ>in the case of derivation. turning off tests changes the derivation result (even if the resulting binary is identical) <OriansJ>This isn't a problem for bootstrappers at all <dongcarl>OriansJ: Right exactly, it would solve "broken test that doesn't really matter and I don't mind rebuilding" <stikonas>well, this is only problem if you use substitutes <OriansJ>however it requires a great many builds for those who use substitutes <stikonas>how do those expiring tests end up in test suite? accidentally? <OriansJ>hence why Guix resists changing derivations lower in the build chain. <OriansJ>stikonas: intensionally. A programmer added a test with a hard coded date in the certificate being tested. <OriansJ>probably assuming it wouldn't matter by the time the next version came out. <stikonas>could just generate certificate on the fly then with appropriate dates <OriansJ>but that takes more work to get right <OriansJ>and programmers are lazy by definition. <stikonas>and I think often people think that tests are for devs only (I might be guilty of this too) <dongcarl>OriansJ: Right, but we already have --without-tests=, which changes derivations, no? <OriansJ>dongcarl: yes but unfortunately how guix uses derivations, it wouldn't work. <dongcarl>OriansJ: Right, that's what I saw. I don't fully understand why though, if you can point me to the right manual pages, I'd be very grateful! <OriansJ>if guix version dd5e879c3a41909b991e5841934251f538c6257c needs derivation abc, you have to do exactly x,y and z to make it <pder>It appears that at least with this very trival test, doubles can be assigned to correctly and we can do simple math. Its more of a display issue <pder>The hex representations match gcc, I just cant do printf for doubles <stikonas>that's why not too many things are broken... <stikonas>although, for bootstrapping perl I have to disable minimum version checks for some reason... <OriansJ>So guix build foo and guix build foo --without-tests=1 are two entirely different derivations and can't be used to satisfy each other's dependent builds. <stikonas>oh, now I understand why without-tests doesn't work <stikonas>although, I don't see when current behaviour might be useful... <OriansJ>dongcarl: see the checksums in the names are based on the inputs to the build; so when you change an input by turning off the test. The resulting checksums must be different and thus if you need abc-foo; fgh-foo will not satisfy in guix, even if abc-foo and fgh-foo are the exact ssame binary. <vagrantc>needs something like --without-tests-recursively <pder>I havent specifically tested floats, but my understanding is that floats are promoted to doubles when calling printf <stikonas>pder: possibly, doesn't matter, you found that it's display issue <stikonas>but I guess it also affects some software that parses display <OriansJ>vagrantc: the problem is guix wants everyone to be on known good binaries <dongcarl>OriansJ: Well, I guess in an ideal world you would recursively rewrite upwards, no? <vagrantc>and yes, debian's reproducible builds tests are doing builds that should catch issues slightly over a year in advance <stikonas>pder: but didn't we have the same problem with mes libc? <pder>I am wondering if it has anything to do with printf being a variadic function. <stikonas>because two different libc's are unlikely to have the same bug <pder>The tcc-mes build prints something random. printf("%f", pi) prints 1.114 or something like that <OriansJ>dongcarl: well that would impact the packages you build as well (like bitcoin) which would result in a potentially different binary (because of embedded paths) <vagrantc>ah, the reproducible builds tests are 398 days ... i think to maximize the chance that the day, month, week, etc aren't going to be the same <vagrantc>+1year and 33days ... oh yeah, also timezone potentially tweaking the day ... that's why 33 rather than just 32 <OriansJ>However looking at guix's history the number of expiring bootstrap essential packages seem to be rare. <dongcarl>OriansJ: Okay so how about we have a flag that's --without-tests-recursively=<blah>, which disables blah's test, changes the derivation/checksum of blah, and propagates this change in derivation/checksum upwards (meaning that everything depending on blah has to rebuild)? Unless we're `guix build`-ing a specific derivation, we're basically just <dongcarl>building a specification, and that specification can still be fulfilled (albeit with a different derivation). <dongcarl>feel free to tell me if I'm completely off base <vagrantc>seems theoretically plausible, all it requires is code :) <dongcarl>Right, all I'm asking for is a sanity check on theory hehe, since I'm very shaky on that <dongcarl>Oh! I found another bootstrap essential failure... This time in `sed` <dongcarl>When running on CentOS 8, the test `testsuite/inplace-selinux` fails... <OriansJ>dongcarl: sounds reasonable however you have to think of it this way: inside of every binary is encoded paths. You can't define the path based on the binary's checksum because we don't know what it will be <OriansJ>So to solve that guix and nix just use the checksum of the sources to define the checksum of the path that is embedded in the binaries <OriansJ>and because we have deterministic builds, matching inputs should also have matching outputs <OriansJ>So package foo-1.2.3 says it needs gnutls-x.y.z not gnutls-x.y.z-disable-test1 <OriansJ>so you have to redefine gnutls-x.y.z to not include the test and then it can continue around it. That is why guix created channels <OriansJ>So (I am probably wrong here but this is my best guess) in theory creating a guix channel and importing it would enable a work around for gnutls in the bootstrap; however it would likely result in a different checksum; which may or may not be an issue. <dongcarl>I tried using a time-machine to build a commit of Guix which had gnutls fixed, but that was unsuccessful... Would a channel/inferior combo work better here? <OriansJ>dongcarl: well time-machine is about picking a git commit to create the build graph to the package in question but I don't think there is a commit where gnutls is fixed yet which could be pointed to. <dongcarl>OriansJ: I think on master they've bumped to a version of gnutls that has this problem fixed... <OriansJ>dongcarl: sadly I can't be of more help but hopefully it gave you a few ideas to explore. <dongcarl>OriansJ: No you were more than helpful as always :-) Thanks! <Hagfish>i'm really hopeful that the hurdles between there and latest perl (plus autotools) are smaller than the hurdles faced so far <Hagfish>it would almost be worth enticing someone from the perl community to help out with this project, but i wouldn't want to deny stik the credit for all this current and future progress <OriansJ>well figuring out getting an ppc64le binary to run *AND* produce output to objdump -d is kicking my ass <OriansJ>I think I figured out the e_entry behavior <OriansJ>it is only reading the first 32bits of e_entry but then it loads a full 64bits of e_entry address from the location where e_entry points to <OriansJ>but atleast e_entry can be moved into the ELF_header <OriansJ>ok first ever mescc-tools official ppc64le test is now up <OriansJ>just need to figure out the blood-elf details to getting a fully debuggable binary for ppc64le <OriansJ>but that'll have to wait till tomorrow ***lukedashjr is now known as luke-jr
***lukedashjr is now known as luke-jr
<plasma41>Is it currently possible to boot directly to a guile REPL from grub? <plasma41>Or to launch a guile repl from grub's commandline? <janneke>plasma41: guile needs a kernel, guix's initrd is written in guile though, so it "boots to guile" in a way <plasma41>Isn't grub complex enough at this point to count as a kernel, lol? <stikonas[m]>fossy: regcomp.pl is not invoked in that PR yet, I noticed it later <stikonas[m]>I can try some other 5.5 patch versions, maybe they'll just work <stikonas[m]>And embed.pl says require 5.003 but doesn't seem buildable with it... <OriansJ>plasma41: well grub could easily load the guile elf file into RAM and even jmp to its main. guile lacks mmu support, disk drivers, tty drivers and basic process management. Which would result in guile crashing and burning on its first syscall (because it didn't setup the syscall table nor populate it with functions to implement the desired functionality) <OriansJ>now mes-m2 with a M2libc native library however would actually run and work just fine. <pder>stikonas: I think I may have found the key to getting floats and doubles working properly. We just have to rebuild musl one more time using our latest build of tcc <bauen1>pder: just out of interest, do many intermediate objects change when recompiling musl with the latest tcc (checksum or diffoscope) ? <pder>bauen1: thats a good question. I'll try preserving the original build and see what exactly changed with rebuilding. The first time musl is built it is using tcc statically linked with mes libc <bauen1>could also help track down potential compiler bugs <pder>it at the very least fixed the issue with printf and floats/doubles, but possibly other things <stikonas>it might be interesting to check if it fixes perl <pder>seq from coreutils now behaves correctly <pder>Unrelated but I updated coreutils to build and install cp which was already in the makefile but just not built and installed <pder>You had it in there, just missing from the lines above <pder>We have been using the minimal cp from mescc-tools-extra the whole time <stikonas>ok, I should recheck some hacks where I mv'ed install dir instead of cp -r ***xwvvvvwx- is now known as xwvvvvwx
<bauen1>also, shouldn't a rebuild of musl with tinycc-musl be unecessary or at least a bug ? <stikonas>bauen1: well, tcc-mes might be more buggy... <bauen1>not saying that just recompiling isn't a fast way to move forward <stikonas>well, yes, in principle it shouldn't be... <bauen1>actually no that's an unimportant one <stikonas>you can switch to GNU pass or KeyPassXC for storing passwords... <bauen1>stikonas: i already use password-store <pder>stikonas: thats great news. I saw your comment about splitting the rebuild of musl into a separate section and wondered if it might be the best to rebuild tcc once again so it is linked with latest libc.a <pder>I think related steps might be able to be condensed. For example, the multiple iterations of perl could be one step similar to bison <stikonas[m]>pder and perl 5.5 should already allow us to run newer automake <pder>I havent looked at all at autotools code. Does it use a lot of perl modules? <Hagfish>(i see that 5.8 is the first version packaged as bzip, and after that the major releases happened about once per year. 2008-12-14 perl-5.8.9.tar.bz2) <stikonas[m]>pder old version don't... I haven't checked new ones <pder>what version of automake and autoconf are we targetting? We need a version that can handle binutils 2.14 right? <pder>I assume we were going to try to build the same toolchain as guix with gcc 2.95, binutils 2.14 and glibc (I don't remember) <stikonas>pder: well, that's one option but that's not a requirement <stikonas>pder: but first I think we need to build any binutils and maybe rebuild musl/tcc again <stikonas>it's a bit better now but I think once we have ar we should use it <stikonas>well, first fossy should merge PRs that you already prepared... <stikonas>pder: by the way, do you know if it was just our tcc that was broken or also on Guix? <pder>No idea on guix, I have never actually run their bootstrap <stikonas>pder: probably worth explaining a bit in README <stikonas>(mention that floats seem to be a bit broken in mes libc) <pder>ok, hopefully I can understand a bit more later tonight and get at the root cause <stikonas>but at least we now bootstrapped past that <pder>bauen1: I tried comparing builds of musl using tcc-mes and tcc-musl and then took a hash of all of the *.o files in each build and 92 out of 1266 files are different <pder>Many are in src/match, but also stdio/vfprintf.o differs <pder>Havent compared the assembly output yet ***ChanServ sets mode: +o rekado
<OriansJ>bauen1: I suggest using ansi coloring to avoid putting in the wrong commands (or passwords) in windows were they don't belong. <bauen1>ah, this was my window manager acting up, so i thought i was in another window, and i typed it by reflex <OriansJ>bauen1: I don't think I've typed in a password in 8 years. Mooltipass is even approved equipment for the State of Michigan (you have to pay for it but hey) <bauen1>OriansJ: sadly you can't really copy+paste over vnc <bauen1>and no matter what you try, nobody here can persuade me to another password manager \o/ <OriansJ>bauen1: mooltipass is treated like a keyboard. <bauen1>if i could figure out the magic behind making my keymap troubles over vnc disappear i could have something emulate keypresses i'm sure <OriansJ>because it literally leverages the keyboard driver <stikonas>well, where it's possible I'm using U2F two factor authentication... Made it myself by reflashing ST-Link V2 debugger... <stikonas>although, that again wouldn't work remotey <stikonas>also got gnuk token for gpg and ssh, but that one I bought from FSF... <OriansJ>stikonas: an understandable choice. Although for some people yubikeys tend to better fit their use case. <stikonas>yeah, I know some people with yubikeys... <stikonas>I guess easier, you just buy it and can use it <OriansJ>Security like freedom requires people to find where they are comfortable <OriansJ>and honestly any 2-factor auth is better than memorizing passwords <bauen1>so i have a few vps, the entire install process has been automated, but now i need to get the public ssh key after install to my laptop so i can add it to known hosts and start ansible to configure them <bauen1>and for backup i want to be able to login over the console, at the same time not just login-as-root-without-password <bauen1>which is hard to do since vnc tends to screw up my keymappings and i can't figure out how to get that to work "seamlessly", so i still have some easy to type passwords for those cases <bauen1>but apart from console login they're not used, and password authentication for ssh is disabled <OriansJ>bauen1: well VPS companies like Linode provide a ssh terminal server (so copy paste will work fine) but companies like DigitalOcean with their Crap interface, you need something like mooltipass to get a strong password autotyped. <fossy>pder: i'm about to merge checksumming, then would you mind rebasing your PRs and adding checksums <fossy>otherwise i'm going to be playing catchup forever lol <stikonas>hmm, and I'm waiting for musl and cp merged... <stikonas>fossy: but now you we have an example of changes that change all checksums... <stikonas>that can probably be merged without CI...