<stikonas>unfortunately we seem to have to jump through all versions, 5.003->5.005 did not work <Hagfish>"Be aware that only 5.004 and later versions of perl are maintained." <OriansJ>fossy: well to a degree bootstrapping is a game of reducing the amount of cheating until it is fully honest. Guix cheats with Guile, you can cheat with generated kaem scripts and it'll be a huge improvement. The goal isn't to do it perfect but make iterative steps towards a sustainable bootstrap chain. Because it all has to be supported *FOREVER* So we have to take into account porting to new hardware architectures and Operating Systems <OriansJ>in the future and the development costs involved. <stikonas>Hagfish: early versions are pre-release anyway <stikonas>Hagfish: but later versions kind-of have dependency on perl unless you cheat a bit <stikonas>there are 3 scripts in perl that are written in perl and spit out 3 C header files <stikonas>anyway, so far live-bootstrap is trying to avoid any cheating <stikonas>it might not be the simplest bootsrap chain <stikonas>but up to kernel it should be as honest as it can get <stikonas>anyway, most of the work in maintaining new hardware architectures is for compiler/libc and binutils <stikonas>so it might be easier to get new arches added if we manage to bootstrap newer versions of these tools <OriansJ>stikonas: I understand the goal is to avoid all cheating. However I am just saying cheating on the build script side is something far easier to fix later than cheating on the source code side. <OriansJ>eg pregenerated source files vs generated kaem scripts. <OriansJ>and the pregenerated source files are the far bigger threat. <stikonas>well, it also depends on how bad they are... <OriansJ>as it is considerably harder to backdoor a kaem script than it is to backdoor C code. <stikonas>and some generated list of enums or defines is less scary <stikonas>well, I don't think we disagree on that anyway <OriansJ>I guess the disagreement could be reduced down to me expressing that I feel that the goal of "finishing now" seems too attractive and it looks like rushing to get done with me having concerns about future support tasks that might be required. <OriansJ>but ultimately it is fossy's choice as who does decides. <OriansJ>Don't get me wrong I appreciate all the hard work being put in right now; but I can't help but be concerned about pushing too hard and hitting burnout. Which makes me want to suggest shortcuts and alternate solutions to delay dealing with some of the harder problems until later when you can hit it from a fresh perspective. <OriansJ>I really need to finish the final bit of the M2libc transistion so that I can merge deesix's work. <OriansJ>but I should be able to finish it in the next hour <Hagfish>i'm hoping that once a modern version of perl is reached, more people will start to "get it", and that steps like m2 will be appreciate as important practical software, rather than curiosities <OriansJ>Hagfish: the news of GCC being "bootstrapped" in Guix went over as well as a wet fart. So I don't see perl as some magical milestone where everything comes together. <Hagfish>yeah, i was just thinking about that <Hagfish>maybe because people aren't so familiar with Guix? <OriansJ>Hagfish: Guix is atleast an order of magnitude bigger than Bootstrappable <OriansJ>Now if we could getting a bootstrap path in Debian, then we will know the best possible reaction to bootstrap work. <Hagfish>maybe there could be some "adversarial interoperability" <Hagfish>does debian publish the precise versions (and hashes) of each package it uses on its build machines? <OriansJ>I imagine getting Guix being able to generate dpkgs would make the transistion much easier. <Hagfish>we could make a CI job that builds those debian packages and checks the hashes against the ones Debian publishes <ericonr>Hagfish: isn't that what they already do in reproduible? <OriansJ>Hagfish: I can't speak to the Debian side but vagrantc would be able to better to the answer to that. <OriansJ>ericonr: I believe they have their own custome build setup <ericonr>afaik they do some fuzzing around to verify stuff, so some machines have usrmerge while others don't, and they do builds with different locales configured and stuff <ericonr>unless Hagfish wants to build those packages from outside debian completely? <Hagfish>i guess the jenkins jobs are all available for inspection <Hagfish>some of the build variability is due to things like the build path <Hagfish>it would be possible to set the correct values for that <Hagfish>i guess if the outputs of debian's builds match the outputs from the same process run under a (more) trusted build environment, then that's fine <Hagfish>it would be nice not to have to rebuild all of debian ourselves, though <Hagfish>just the machine images that run the build process <OriansJ>in short Debian varies things intensionally to shake out code that makes builds non-reproducible; while Guix freezes things solid so that builds are reproducible by default <ericonr>testing "noarch" packages between different archs is also pretty cool <OriansJ>I find the pseudo-antagonistic personality conflicts in regards to style and method of solving the bootstrapping problem actually surprisingly helpful to reducing the problem long term. Despite the short term wasted duplication of efforts. <OriansJ>finally managed to merge deesix's amazing unification work in M2-Planet <OriansJ>This will really simplify the ppc64le work <bauen1_>with "just" 3 perl scripts required to build perl, maybe it is possible to convince upstream to replace these ? ***bauen1_ is now known as bauen1
<bauen1>but i'm not sure what tools exactly it requires, but i would image: perl, gcc, binutils, coreutils, autotools, make, bash would go a decent way <fossy>stikonas[m]: whenever you have time, please review the checksumming PR <pabs3>(rebootstrap is focussed on the case of automatically bootstrapping Debian arches from other Debian arches) <bauen1>ah right i mixed something up again <pabs3>the DebianBootstrap wiki is more general <pabs3>#debian-bootstrap (OFTC) is the place where helmut/josch/etc discuss it <stikonas>well, rebootstrap is not that low level as what people do here... <stikonas>I guess it just does some cross-compilation <bauen1>then there's half of debian core development tools written in perl lol <stikonas>well, yeah, although, I found perl easier to build than expected <stikonas>(and different perl versions are quite similar in terms of building) <stikonas>so not too hard to jump via a few intermediate versions of perl <gforce_de1977>stikonas: for later the question is how far away is recent perl 5.32.1 <stikonas>the more important questions I think is how soon we can bootstrap perl's ./Configure script <stikonas>I am not going to write custom Makefiles for all perl versions up to 5.32.1... <gforce_de1977>8-) as long as './Configure' runs, the road to 5.32.1 is straigt <stikonas>gforce_de1977: not that simple, you need to create ./Configure, not just run it <stikonas>and I don't know if new perl's build with tcc. <stikonas>so we need to bootstrap perl a bit before we can use it <stikonas>gforce_de1977: but it might not be necessary to do it now <stikonas>at least some versions of automake already run on what we have <stikonas>adn probably if we get to 5.6 or 5.8 most will run <stikonas>gforce_de1977: I suspect we still need miniperl that I built to regenerate those opcode.h, keywords.h and embed.h <stikonas>i.e. microperl probably deals with ./Configure only <OriansJ>gforce_de1977: well he certainly isn't wrong in those 2 cases <nore>(beware, compilation is single-threaded for now, and takes 14h on my machine) <gforce_de1977>nore: very good. time is *not* the problem, computers have plenty of <stikonas>nore: oh, ocaml also depends on itself :( <stikonas>well, good to hear that it's solved though <nore>yeah, the ocaml compiler is written in ocaml <nore>and the lexer generator used for the compiler also depends on itself (and ocaml) <nore>well, ocaml commits a bytecode version of the compiler and the lexer generator to its repositories <nore>so it just uses those I guess? <stikonas>would be nice to have a list of all these bootstrap projects ***ChanServ sets mode: +o rekado_
***rekado_ is now known as rekado
<stikonas>nore: by the way, maybe write to mailing list instead of just on IRC? <roptat>yes, we simply use the bytecode compilers that are provided with the sources <roptat>I'm planning to write a recipe for camlboot and fully bootstrap ocaml 4.07 (which is not the latest, but nore has plans for more recent versions too :)) <stikonas>but can't newer ocaml be bootstrapped with ocaml 4.07? <stikonas>(at least until camboot supports new ocaml) <roptat>nore knows better than me, but I know there's at least a problem with the parser generator (they switched from ocamlyacc, written in C, to merlin, written in OCaml) <roptat>hopefully, we can build menhir from ocaml4.07 and generate the grammar from it (its own grammar is compatible with ocamlyacc iirc) <roptat>also, I think recent versions of ocaml are not compatible with ocaml4.07. the bootstrap compiler is regenerated from time to time because of that, and not necessarily on release commits <stikonas>roptat: ouch, that's even worse than rust then, rust can at least be bootstrapped using previous release. Although, there are not quite a few jumps from 1.29 to latest <roptat>right, 20 versions for rust, and only 5 for ocaml (but I don't know how many bootstrap commits) <g_bor[m]>I am looking at getting some custom firware work on qemu. <g_bor[m]>Is the port 0x80 implemented in qemu? How can I get a log of what is sent there? <stikonas>g_bor[m]: isn't that question for qemu channel? I don't think people here would necesserily know <g_bor[m]>I was just wondering if anyone was playing with firmware here... <g_bor[m]>you can add -d in_asm to your qemu command line, and it outputs every single guest instruction executed <fossy>nore: very well done, I had been interested in ocaml bootstrap for a while, glad to see someone did it <stikonas>gforce_de1977, fossy: so I briefly checked that microperl, it indeed deals only with ./Configure, so we need earlier miniperl's to get to 5.7. But then in principle microperl might be useful (to avoid writing our own makefiles) <stikonas>I think they just abandoned idea of rewriting configure in perl... <gforce_de1977>stikonas: what is the problem with "just" download a recent version, build 'miniperl' aka 'metaconfig', generate ./Configure and let in compile? <stikonas>miniperl depends on some pre-generated header files <stikonas>if you are fine with those, then you might be able to build it <stikonas>(although, not sure how good tcc will be with modern perl) <stikonas>but we are trying to cleanly build those header files too <roptat>mh... reading the wiki after a while <roptat>"Julien started writing a bootstrap compiler for Scala in Java, which is already able to produce an AST for Scala files and produce JVM bytecode" well, I never produced JVM bytecode, and the AST was incorrect, I never managed to write a correct parser for any complex scala file from scala sources <roptat>I recently looked at this work again, and I decided to not use a parser generator, because they add to much constraint, compared to what scala does <roptat>even the lexer will not give you the same symbols depending on the state of the parser, so I just gave up on this idea ^^' <g_bor[m]>roptat: iirc that compiler actually did something, don't know eaxctly what, but some parts of it were working... <roptat>last week-end I spent some time on that again, and managed to find and reimplement scala's lexer (well, a smaller version of it because I don't really need nice error messages or scaladoc support) <roptat>I think my lexer is correct, but I still need to tackle the parser. Hopefully this time the AST it produces is correct <g_bor[m]>I guess scala is one of the most complex languages I have ever encountered <roptat>a nice thing I found: when it encounters a <, it will look at its surrounding and determine whether it's part of an operand (like -> for instance) or if it's the beginning of an XML literal <roptat>then if the parser encounters the XMLSTART symbol, it's supposed to switch lexer temporarily and use it to parse the XML literal, then switch back to scala's parser <roptat>some tokens depend on whether we're inside braces or parenthesis, ... <roptat>scala has "interpolated strings", that you can write like symbol"something ${escape}". Normally "symbol" would be an identifier, but in this case they need to produce a different token <roptat>and obviously inside ${} you can put arbitrary scala code, so the lexer needs to rember it was inside an interpolated string <roptat>no wonder I couldn't do it with a normal parser generator <roptat>anyway, I'm too busy to continue that before march I think, but it's already been at least two years since I started looking into that, so that's not so long to wait for more news :p <roptat>oh, and during the guix day I tried to contact the author of JaCo, a compiler for a superset of Java 1.4, that was used to build the first versions of Scala, but never received an answer, unfortunately <roptat>the one on the webpage, but I tracked the author down and sent them an email to another address :) <OriansJ>roptat: well perhaps Guix needs to consider the forbidding of the use of generated "Source" files for builds to help highlight the bootstrapping problem. <stikonas>there is no way I would call bytecode source