IRC channel logs

2020-11-04.log

back to list of logs

<siraben>Hello, any Nix users here?
<xentrac>haha
<rain1>not atm but i tried nix in the past
<xentrac>certainly many of us have been strongly influenced by nix
<OriansJ>siraben: how can we help?
<siraben>OriansJ: I was wondering if it was possible to build Guix's small bootstrap using Nix
<siraben>Or bootstrapping TCC/GCC from stage0, if that is done.
<OriansJ>siraben: that would be a yes (as of the last time I checked) but stage0 to MesCC isn't complete yet
<siraben>OriansJ: Do you know where I can find that Nix derivation?
<siraben>What's the status of the stage0 project, is it still active?
<OriansJ>it is quite easy to build Guix's bootstrap binaries (it is mostly just building guile)
<OriansJ>statically of course
<rain1>afaict there is no nix derivation, but it would be a nice thing to have
<OriansJ>the status of the stage0 project is I am slowly making progress. The jump from getting a working C compiler to a suitable scheme turned out larger than expected.
<siraben>Oh looks like there's a mention of TinyCC bootstrap via guile and mescc https://reproducible-builds.org/files/ReproducibleSummit4EventDocumentation.pdf on page 69
<siraben>Requires i686-linux
<siraben>OriansJ: I see. So is the goal to write a Scheme interpreter to run the mescc bootstrap?
<OriansJ>siraben: MesCC can run on guile and be used to build TCC and do the full bootstrap; there are bash shell replacements with gash and core-utils replacements with gash-utils
<siraben>I see. And how small does the initial Scheme have to be? R5RS conforming?
<OriansJ>siraben: well that is the hard part; to run guix, one has to be very compatible with guile but most of the guix devels do not restrict themselves to a subset of guile primitives while programming.
<siraben>OriansJ: do you need help with the scheme part?
<siraben>Where is development taking place?
<OriansJ>siraben: I never reject help
<OriansJ> http://github.com/oriansj/mes-m2
<OriansJ>it contains a snapshot of MesCC which can be run using the example guile command
<siraben>Nice. Which parts are still remaining to be implemented?
<OriansJ>siraben: Well right now fixing macro support is the big problem (https://github.com/oriansj/mes-m2/blob/master/mes_macro.c)
<siraben>Also, this might be interesting, https://github.com/blynn/compiler (bootstrapping from Haskell)
<siraben>From ANSI C to a mini Haskell
<siraben>Ah hygienic macro expansion.
<siraben>Why do that part in C instead of Scheme?
<OriansJ>siraben: so that evaluation of exit or display do not occur doing the macro expasion.
<siraben>OriansJ: what do you mean by that?
<OriansJ>so when (define-macro (backwards . body) is used; and (backwards (display 1) (display 2) (display 3)) it shows 3 2 1 and not 1 2 3
<siraben>I see.
<OriansJ>minor truncation: (define-macro (backwards . body) (cons 'begin (reverse body)))
<OriansJ>now mes-m2 does that correctly
<siraben>Are you going to implement syntax objects like in R5RS?
<OriansJ>siraben: I am avoiding syntax-case for as long as possible
<siraben>Oh, how come?
<siraben>I'd imagine implementing that would let you go so much higher up
<OriansJ>siraben: well if you wish to write it in C code; I will accept it
<OriansJ>as no one has ever implmemented it in C
<siraben>I've seen it done in Scheme 9 From Empty Space
<siraben> https://github.com/reflectionalist/S9fES/blob/0ade11593cf35f112e197026886fc819042058dd/s9.c#L1850
<OriansJ>ok; then it would be mechanicial work to reduce it to the C subset used in M2-Planet
<siraben>Ah, interesting.
<OriansJ>but I must be going to work; I'll check back over lunch
<siraben>Looks like the entire implementation is a single file, tool.
<siraben>Ok, talk later.
<rain1>thanks for the blynn link siraben
<siraben>rain1: no problem! do you find it interesting?
<rain1>yeah
<siraben>It's the most complete toy implementation of Haskell I've seen
<siraben> https://github.com/blynn/compiler/blob/master/vm.c
<siraben>Oh I should also link his blog posts where he explains it all
<rain1> https://bootstrapping.miraheze.org/wiki/Main_Page added it to my page
<siraben> https://crypto.stanford.edu/~blynn/compiler/quest.html
<siraben>To summarize, the core language is SKI combinators (with extra combinators for brevity)
<siraben>Then each successive compiler can parse more and more, such as lambda calculus, then adding ADTs, typechecking, typeclasses
<siraben>I don't have time ATM but writing a Scheme in Haskell to bootstrap mes-m2 would be possible
<rain1>One thing I think would be good is to formalize the sets of packages we need at every stage to build the next thing
<siraben>What does it look like right now?
<siraben>Here's one I have in mind now; ASM → SKI → Lambda calculus → mini Haskell → R5RS Scheme
<siraben>Or, ASM → SKI → Lambda calculus → R5RS Scheme
<siraben>as an alternate to the current mes-m2 path
<rain1>the thing about thath is that SKI code is hard to read, isn't it? it's almost a binary blob
<siraben>Yeah, but it can be verified very easily. Does that help with trusting trust?
<siraben>The bootstrapping programs would be completely pure too (simple inspection revealing no IO combinators)
<siraben>Ben Lynn mentions trusting trust here https://crypto.stanford.edu/~blynn/compiler/sing.html
<siraben>"Nonetheless, each of the our terms seem short enough that a sufficiently motivated human could verify them."
<OriansJ>now siraben that is an idea I never considered. Bootstrap GHC to bootstrap scheme.
<siraben>OriansJ: Not bootstrap GHC, but Ben Lynn's Haskell compiler say.
<siraben>Which is super minimal and has a clear bootstrapping path itself
<OriansJ>The license terms I could not find
<siraben>I'll open an issue
<siraben>What should it be under, MIT?
<OriansJ>siraben: any FSF approved license will be fine
<siraben>Does that include MIT?
<OriansJ>yes and the ISC
<siraben>Ok.
<OriansJ>also I can imagine extending Ben Lynn's Haskell compiler to build GHC
<OriansJ>It'll be alot of work but not something impossible
<rain1>that's an awesome idea
<OriansJ>the problem is getting someone to put the man hours in to make it happen.
<OriansJ>but it would ultimately solve the GHC bootstrapping problem
<OriansJ>(even guix and nix just use a binary GHC)
<siraben>But Ben Lynn → Scheme → GCC → GHC would be easier, no?
<siraben>Since Scheme → GCC → GHC has been done
<OriansJ>siraben: even with GCC, one can not bootstrap GHC
<siraben>Oh? I didn't know that.
<OriansJ>One needs GHC to build GHC
<rain1>at the very very low level we have: Hex -> Hex with labels -> Hex with labels and macros -> Assembly
<OriansJ>-> Minimal C -> cross-platform C -> what we are working on now
<OriansJ>but yes if we can do M2-Planet -> Ben Lynn -> Scheme -> GCC; I will be very happy
<OriansJ>but there are the two problems of getting a guile compatible scheme in that haskell subset and getting Ben Lynn's working into M2-Planet's Subset of C.
<rain1>one thing I think is worth taking advantage of is the fact that guile itself is already bootstrappable
<rain1>this should mean we don't need to implement a new implementation of guile
<OriansJ>rain1: psyntax.pp is still a bootstrap issue but it is being worked on as I understand by the guile time.
<rain1>ah thats good
<siraben>OriansJ: what's even possible is if we have Assembly → Ben Lynn
<siraben>since it's a single file C program that can be hand-translated
<siraben>Or even written in Forth
<xentrac>uhoh, now you're about to find out that OriansJ is allergic to Forth :)
<siraben>Yeah I read his comments on Forth in the forth file
<siraben>That it was harder to implement than Lisp, heh
<siraben>xentrac: github.com/siraben/zkeme80
<OriansJ>siraben: there is no magic in bootstrapping, just hard work...
<siraben>I'm aware
<siraben>Though I'd like a path of least resistance, so to speak
<siraben>And Haskell → Scheme seems pretty trivial
<OriansJ>ok but getting from M2-Planet to haskell still needs to be considered carefully
<siraben>Ben Lynn* so not full Haskell
<OriansJ>yes I admit it is promising
<civodul>rekado_ wrote about an attempt to bootstrap Haskell at https://elephly.net/posts/2017-01-09-bootstrapping-haskell-part-1.html
<civodul>(expired certificate)
<OriansJ>but the license has to be addressed before we can use any of it
<siraben>Ok
<OriansJ>Then we can convert the root into M2-Planet form and go from there
<OriansJ>We never reject ideas, only ignore those which no one wishes to do the work for.
<siraben>Right
<siraben>Ok PR opened
<siraben>OriansJ: alternatively it could bootstrap from SKI combinators to a minimal Scheme, there's nothing haskell specific about ben lynn's compiler except that it uses lazy graph reduction
<OriansJ>So if we get past the license problem, I'll help with converting the root into M2-Planet and then the rest will just be a kaem script
<OriansJ>then we need people to choose to do the scheme work in the haskell subset allowed by Ben Lynn's work
<xentrac>siraben: haha, awesome!
<emorrp1>Hi there, I'm fascinated by guix's plan to bootstrap all the compilers from scratch, down to 60MB is cooln
<emorrp1>We're currently working on introducing kotlin to Debian by rewriting the build steps in groovy: https://salsa.debian.org/android-tools-team/admin/-/issues/27
<Hagfish>emorrp1: that's great to hear
<emorrp1>disclaimer: the second build step doesn't quite work yet, and we are still initially relying on upstream provided jar
<Hagfish>there are lots of languages and ecosystems out there that don't seem to value bootstrapping, but there has been some good work done within Debian towards these efforts
<Hagfish>thanks to the great work of OriansJ and others in this channel, it should be possible to get the guix bootstrap down well below 60MB of binary
<emorrp1>Yes, I saw references to a stage0 written entirely in 1M of assembler or some such
<emorrp1>As I said, fascinating idea and glad to see core projects are receptive to it with e.g. the XZ situation
<civodul>ericonr: nice! IIRC roptat has been looking at kotlin and other Java things
<civodul>so there might be ways to share the work!
<ericonr>civodul: I think you meant to mention emorrp1 :)
<civodul>oops, right :-)
<ericonr>fwiw, void linux bootstraps java with gcc6
<civodul>apologies! tab completion and all
<civodul>nice, i didn't know there were other distros doing that
<civodul>IIRC we use to use GCJ as well, but rekado_ managed to get rid of that step (?)
<ericonr>well we don't start from a tiny seed, but it's kinda reasonably self contained
<civodul>yeah, it's not black and white anyway
<ericonr>fossy understands better about this stuff, I'm mostly a lurker here
<civodul>having Java bootstrapped is already nice and an improvement over what many distros do
<civodul>i'm also a lurker :-)
<fossy>yes we use gcj
<fossy>but bootstrappable isnt really a void priority from whenever I've tried to bring it up
<fossy>well, bootstrapping the package building system is
<fossy>but not like a rust bootstrap or that
<ericonr>I think no one wants to maintain all the old LLVM versions and consecutive rust versions necessary for that :P
<fossy>fair
<civodul>"no one"? :-)
<emorrp1>Happy to hear from roptat if they have any ideas
<ericonr>civodul: on void, I meant :)
<ericonr>fossy: but we also moved off of the go bootstrap package, for example
<emorrp1>Debian has a weaker bootstrap requirement in that the initial upload can be a binary package, but it makes porters efforts easier
<ericonr>so yes, bootstrappable as in this channel isn't a priority
<civodul>as for reproducible builds, i guess it takes time to raise awareness
<civodul>but that's ok, y'all can be the visionaries ;-)
<emorrp1>IIRC there's a plan to mass re-upload all the old Debian packages that haven't been recompiled for reproducibility
<lfam>It's interesting to ditch the Go bootstrap, because it's so easy to use
<lfam>I mean, it's easy compared to what other languages might do
<fossy>well it was due to architecture support
<fossy>I'm going to PR readding it for archs supporting it
<lfam>I see, that makes sense
<OriansJ>emorrp1: stage0 is 250byte bootstrap binary not 1MB; that is Mes.c built via MesCC into M1 macro assembly
<OriansJ>it would be quite unreasonable for any distro that doesn't have a functional package manager to maintain the long string of compilers required for most complex languages. Fortunately guix and nix can provide the binaries for all other distros. Debian and void should just build exporters for guix; so that they can just use guix for their build farms and save alot of trouble.
<lfam>I've had a similar thought :)
<OriansJ>if only someone familiar with guix and debian build processes; put in the required functionality. Then it would be just a cultural debate inside debian, rather than another pie in the sky dream.
<OriansJ>*_*
<OriansJ>^_^
<OriansJ>-_-
<OriansJ>^_^
<emorrp1>The idea of being able to compile everything in debian like it's gentoo starting from 250 bytes is enticing, but you'd need a lot more languages to reach that.
<vagrantc>emorrp1: i would love to see Debian get rebuilt; some packages are nearly a decade since they've last been built ...
<vagrantc>cultural debates in debian usually take at least 5-10 years :)
*vagrantc might be a person familiar enough with both guix and debian
<vagrantc>i've at least tricked both distros into letting me upload packages
<vagrantc>er, update packages ... upload doesn't quite make sense for guix (and kind of not for debian anymore either, thankfully)
<emorrp1>vagrantc: didn't holger essentially get the support for mass NMU at this year's debconf?
<vagrantc>emorrp1: debconf is never the right place to get decisions, though sometimes is a good starting point for a discussion
<emorrp1>well I could write up a summary for debian-devel, just searched and found no recent comments
<emorrp1>vagrantc: but I am a) pretty unknown and b) not a DD so can't actually action any resulting uploads
<emorrp1>sorry everyone else for the somewhat off-topic conversation