IRC channel logs

***terpri__ is now known as terpri

<OriansJ>Hagfish: well getting people on board with bootstrapping is about adopting cultural values that few outside of the Free Software culture have managed to preserve at great cost.

<OriansJ>Even inside the Free Software community, it does take considerable work to convince people to put forth the effort required to keep bootstrapping possible.

<pabs3>its hard enough to convince Free Software people not to store build artefacts in .git :(

<OriansJ>also congrats janneke on the announcement.

<OriansJ>pabs3: or depend upon build artifacts to build their software (guile psyntax anyone?)

<OriansJ>but that is why mes-m2 is such a vital piece of the bootstrap process for guix

<OriansJ>The problem is it is really hard to find C programmers who are also good Lisp programmers, who are willing to work together to get this done correctly.

<OriansJ>As it is basically the same sort of task that RMS had to do with GCC to build Emacs but we are attempting to build something even bigger using M2-Planet.

<OriansJ>It has never been done before and I can see why. Syntax-case is a bitch to implement even in scheme and we have to do it in a C subset that doesn't even support C macros or floating point operations.

<OriansJ>It would be another story if the Guile devels hopped on and didn't require syntax-case to bootstrap their version of syntax-case but we are not that lucky

<OriansJ>who knows maybe the solution is to get RMS out of retirement and onto finishing mes-m2 for us. Then we could be done this year

<Hagfish>not requiring syntax-case to bootstrap syntax-case seems like a good balance between amount of work, amount of benefit, and correctness of the solution

<Hagfish>i'm not trying to convince you of that, i'm just explaining my disappointment at the Guile devs for not prioritising that work

<OriansJ>Hagfish: honestly, I've come to accept that we just can't depend upon others to help us save them.

<OriansJ>Thus far no one from the Haskell community is working on bootstrapping (I would love to be proved wrong here)

<OriansJ>Thus far I am the only person who is working on the scheme bootstrapping problem.

<sbp>hey, I'm having trouble following the current Guix bootstrap story

<sbp>is there a canonical place to learn about this?

<sbp>e.g. I'm looking at the discrepancies between https://guix.gnu.org/static/blog/img/gcc-core-mesboot0-graph.svg and http://bootstrappable.org/images/gcc-mesboot0.png and the documentation on bootstrappable.org and guix.gnu.org in general

<sbp>I don't understand even very simple things, such as why mes (Scheme <-> C mutual self-hosting) is being used as a dependency of tcc (C only, self-hosting)

<sbp>surely more complex things should depend on less complex things

<sbp>and I don't understand how you intend to bridge the gap between stage0 and, I suppose, mes. I found https://lists.gnu.org/archive/html/guile-user/2017-03/msg00220.html which essentially just says that the work is yet to be done, but it's an old post

<sbp>but I mean e.g. there are FORTH, LISP, and C experiments in stage0 and it's not clear which of them would be used in bootstrapping further towards mes. in the 2017-03 post, Jeremiah even speculates that using something like PL/0 might be helpful

<sbp>hi OriansJ`

<OriansJ`>hello sbp

<OriansJ`>to answer your questions

<OriansJ`>here is the current bootstrap map: https://github.com/oriansj/talk-notes/blob/master/Current%20bootstrap%20map.pdf

<sbp>oh thanks, this looks like exactly what I needed!

<OriansJ`>the reason for mes.c <-> MesCC is to simplify the bootstrapping problem into large chunks to allow independent people to make progress collectively

<sbp>I think I saw some mention of that somewhere, like the idea of splitting out a C preprocessor that can be implemented in LISP (the stage0 kind, I think it was, in whatever I was reading. I've been reading a lot of stuff, losing track)

<OriansJ`>as writing a C compiler in scheme capable of bootstrapping GCC is much easier than writing a C compiler in assembly capable of bootstrapping GCC

<sbp>oh it was in 2017-03 post, heh

<sbp>yeah, but I suppose you wouldn't have to write a C compiler in assembly. you'd take an existing C compiler, perhaps one of the many "subset of C" compilers, and use that to compile itself. then you'd audit the binary that it produces by decompiling it

<OriansJ`>Guix reduces down to just Guile (48MB of binary) and stage0 grows up to mes-m2; which reduces it down to just 256Bytes (or 89 bytes for the DOS port)

<sbp>that's what I don't understand - why that path wasn't taken

<sbp>well, one of the *many* things I don't understand ;)

<OriansJ`>sbp: I did write a C compiler in assembly https://github.com/oriansj/mescc-tools-seed/blob/master/x86/NASM/cc_x86.S

<OriansJ`>but I have no fantasy that it'll ever grow to the level required to bootstrap GCC.

<sbp>okay, but you'd written that directly in assembly

<sbp>I was thinking rather that you'd use a small C compiler on itself

<sbp>then disassemble the binary

<sbp>and then audit the disassembled binary against the small C compiler source

<OriansJ`>sbp: why bother?

<sbp>wouldn't it be easier?

<OriansJ`>we already used our hand written C compiler to bootstrap a simple cross-platform C compiler https://github.com/oriansj/M2-Planet

<sbp>the benefit is that you wouldn't have to write the assembly, only audit it

<OriansJ`>which is written in the C subset our assembly C compiler provides

<OriansJ`>sbp: writing assembly is fun

<sbp>I agree. I was just thinking about speeding development

<sbp>but it does seem from Current%20bootstrap%20map.pdf that you're very nearly done now

<sbp>there are only a few red paths

<OriansJ`>sbp: essentially finishing slow-utils in Scheme and mes-m2 in the C subset are basically the last steps

<OriansJ`>all that really remains is boring, slow work in high level languages

<sbp>which of those two are you working on? both?

<sbp>I see https://github.com/oriansj/mes-m2 was last committed to in March

<sbp>and https://github.com/oriansj/slow-utils in January

<OriansJ`>my 3 month old son

<sbp>hehe, aww

<sbp>congrats!

<sbp>apart from janneke, are there any other primary developers? I don't see much in the way of contributions from people who aren't you, which is surprising for such an excellent and important project. I'm also surprised at the comments in https://guix.gnu.org/blog/2019/guix-reduces-bootstrap-seed-by-50/ about needing to push other GNU project people into supporting this

<sbp>bootstrapping and reproducible builds are Step Zero in my mind

<sbp>ah, I see you mentioned this yourself earlier today. http://logs.guix.gnu.org/bootstrappable/2020-06-16.log#021152

<OriansJ`>sbp: only a few people actively contribute as bootstrapping requires hard work and actual ability to program.

<sbp>you mentioned nobody bootstrapping Haskell. I saw https://elephly.net/posts/2017-01-09-bootstrapping-haskell-part-1.html a while ago, and I assume you've also seen it - no doubt you meant nobody is trying to make Haskell (GHC) bootstrappable as a finished, repeatable toolchain

<OriansJ`>well the person behind that post is rekado and after that he focused on getting java bootstrapped

<sbp>ha. didn't know that

<OriansJ`>Things like bootstrapping Rust, Go and javascript end up being higher priorities than Haskell because they are more commonly used in software

<OriansJ`>some of it is just untangling the build chain for thousands of packages

<OriansJ`>far from a fun or easy job but absolutely essential

<janneke>sbp: samplet has written more than half of gash, dannym is writing the arm port of mes

<janneke>fwe are still reaching out, it needs some time, we need others to help mature this

<OriansJ`>janneke and I just kinda lucked into the more impressive looking pieces where as the guix package work is far larger and in someways more important

<janneke>yeah

*janneke needs to go afk for a bit!

<sbp>thanks for the info, janneke!

<sbp>I wonder if it'd be possible to bring some of the Guile devs in to help at least?

<sbp>this seems exactly the sort of thing Wingo would be interested in helping with

<sbp>even if just on his end, making Guile more amenable to bootstrapping

<sbp>I see civodul is here, so I'm sure the links exist already and you've already tried

<OriansJ`>sbp: some problems just seem so big people just give up

<sbp>makes sense. I'm very guilty of that myself

<OriansJ`>It takes alot to keep hammering on a problem for years and years with no external support and lots of people bitching along the way

<OriansJ`>I mean we have had people leave over minor differences in goal, arguments of where to host the master, etc

<sbp>ugh, heh. I assume the hosting the master argument was about Savannah vs GitHub

<sbp>I've had that argument myself before

<OriansJ`>yep and then they left

<OriansJ`>I honestly don't care, I just need people to help. SO I don't end up responsible for everything.

<OriansJ`>*be back later*

<sbp>thanks very much for the chat, OriansJ`

<civodul>hi!

<civodul>Guile is actually "boostrapped" in the sense that it has an interpreter in C that is used when building the compile

<civodul>*compiler

<civodul>*bootstrapped

*civodul must be using a broken keyboard

<sbp>civodul: do Wingo and the other Guile devs know about OriansJ's work?

<sbp>greetings by the way, long-time admirer of your work

<civodul>i can't speak for wingo, but we've definitely discussed bootstrapping a number of times

<civodul>Guile was "bootstrappable" from the start (2.0)

<civodul>except for psyntax

<civodul>janneke knows that very well :-)

<civodul>rekado used Guile as an example when first writing https://bootstrappable.org/

<civodul> https://bootstrappable.org/best-practises.html

<civodul>someone™ should update the web site to mention the latest achievements

<sbp>psyntax was what made me wonder (OriansJ mentioned it earlier)

<jelle>janneke: nice blogpost https://guix.gnu.org/blog/2020/guix-further-reduces-bootstrap-seed-to-25/ :)

<markjenkinsznc>OriansJ Re: "Thus far I am the only person who is working on the scheme bootstrapping problem." True in the important public sense. Just wanted to share that I'm quietly working behind the scenes on a private project which is building up my understanding of scheme and tackling an experimental idea which may result in code that can bridge mes and mes-m2 as it stands and also provide more test cases for mes-m2

<markjenkinsznc>Could also be a false path that better educates me along the way.

<markjenkinsznc>It's clear people have had enough of ideas and no working code, so I'll only take this public and go into collaberation mode if and when I have a prototype written in a high level, non-bootstrapped language that illustrates viability. Then I'll rewrite in a bootstrapped language. I recognize that supporting the different kinds of macros used by mescc, especially syntax case is the hardest part of the problem here so I'm going to make sure I've

<markjenkinsznc>tackled that before going public

<markjenkinsznc>Worked in two surges so far, 26 commits in February and currently ongoing 32 master commits (+ squashed wip branches) since start of May. "just a hobby, won't be big and professional like gnu"

<dannym>janneke: (abs -2147483648) in ./bin/mes results in -2147483648. No :P

<janneke>dannym: oops :-)

<janneke>jelle: thanks!

<xentrac>markjenkinsznc: awesome!

<dannym>janneke: I guess it's ok to have weird special cases like this if we don't have arbitrary-precision integers implemented (maybe a waste of time), but we could have math.c minus check at least for this awful case.

<dannym>janneke: I don't know of a portable way to check the general cases for overflow in C, though

<dannym>janneke: Worse, C does not specify what happens when a signed long overflows (i.e. it assumes that that cannot be intentional and optimizes away ruthlessly all your branches where it would have to happen)

<janneke>dannym: i agree; not handling this special/interesting case is asking for trouble

<janneke>it's a surprising bug, in a way -- meaning that it surprises me that we're able to build a functional software stack with this bug in place

<dannym>janneke: (Just spent almost a day finding a bug in the ARM backend that basically was because the backend assumed that (abs x) >= 0)

<janneke>right!

<janneke>that doesn't surprise me at all, and that's terrible!

<dannym>Basically I'd suggest this stop-gap measure:

<dannym>SCM

<dannym>minus (SCM x) ///((name . "-") (arity . n))

<dannym>{

<dannym> assert_number ("minus", CAR (x));

<dannym> long n = VALUE (CAR (x));

<dannym> x = cdr (x);

<dannym> if (x == cell_nil)

<dannym> {

<dannym> if (n == LONG_MIN)

<dannym> error (cstring_to_symbol ("overflow"), n);

<dannym> else

<dannym> n = -n;

<dannym> }

<dannym> else while (x != cell_nil)

<dannym> {

<dannym> assert_number ("minus", CAR (x));

<dannym> n -= VALUE (car (x));

<dannym> x = cdr (x);

<dannym> }

<dannym> return MAKE_NUMBER (n);

<dannym>}

<dannym>That doesn't catch (- 0 INT_MIN) though (etc).

<xentrac>well, some C implementations optimize it away ruthlessly, yes

<janneke>dannym: hmm...that's...

<janneke>dannym: can't we do it from scheme, in define abs?

<janneke>oh, but that won't solve (- LONG_MIN) ... right

<janneke>hmm

<janneke>okay, if you think this is better than (if (= x LONG_MIN) ... ...) in (define (abs), then let's do it

<dannym>janneke: I've thought about putting it in abs, might be better; otherwise minus is kinda pretentious in overflow checking

<dannym>would be, with the above

***tazjin is now known as JSFzKHVpbnQ4PTEx

***JSFzKHVpbnQ4PTEx is now known as tazjin

<janneke>dannym: okay

<dannym>janneke: Another thing, can I use define-syntax in the backend?

<dannym>I've tried:

<dannym>;; If 0 <= exp < #x100, use positive-body.

<dannym>;; If #x-100 < exp < 0, use negative-body.

<dannym>;; Otherwise, use general-body.

<dannym>(define-syntax optimize-zero-page

<dannym> (syntax-rules ()

<dannym> ((optimize-zero-page exp positive-body negative-body general-body)

<dannym> (let ((temp exp))

<dannym> (if (>= temp 0)

<dannym> (if (< temp 256)

<dannym> positive-body

<dannym> general-body)

<dannym> (if (> temp -256)

<dannym> negative-body

<dannym> general-body))))))

<dannym>And then for example

<dannym>(define (immediate->r0 v)

<dannym> (optimize-zero-page v

<dannym> `(((#:immediate1 ,v) "mov____$i8,%r0"))

<dannym> `(((#:immediate1 ,(- -1 v)) "mvn____%r0,$i8"))

<dannym> `(("mov____$i32,%r0" (#:immediate ,v))))))

<dannym>But "define-syntax" is undefined.

*xentrac is undefined

<xentrac>janneke_: did you see dannym's code for you above?

<janneke_>xentrac: yes, thanks

<janneke_>oh wait, maybe not

<janneke_>i was away for a minute or two

***janneke_ is now known as janneke

*janneke checks the logs

<janneke>dannym: define-syntax...hmm

<janneke>we're using mes/module/mes/pmatch.scm that's defined with syntax-rules

<janneke>(we don't have syntax-case)

<janneke>but our syntax-rules is not 100% correct -- no idea what the problem is

<janneke>i've been avoiding it and use define-macro whenever possible

<dannym>janneke: Yeah, I've also thought about define-macro, but it could evaluate exp three times

<dannym>janneke: In the cases I use it now that doesn't matter because exp is just the name of a local variable

<dannym>But that could change

<dannym>And future-me would hate me :)

<janneke>maybe i've been over-cautious

<dannym>;; If 0 <= exp < #x100, use positive-body.

<dannym>;; If #x-100 <= exp < 0, use negative-body.

<dannym>;; Otherwise, use general-body.

<dannym>(define-macro (optimize-zero-page exp

<dannym> positive-body negative-body general-body)

<dannym> (list 'if ('< exp 0)

<dannym> (list 'if ('>= exp #x-100)

<dannym> negative-body

<dannym> general-body)

<dannym> (list 'if ('< exp #x100)

<dannym> positive-body

<dannym> general-body)))

<dannym>(optimize-zero-page 2 'one 'two 'three)

<dannym>one

<dannym>does work...

<janneke>yeah...i think that's what i would do

<dannym>mes> (optimize-zero-page (begin (display "\nhaha") 2) 'one 'two 'three)

<dannym>[sexp=(optimize-zero-page (begin (display

<dannym>haha) 2) (quote one) (quote two) (quote three))]

<dannym>less_pexception:not-a-number:((begin (display

<dannym>haha) 2))

<janneke>great!

<dannym>Not the last part ;)

<dannym>Not that I need that part right now, but still...

<dannym>This does work though:

<dannym>(if (< 5 (begin (display "\nhaha\n") 3)) 'yes 'no)

<dannym>[sexp=(if (< 5 (begin (display

<dannym>haha

<dannym>) 3)) (quote yes) (quote no))]

<dannym>haha

<dannym>$0 = no

<dannym>I've tried (define-macro ... ((lambda (temp-name) (list 'let (list (list temp-name .... )))....) (gensym)) but that reminds me too much of the bad old time back in the day ;)

<janneke>dannym: what about

<janneke>(define-macro (optimize-zero-page exp

<janneke> positive-body negative-body general-body)

<janneke> `(if (< ,exp 0)

<janneke> (if (>= ,exp #x-100)

<janneke> ,negative-body

<janneke> ,general-body)

<janneke> (if (< ,exp #x100)

<janneke> ,positive-body

<janneke> ,general-body)))

<janneke>(untested)

<dannym>Yeah, more readable that way :)

<janneke>avoiding quasiquote is only necessary during boot

<janneke>*booting mes' scheme modules

<dannym>(define-macro (optimize-zero-page exp

<dannym> positive-body negative-body general-body)

<dannym> ((lambda (temp-name)

<dannym> `(let ((,temp-name ,exp))

<dannym> (if (,>= temp-name ,0)

<dannym> (if (,< temp-name ,#x100)

<dannym> positive-body

<dannym> general-body)

<dannym> (if (,> temp-name ,#x-100)

<dannym> negative-body

<dannym> general-body)))))

<dannym> (gensym))

<dannym>mes> (optimize-zero-page 2 'one 'two 'three)

<dannym>[sexp=(optimize-zero-page 2 (quote one) (quote two) (quote three))]

<dannym>exception:not-a-pair:((() . car))

<dannym>Aha, this does work:

<dannym>(define-macro (optimize-zero-page exp

<dannym> positive-body negative-body general-body)

<dannym> `(let ((exp ,exp))

<dannym> (if (,>= exp ,0)

<dannym> (if (,< exp ,#x100)

<dannym> ,positive-body

<dannym> ,general-body)

<dannym> (if (,> exp ,#x-100)

<dannym> ,negative-body

<dannym> ,general-body))))

<dannym>And shouldn't have a variable capture problem either

<janneke>\o/ -- the gensym was in the wrong spot

<xentrac>yay!

<janneke>better!

<dannym>janneke: All tests passed with the non-abs-using backend without macro, including the new ones I added :)

<dannym>(btw)

<dannym>janneke: mes master commit 341682f474c42f8cde8be5c8b80df4b58d31f7b9

<dannym>And now I can de-uglify it again by using the macro

<dannym>Thanks :)

<janneke>very happy, awesome work dannym!

<dannym>janneke: Thanks :)

<dannym>Even this works:

<dannym>(optimize-immediate 1000 'one 'two (error "no")) => exception: error "no"

<dannym>(optimize-immediate 1 'one 'two (error "no")) => one, no exception

<janneke>nice

*janneke -> zZzz

<dannym>janneke: good night :)

***ChanServ sets mode: +o rekado