IRC channel logs

2019-12-13.log

back to list of logs

<fosslinux>oriansj: what's that?
<fosslinux>knight?
***Server sets mode: +cnt
<oriansj>fosslinux: knight is the endgame, no operating system, no microcode, no bios, no firmware; just pure bare metal bootstrapping on TTL with publicly available schematics
<oriansj>gio: somehow I remember discussing that video back in July when you did your talk. Also in weird and related news we actually now know where the first yogurt culture came from. (Fruit Flies)
<oriansj>(and microbe gene swapping with a lactose phylic baterial strain)
<oriansj>fosslinux: in your github issue, you indicated that the slow_lisp hanged. If you type say (+ 1 2 3) does it not echo the result? (I could tweak how the REPL looks and behaves to better match guile)
<fosslinux>yeah
<fosslinux>when running mes-m2 on mescc, it hangs
<fosslinux>mescc.scm
<oriansj>you mean running mescc on mes-m2?
<fosslinux>correct
<fosslinux>yeah that's what I mean
<fosslinux>I'm just stupid
<oriansj>fosslinux: no
<fosslinux>no what
<oriansj>fosslinux: inexperience in something that you haven't had the chance to properly is not an indicator of intelligence
<oriansj>fosslinux: you are reasonably intelligent person
<fosslinux>oh I mean I'm stupid because I said mes-m2 on mescc not the other way round
<fosslinux>what I mean is that
<oriansj>fosslinux: have you not seen the laundry list of typos I make constantly?
<oriansj>making mistakes is part about human
<fosslinux>when I run mescc using mes-m2 as the scheme interpreter it hangs
<fosslinux>oriansj: this is probably related to my strjpping out of things that don't work like modules
<oriansj>to make a mistake to normal, to refuse to admit one makes mistakes is however a sign of stupidity
<fosslinux>spot on
<oriansj>fosslinux: well hanging could be an indicator of not enough processing time (did CPU stick at 100% for one core?)
<fosslinux>I didn't look at that
<fosslinux>i will retry and look at that
<oriansj>also I still haven't figured out the true system needs of Mescc to self-host Mes.c
<oriansj>because it can take quite a while to build Mes.c using Mescc (you might even be farther along than me in getting Mescc running on slow_lisp)
<fosslinux>oriansj: I haven't tried using mescc to compile mes.c
<fosslinux>oriansj: I was trying to get mescc running on slow_lisp rather, as you said after
<oriansj>fosslinux: ultimately we need to go through mes_builtins in mes_builtins.c and figure out which of those primitives are absolutely required to be in the C code and which we can just punt into the scheme.
<fosslinux>I honestly need to start learning scheme to help here
<fosslinux>heh
<oriansj>well the good news is that the slow_lisp tests are great examples of scheme code to learn from
<fosslinux>yes
<fosslinux>I get operations
<fosslinux>and display
<fosslinux>I can do
<fosslinux>display (+ 1 2)
<fosslinux>yay
<fosslinux>:P
<oriansj>fosslinux: the key is that mes-m2 behavior needs to match guile; so that later we can be a drop in replacement for running guix too
<fosslinux>oriansj: i see.
<fosslinux>oriansj: what's the state of boot-0.scm?
<fosslinux>on slow_lisp
<oriansj>fosslinux: well, big chunks of it are going to be removed because they o longer are needed.
<fosslinux>oriansj: oh the hang is actually a REPL loop
<fosslinux>but why would
<fosslinux>MES_CORE=0 bin/mes-m2 -f mes/module/mescc.scm result in a scheme repl loop?
<oriansj>detection of TTL needs to be figured out at an M2-Planet level; otherwise I'd have to seperate out that behavior
<oriansj>fosslinux: mes-m2 defaults to a REPL unless there is an exit in the code
<fosslinux>ohh, ok
<oriansj>So one could make a file foo, fill it with a bunch of scheme to play with and do ./bin/mes-m2 -f foo and they then in play in that code
<oriansj>but if you do: (primitive-load "/dev/stdin") (exit 42) it'll go to the REPL and when you ctrl-d to terminate input, it'll then return to the exit 42
<oriansj>(exit) will always terminate no matter how much you nest loads
<gio>I came up with the four stages metaphor because I realized that it is hard to convey what different bootstrapping projects are about. There is a quite tangled mess of different projects, and I believed that without some images it would be difficult to explain their relationships.
<vagrantc>janneke: very cool regarding tcc mes and mes-guile-mescc :)
<janneke>vagrantc: you got me thinking; i love how that works :)
<vagrantc>i'm all about delegation :)
<civodul>hey gio, that was a good idea
<civodul>oh, mes-guile-mescc sounds fun :-)
<vagrantc>janneke: the weakest point of doing "true" diverse double-compiling is that probably our tcc and our guile hae been built with gcc somewhere in it's lineage
<vagrantc>at least on debian that's the case ... maybe guix is more exciting :)
<vagrantc>which, from my re-reading of dwheeler's paper, it talks about ideals ... and i thnk we can fall short of ideals while still making reasonably strong assertions
<vagrantc>the fact that mes itself was written in very recent history is a big plus
<civodul>at the end of the day, it's all probabilistic though
<janneke>civodul: i asserted that guile+mescc -> mes gives us also the bit-for-bit same mes
<vagrantc>if we can get the microsoft folks building mes, we've got a very strong case though :)
<janneke>which "only" means that mes is a decent guile implementation
<janneke>as far as mescc is concerned ;)
<vagrantc>janneke: the naming convention is starting to get curious with the guile+mes variant :)
<vagrantc>yeah, it's all about probabilities ... the *more* diverse your compile environment, the greater the probability is it's fine
<vagrantc>presuming matching hashes, etc.
<civodul>"the greater the probability", though that probability cannot be estimated :-)
<vagrantc>we are infinitely approximate
<civodul>heheh
<vagrantc>janneke: so if i add tcc to build-depends with your latest patches, will that also generate mes-tcc ?
<vagrantc>and looks like i could start experimenting with adding --with-courage for a few architectures, too ...
<janneke>vagrantc: hah, no -- it's either tcc or gcc; and configuring for tcc needs a bit of extra handholding
*vagrantc is really curious to see how far Debian GNU/kFreeBSD would get :)
<janneke>vagrantc: didn't i mail you the configure curse to do so
<vagrantc>janneke: yes, you did ... i was just getting too speculative without looking at what i had available :)
<janneke>ah, right -- why not :)
<janneke>vagrantc: is there any good reason for me not to order a pinebook64 right now?
<vagrantc>it's just a different configure argument, though ... no further handholding?
<vagrantc>janneke: pinebook pro?
<janneke>ah, yes the 4gb pro
<janneke>vagrantc: promised, no further handholding, completely bug-free!
<vagrantc>janneke: i'll have better first-hand information in a few days :)
<vagrantc>janneke: hah!
<janneke>aka "it worked for me"
<janneke>;)
<vagrantc>janneke: but in theory the pinebook pro would allow you to test both aarch64 and armhf/armv7
<janneke>vagrantc: exactly; better help dannym with the port, and help myself/the world move away from intel addiction
<janneke>now if they would mount an e-ink display on arm, 3wk of battery life...
<vagrantc>heh
<vagrantc>looks like mescc-tools will need support for hurd-i386 and kfreebsd-i386 and kfreebsd-amd64 before i can dream of it building mes
<vagrantc>janneke: i'm somewhat regressing the MES vs. Mes that i seem to have accidentally convinced you of :)
<janneke>vagrantc: good :) any particular insight you want to share?
<vagrantc>janneke: too many capital letters is hard on the eyes ... but i can live with either way
<vagrantc>s,regressing,regretting,g
<janneke>vagrantc: mescc-tools "just builds" on hurd, possibly not all tests pass
<janneke>vagrantc: you had me well-enough convinced to draft a patch
<janneke>having it reduced to a simple decision sometimes makes it easier, i'm not convinced either way
<janneke>i do think that it used to be G.U.I.L.E. -> GUILE -> Guile; and having seen such patterns made me go for Mes directly (although i seldom do shortcuts :)
<vagrantc>janneke: there are some things worth skipping the bootstrap
<janneke>vagrantc: note that on hurd, "only" mescc -S will work, no forking to run M1 or hex2
<janneke>and on freebsd...well, nothing much works yet except for configure and 5 initial ELF scaffold tests
<vagrantc> https://buildd.debian.org/status/package.php?p=mescc%2dtools some issue with a sha256 tool?
<janneke>omit the check/test phase on those (or fix sha256sum? :-D
<janneke>hmm, weird; mescc-tools has a script for that
*janneke goes afk for a bit
<oriansj>vagrantc: well sha256.sh depends upon get_machine --OS to return the correct result. So if you can get get_machine to return the correct OS name, the script should only run the sha256 form required for a particular host.
<vagrantc>oriansj: yeah, looking at that now
<vagrantc>apparently, on Debian GNU/Hurd it reports "GNU"
<vagrantc>debian kfreebsd will be more interesting :)
<vagrantc>er, debian gnu/kfreebsd
<oriansj>well as I have not been testing on those targets, it'll be educational
<vagrantc>i suspect both hurd and debian gnu/kfreebsd will be the same as Linux is currently
<vagrantc>since they all use sha256sum from GNU coreutils
<oriansj>well get_machine just does uname(unameData) and grabs the field with the desired information
<vagrantc>gnu/kfreebsd will require checking for something else
<vagrantc>what about instead of checking for the running kernel you check which utility is available, since they appear to be different for the three currently supported so far?
<oriansj>and the M2-Planet way of doing that is: https://paste.debian.net/1120900/
<oriansj>what would be the standard alternate command for kfreebsd to get the utsname?
<vagrantc>so, debian gnu/kfreebsd is the freebsd kernel with the GNU userland
<vagrantc>it's a bit of a weird beast
<oriansj>vagrantc: I can deal with weird if it is consistent
<vagrantc>the kernel isn't a great indicated of what userspace tools are available
<vagrantc>indicator
<vagrantc>for sha256sum, sha, and sha256, fortunately, the tools all have different names
<vagrantc>so in that case, just checking for which tools are present would be better
<vagrantc>presuming mescc-tools can assume they're present
<vagrantc>things like tar are a bit more inconsistant
<oriansj>uname is a syscall
<vagrantc>and it may not always be the right system call to determine which checksum tool to use, is all i'm saying
<vagrantc>it's asking the wrong question
<civodul>+1, i'd recommend trying to execute "sha256sum" and "sha256"
<vagrantc>and "sha" :)
<civodul>is that a thing?
<civodul>BSD userland has "sha256" IIRC, no?
<civodul>but anyway, something like that :-)
<vagrantc>apparently, there's a statement for NetBSD that uses "sha"
<civodul>oh, fun, and that's SHA256?
<vagrantc>janneke: does the ./configure CC=tcc ... --with-courage ... generate bin/mes-gcc or bin/mes-tcc ?
<vagrantc>civodul: it takes an argument for which algorithm
<vagrantc>according to mescc-tools, anyways :)
<vagrantc>janneke: does the tcc build produce bin/mes-gcc ... or am i failing to get configure to work correctly?
<janneke>vagrantc: yes, it produces bin/mes-gcc
<janneke>"gcc" being the flavour of assembly used in lib/<cpu>-mes-<cc-flavour>
<janneke>you can double check the value of CC in config.sh
<vagrantc>CC=${CC-"/usr/bin/gcc"}
<vagrantc>which ... doesn't make it obvious to me :)
<vagrantc>but maybe it's already on to the mescc build?
<vagrantc>looks like it's using gcc from grepping through the build log more...
<janneke>Yeah, that's not good
<janneke>i have: CC=${CC-"/gnu/store/b86cbcp53pndg3b3jl4m2zhy425asknd-profile/bin/tcc"}
<janneke>you need a recent `wip' for this
<vagrantc>does it need the full path?
<vagrantc>e.g. /usr/bin/tcc vs. tcc ?
<janneke>vagrantc: no, that's what configure puts in after i do
<janneke>./configure CC=tcc --with-courage --host=i686-unknown-linux-gnu
<vagrantc>excellent feedback "Applying courage" :)
*vagrantc uses the shortgun approach to export the variable
<vagrantc>janneke: i've got everything except the Mes -> MES patch :)
<janneke>vagrantc: good!
<vagrantc>seems to fall back to cc instead
<janneke>for me, configure says:
<janneke>checking for cc [0]... no (several times)
<janneke>checking for tcc [0]... 0.9.27
<janneke>yeah, i feared that much; does adding `--verbose' to configure give some insight?
<vagrantc>maybe i need to install libtcc-dev
<vagrantc>no dice, it still decides to use cc
<vagrantc>checking for cc [0]... command[256]: "/usr/bin/tcc --version" => [tcc: error: invalid option -- '--version'
<vagrantc>that looks curious
<vagrantc>and later: checking for cc is Tiny CC....config.c:2:2: error: #error no tinycc
<vagrantc>and checking for cc is GNU C... yes
<vagrantc>it's entirely possible i've failed to pass CC to it...
<vagrantc>still find makefiles a bit odd.
<vagrantc>curious ... now config.sh has: CC=${CC-"/usr/bin/cc"}
<vagrantc>as opposed to gcc
<janneke>that's weird
<janneke>if someone could devise a great way for people to interactively enter build scripts, that would be nice
<janneke>vagrantc: after the `--version' it should also say something like:
<janneke>checking for tcc [0]... command[0]: "tcc -v" => tcc version 0.9.27 (i386 Linux)
<janneke>vagrantc: ah...i think i found it
<janneke>i see this line just before it finds tcc:
<janneke>checking for cc [0]... command[32512]: "cc --version" => [/gnu/store/29jhbbg1hf557x8j53f9sxd9imlmf02a-bash-minimal-5.0.7/bin/bash: cc: command not found
<janneke>
<janneke>and i am guessing that debian has /usr/bin/cc, which then wins
<janneke>we should probably check for tcc first, does that make sense?
<vagrantc>yes
<janneke>or actually, check for env var CC first, before attempting `cc' or 'gcc'
<vagrantc>yes, even better :)
<janneke>those silly Guix people, they can check for anything they like because its never there anyway
<vagrantc>heh
<vagrantc>yes, this is definitely making me envious of being able to have a more minimal build system
<vagrantc>or minimal base dependencies
*vagrantc tries a quick-and-dirty patch putting tcc higher up in the list...
<janneke>good, i was going to say: this needs a bit of attention/time for me to do right
<janneke>hmm, changing the order might not be "right" but is certainly "better" and should work
<vagrantc>or is it ... ("@CC@" . ,(or (file-name "cc" deps) (file-name "tcc" deps) "")) ?
<vagrantc>i want to get that hash working :)
<janneke>ah yes, surely that!
<janneke>this could really do with some code removal and sanitation
<vagrantc>looks like i should rearrange that instead
<janneke>i even added "TCC" i see now, that does not make any sense
<vagrantc>hrm. rearranging the @CC@ line didn't work either
<vagrantc>by passing CC=tcc to the build target it get it to fail to build ... which is a form of progress :)
<vagrantc>still embeds the fallback in config.sh ...
*vagrantc throws some more paint around
*vagrantc goes back to just trying the lastest patches with what previously worked :)
<janneke>vagrantc: thanks, i will fire up a debian vm and have a look at this mess
<janneke>very grateful for you having a go at this, ty
<vagrantc>very grateful to have something to poke at :)
***ng0_ is now known as ng0
<vagrantc>no regressions for me with the latest wip patches
<janneke>\o/
*janneke just got configure to come up with: CC=${CC-"/home/janneke/bin/cc"}
<janneke>disregarding my tcc preference
<vagrantc>yay! sort of.
<vagrantc>:)
<janneke>vagrantc: `wip' now has a patch with a configure CC fix
<janneke>civodul: i wrote some bootstrapping in reproducible-build-summit-2019.md
<civodul>janneke: awesome, thank you!
<janneke>i was hoping if ae_ could chime in; did you mail/ping him before?
<civodul>i think ae was one of the recipients
<civodul>also cbaines (hi! :-))
<civodul>maybe i'll ping again over the week-end
<janneke>yeah, it's very much my perspective, then there was this other guix committer, eh vag... but he might be wearing another hat for this report
<janneke>;)
<vagrantc>janneke: yay! will take it for a spin.
<civodul>ah ha! gotcha!
<civodul>:-)
<vagrantc>i'll look it over :)
<civodul>cool, thanks!
<civodul>also, i hope i didn't write anything obviously wrong
<janneke>vagrantc: that would be great, (2x)
<janneke>i tried to avoid announcing any 9*fb ddc thing, yet have a cliff-hanger of sorts
<vagrantc>ooh, i had crazy-looking ideas :)
<janneke>now you know!
<civodul>didn't you? :-)
<civodul>janneke: a teaser is fine!
<vagrantc>it seems to recognize CC=tcc now, but it seems like it skips that build and leaps straight ahead into the bootstrap build...
<vagrantc>very quickly does: /<<PKGBUILDDIR>>/scripts/mescc: line 60: /<<PKGBUILDDIR>>/bin/mes: No such file or directory
<vagrantc>ah...
<vagrantc>cp: cannot stat 'bin/mes-mescc': No such file or directory
<vagrantc>it looks like it's trying to bootstrap mes-mescc with mes-mescc...
<janneke>ah, hmm -- iwbn if that would also work...but that's a different story; didn't look into that now
*vagrantc tries a conventional gcc build and it seems to be working
<vagrantc>doesn't break the gcc build, at least :)
*janneke cheers, somewhat
<oriansj>vagrantc: you are probably right
<oriansj>is something like [ -e /usr/bin/sha256sum ] portable enough for the BSDs?
<ng0>no
<ng0>for some
<ng0>FreeBSD has sha256sum(1), others don't
<ng0>what do you want to test?
<fosslinux>janneke, vagrantc: I would be interested in helping with aarch64/armv7l, I have 2 pi's which can run both
<ng0>freebsd 11.0: sha256 $FILE. other versions(?): shasum -a 256. NetBSD: sum -a SHA256 $FILE. I'm not aware how others implement it, but referneces are online
<oriansj>ng0: essentially do [ -e $PATH ] for all of the tools for doing sha256 sums and just walk down that if elif else list
<ng0>ah
<ng0>shasum can also be a perl tool, I'd put that last, or something
<ng0>at least for me it ships with perl
<oriansj>ng0: well I still need to know the paths for the BSDs, so that I can start in the right direction
<ng0> https://man.openbsd.org/sha256.1
<ng0>beats me. i can only tell you about netbsd right now: /usr/bin/sum
<ng0>unless OpenBSD diverged too much, they might still have cksum?
<oriansj>ng0: well hopefully someone on those platforms gets annoyed by it not working and submits a pull request or patch
<ng0>that's how it usually goes, yes
<ng0>ah, openbsd still has cksum.
<ng0>it seems like a wonky bet:
<oriansj>we probably should also check for the guix sha256sum tool in guile first; janneke/civodul how to go about detecting that and using it?
<ng0>The cksum utility is compliant with the IEEE Std 1003.1-2008 (“POSIX.1”) specification.
<ng0>All the flags are extensions to that specification.
<oriansj>ok, potential solution patch is up (now to wait for people to complain)
<janneke>fosslinux: that's great. aarch64 is still a bit out of reach, but armv7l is supported by mescc-tools and you could check out the wip-arm and wip-arm2 branches on mes
<janneke>if you do, you're in for a bumpy ride, wip-arm* needs some serious development still
<janneke>dannym will start to work on mes wip-arm some time soon
<oriansj>janneke: actually aarch64 is already supported in mescc-tools
<oriansj>(it was part of the work to help dddddd add aarch64 support to M2-Planet)
<janneke>oriansj: ooh, that's just great!
<oriansj>janneke: also because of the cross-platform work in M2-Planet mes-m2's behavior will be host neutral (I just need to shave the memory usage down a bit)
<civodul>janneke: great writeup for the r-b report (both the tone & content), thanks a lot!
<janneke>oriansj: yes, that's beautiful
<janneke>civodul: okay, thanks. i was mainly wondering what non-mes'y things or people i forgot
*janneke -> zZzz