<yt_>OriansJ: deesix: M2-Planet with #if #elif #else #endif works for all constant integer expressions; that's probably enough for today. I'll clean it up and push a pull request sometime tomorrow. Then on to #define FOO. #define FOO(a,b) is a whole different ball game so I probably won't get that far tomorrow. <deesix>yt_, Thanks. Watch out for &&, it won't short-circuit. <deesix>yt_, sorry, I wasn't clear enough. I mean in the code, not in the expressions you're parsing. Code like: <deesix>+ if (NULL != conditional_inclusion_top && !conditional_inclusion_top->include) <deesix>Anyway, that was fast and I'm loving your incremental commit style (: <deesix>yt_, for the clean up: purge_macro() has now uses now, it seems. <stikonas>futher bootstrap needs some more thoughts, since kaem doesn't support IO redirection. Either it has to be added, or try to avoid using it in blynn-compiler <deesix>yt_, the example in the commit message is incomplete, lines parsed as comments I guess. <fossy>stikonas: thanks for your PR! <fossy>pder: said they would make it so it does not need io redirection <fossy>I want to avoid the need for pipe support in the bootstrap kernel <stikonas>fossy: then maybe we need to do something in blynn-compiler... <stikonas>although, it's bed time for me now, so maybe can discuss it some other time <stikonas>in any case newer M2-Planet in that PR should be useful for wip-mes2 branch <pder>I can update the blynn-compiler build so the last 3 or 4 stages take file input and output arguments so we dont need the IO redirection <siraben>In fact it's possible to move all the shell stuff to blynn compiler since we have an FFI <pder>siraben, yeah I wondered if the cleanest way would be through parsing args in Haskell and extending the FFI so we can open an input and output file <pder>I can implement a simple solution in C for everything up to precisely, then we could update precisely to have a better command line interface <pder>siraben: Also I am interested in your ideas on how we could implement an includeFile feature so we could possibly automatically include a prelude <siraben>pder: we can do a statement like includeFile in haskell <siraben>see the wrappers for how those get turned into #include etc. using GHC's CPP expansion <siraben>so maintain ability to compile with GHC <siraben>Which reminds me, I'll need to fiddle around with FFI for file reading/writing in the generated C code <pder>I pushed a commit that builds precisely without using any shell io redirection. We will definitely want to revisit how we parse arguments in precisely <siraben>pder: you might want to rebase your commit on oriansj/master to make the CI build again <janneke>mescc-tools-seed produces m2-planet, which can be used to build the wip-m2 branch of mes <gforce_d11977>janneke: ok, but this "bin/mes-m2" has not yet the capabilities to compile the .scm file for producing mescc? <janneke>mescc is written in scheme, you can think of it as mes.c (or mes.m2) and mescc.scm <janneke>mes-m2 can run mescc.scm to produce itself (mes) and the mes c library <gforce_d11977>janneke: sorry, i mixed this up. thanks for explanation, i will try these steps <janneke>have i seen the wip-full-source-bootstrap branch in guix? <gforce_d11977>janneke: yes, i have the branch opened here, but ... still dont get it (give me some time...) <janneke>np, just good to not unknowingly miss info :) <gforce_d11977>janneke: after several attempt to run 'bin/mes-m2' with an *.scm file, i cannot get it to work: e.g.'Unbound symbol: define-module' - i see, that in theory i can run: guile --no-auto-compile -L module -e main scripts/mescc.scm -I include -S scaffold/exit-42.c -o foo.S <janneke>gforce_d11977: you can run ./pre-inst-env mescc <janneke>./configure or ./configure.sh creates scripts/mescc <malina>well, I guess it's good to observe this, since I still haven't downloaded guix and still don't know quite how scheme's work, although luckily they are very clean and easy to read. <gforce_d11977>janneke: i tried the './configure.sh' from 'mes-0.22-305-g2ab4c5c67' and the 'sh bootstrap.sh' afterwards. it works somehow, but is polluted with a lot of binaries inluding gcc and . so thats not the idea of bootstrapping 8-) <janneke>gforce_d11977: if you don't want gcc to be picked-up, you can use CC=mescc ./configure.sh <janneke>crt1.s is built as mescc-lib/crt1.s and mescc-lib/x86-mes/crt1.s <gforce_d11977>janneke: now i'am really lost: i have built bin/mes-m2 - how can i have 'mescc' (which is not build yet) to compile 'mes'? sorry for dumb questions, all the other 19 steps (from hex0) are clear and working, but now....? bootstrap.sh even calls 'perl' 8-) <janneke>perl is for help2man in the full build; that's probably not supported or needed in the bootstrap build <janneke>it's probably nice that configure.sh leaves the same OK state as the development ./configure does <janneke>gforce_d11977: do you have a bin/mes ? <janneke>what doet ./pre-inst-env mescc --help say? <stikonas>I actually get something like "unhandled exception: unbound-variable: (abort)" with mescc from wip-m2 branch (manually built with M2-Planet) <gforce_d11977>janneke: interesting, it searches for 'nyacc/lex.scm' - i have to install that... <gforce_d11977>janneke: wow, now it says: 'usage: ... \nC99 compiler in Scheme for bootstrapping the GNU system.' <gforce_d11977>janneke: now i really have to re-try what i have done in the last hour and think about it, why it worked 8-) <janneke>stikonas: you may try MES_DEBUG=2 ... and see if you get more info <janneke>stikonas: and a (hopefully not too-annoying) reminder that the full build sequence is documented/scripted now as guix packages in wip-full-source-bootstrap branch <stikonas>yeah, it's probably some shell variables pointing to the wrong places... I'll look into it a bit later <janneke>that's what i mean, some things have to be exactly right (versions, variables, etc) <janneke>but there's not too much required anymore in the way of manual scripting <janneke>building (the unpatched) tcc-boot and packages beyond becomes a bit trickier <civodul>janneke & everyone: congrats on bridging the gap to stage0! <malina>++ as I say, looks like I was lucky on the timing :D <janneke>civodul: thanks & expect a mail to guix-devel "any moment now" *janneke has been sayng that since yesterday night... <stikonas>well, testing that wip-full-source-bootstrap branch is still a bit tricky because it has gnutls-3.6.12 which fails to build these days... <malina>I am aso reminded that in the past 5-10 years, various projects have existed, we all approach it differently, and guix is a 'competitor' but then again, I never managed to get anyone really with any interest to join devops <malina>it even states the issue though; is openssl not installed? and/or did you try building with disablig tests, alternatively patching out the failing one? <malina>hell, even make check || true but that depends on I don't know if guix / schemes do something very odd or so <civodul>one can do --without-tests=gnutls as a workaround <malina>I mean, I hope whoever tried building 71 times didn't lack the audacity to actually change configuration <malina>ye, so even if a/some test(s) need some patching then, it's not exactly "gnutls is failing to build these days". the latter being a good enough argument for me to not really feel for trying guix <malina>[17:59:49] <civodul> one can do --without-tests=gnutls as a workaround <- sounds like something I would do, if it's time critical, then simply patch the test(s), pass it on to upstream and/or re-enable tests if that is desirable in recipe. <janneke>stikonas: i didn't pay much attention to guix development the past months, it's been a ride <janneke>however, wip-full-source-bootstrap must be rebased on core-updates anyway and i would like have to have a propper release of mes first ***lfam_ is now known as lfam
<rain1>so stage0 is now building all the way from hex up to prototype lisp? <janneke>yes, m2-planet can build gnu mes (wip-m2 branch) <janneke>and this has been integrated in the guix bootstrap <rain1>so i get stage0 and do make all but that is invoking gcc <rain1>is that just for the prototypes and i can ignore that <janneke>trying that in a development environment can be tricky, as most tools can also be built by gcc <janneke>as a start, we currently use the package "mescc-tools-seed" <stikonas>I briefly tried but got some crashes... Probably need to adjust some env variables <rain1>i don't have guix and want to build some things from scratch so I do this <stikonas>rain1: you probably want to start with mescc-tools-seed <Hagfish> ;; The initial bootstrap package: no binary inputs except for a 357 byte binary seed <stikonas>and make is not available in early bootstrap :) <stikonas>well, you still need POSIX kernel which is much more than 357 byte binary <stikonas>and some driver do drive the process (kaem-minimal is a bit under 1 KB) <rain1>stikonas: stage0 contains this repo under the name Linux Bootstrap doesn't it? <stikonas>then it runs kaem.run which recompiles kaem and hex0 seed from source <Hagfish>hmm, good point about the POSIX kernel. i thought tccboot was capable of building linux from source, as a bootloader without a (precompiled) kernel? <stikonas>ok, I guess that's what makefile runs for you... <Hagfish>sure, but that's smaller than linux :) <Hagfish>but yeah, it's kinda cheating if you trust tcc binary, and then go on to build "trusted" tcc <stikonas>it's either linux + hex0 or hex0 + tcc which is as good/bad as tcc on its own <stikonas>and tccboot runs in some software environent too (I guess UEFI) <Hagfish>would it be possible to adapt hex0 into a bootloader or something? (sorry if i'm not understanding these low level processes) <stikonas>well, OriansJ wants to build computer from TTL logic... <Hagfish>i wonder if he could convince someone who has already made such a hobbyist computer to run the bootstrapping process and share the output (hash?) with us <stikonas>maybe you can do something with hex0 to boot existing hardware... <stikonas>but with current hardware, I guess often you'll hit the problem that bootloder is fairly complex, e.g. on rk3399 which does not use any blogs for booting, bootloader (U-boot)still has to train RAM, and quite a few other drivers to initialize boot media <stikonas>and those things are not written in M2-Planet C... <Hagfish>yeah, it's a pity that hardware is compounding the software bootstrapping problem <rain1>so I have built stage0 and linux bootstrap <rain1>I have added the binaries from stage0 to my PATH <rain1>./configure --prefix=`pwd`/out/ <rain1>How can I configure it to use mescc? <rain1>do I need to build mescc as a standalone binary first? <janneke>try: CC=mescc ./configure --host=x86 <rain1>./Linux Bootstrap/mes-m2/mes/module/mescc <rain1>./Linux Bootstrap/mes-m2/module/mescc <rain1>./Linux Bootstrap/mes-m2/scripts/mescc <janneke>stikonas: bin/mes-m2 is built by ./bootstrap.sh, after running configure <rain1>CC=mescc ./configure --host=x86 --prefix=`pwd`/out <rain1>but it still says GNU Mes is configured for compiler: gcc <stikonas>I was trying to avoid .sh scripts since they need bash... <rain1>I think it didn't find mescc <janneke>then you may have gcc in your path; don't do that <janneke>hmm, /me goes to check this, though... <rain1>this was in the ./configure log <rain1>which package builds/provides the mescc binary? <janneke>mescc is a script, that is created by ./configure|configure.sh in scripts/ <stikonas>argh, I'm getting those unbound variable errors even from bootstrap.sh script <rain1>stikonas: ah I have this same error too <rain1>error:include-from-path: not found: :(nyacc/lex.scm) <stikonas>oh, this one you need to clone separately <rain1>lalr.mes lang lex.mes parse.mes util.mes version.mes <rain1>the thing is, it's not empty <stikonas>oh yeah, if you did recursive clone it should be there <janneke>you need to unpack nyacc-1.00.2 somewhere and set GUILE_LOAD_PATH=.../nyacc-1.00.2/module <janneke>it would be nice if configure.sh handled this! <rain1> +> ./bin/mes-m2 -c (display 'Hello,M2-mes!) (newline) <rain1>cp: cannot stat 'crt1.s': No such file or directory <rain1>this is looking good, i think it is running scheme code. but im not sure about the crt1 error yet <rain1>so I think that worked for nyacc <rain1>find . -iname crt1.s is giving no results <rain1>$CC -c $AM_CPPFLAGS $CPPFLAGS $AM_CFLAGS $CFLAGS ${srcdest}lib/$mes_kernel/$mes_cpu-mes-$compiler/crt1.c <rain1>i suppose this line was to generate that file <janneke>that's right, so compiling crt1.c failed <rain1>./configure.sh expects mescc to already exist <stikonas>and I guess you need to set some more variables like MES_PREFIX <janneke>ah, there's an old hack/workaround in the guix recipe; thanks <rain1>in config.sh I see CC=${CC-"[...]/mes/pre-inst-env mescc"} <janneke>configure.sh is supposed to set CC to ..exactly! <janneke>ah config; yes configure.sh should create that setting <rain1> CC lib/linux/x86-mes-mescc/_write.c *janneke removes obsolete workarounds from the guix recipe <stikonas>I guess you don't need to set GUILE_LOAD_PATH twice to different variables <stikonas>although, bash is not available there... <stikonas>so instead of running configure.sh I might need to do some things manually <janneke>ah yes, there is only a kaem.run script for the mes-m2 build <janneke>nothing to replace configure.sh, bootstrap.sh, install.sh <stikonas>let's see how well it works not that I sorted out other problems <stikonas>(I guess initially I also had wrong nyacc version) <stikonas>janneke: ok, so I think I only need to get mescc.scm.in -> mescc.scm step working in kaem <stikonas>although, this can be done in advance... <stikonas>while we still have bash outside bootstrap environment <janneke>stikonas: yes, or otherwise script around it (i.e., do what the relatively simple scripts/mescc.in script does) <stikonas>oh, actually .in file seems to run just fine... <stikonas>./bin/mes -e main ./scripts/mescc.scm.in <stikonas>fossy: had some other more verbose trick, I might try that one first... <stikonas>anyway, I have something to try now and try to get it running in live initrd environment <rain1>for building tinycc with mes-mescc, i have a problem that mes-mescc is not finding boot-0.scm. That file is at ../mes/mes/module/mes/boot-0.scm <rain1>is there a variable or some way to point mescc to that? <rain1>or did i miscompile in some way that it lost track of it <janneke>rain1: that's probably MES_PREFIX, but it should be correctly set in mes' scripts/mescc when you ran configure.sh in mes <janneke>rain1: did you install mes, by running install.sh? <janneke>for development, it can be run from the build tree, but that requires some trickery (setting MESCC, MES_LIB MES_SOURCE all just right) <janneke>they are settings for the garbage collector, <janneke>rain1: what is mes-mescc? this looks like mes itself <rain1>ahh I was invoking the totally wrong program! <rain1>so I actually need to use ../mes/out/bin/mescc <janneke>mes is the scheme interpreter, and it doesn't know 'int' :-) <rain1>brilliant, mescc is compiling hw.c <stikonas>hmm, I have a bit less luck in fossy's initrd environment... <stikonas>/after/bin/mes --no-auto-compile -e main /after/bin/mescc.scm -- -c -I include -I include/linux/x86 lib/linux/x86-mes-mescc/crt1.c <stikonas>oh, maybe because host is not set correctly... <stikonas>but tinycc is being compiled outside live environment here, so that's good *janneke cherry-picks `abort' from the wip-module branch <fossy>stikonas: hmmmm, that should work fine <stikonas>fossy: it will, just need to fix some dirs, but dinner first <stikonas>I think nyacc was pointing to the wrong dir <fossy>stikonas: inside the live environment, nyacc's modules are already inside the mes module dir <fossy>Although that might still be in my local tree :x <stikonas>well, I switched to mes-m2 branch, so it's probably not in my tree <stikonas>fossy: so for now I'm seeing errors like this unhandled exception: error: (("mescc: file not found: \"x86-mes/x86.M1\"")) have you seen then before? <stikonas>probably something is missing in my mes.kaem... <fossy>stikonas: there is a missing link i believe <stikonas>yeah, I thimk there is, rootfs.sh creates it <fossy>can you run file sysa/tmp/after/mes-0.22/x86-mes ? <fossy>from outside the qemu env, ofc <stikonas>well, I not have just sysa/tmp/after/mes/ <fossy>wait so you have a diff between master and what you're currently running? <stikonas>fossy: yeah, trying to switch to non blobbed mes <fossy>stikonas: ah, can i have a look at your diff <fossy>thanks for doing that, i was going to do that next, but you beat me to it <stikonas>lrwxrwxrwx 1 andrius andrius 1 2021-01-04 21:27 sysa/tmp/after/mes/x86-mes -> . <stikonas>andrius@laptop ~/repositories/bootstrap/live-bootstrap $ <stikonas>but maybe two pairs of eyes will be quicker <fossy>that symlink is expected i mean <fossy>probably something changes between wip-m2 and mes-0.22 <stikonas>argh, I didn't push mescc.scm from my local checkout <fossy>where do i need to put that in your tree <fossy>A: no gash does not run in mes <stikonas>it might be that might mescc.scm is unfinished <stikonas>maybe need to adjust includedir or libdir... <fossy>one thing, please update the gitlab janneke's mes submodule to use https not ssh <fossy>because it requires auth over ssh <fossy>hm, no your mescc.scm looks sane, %prefix evaluates to /after, andd so%libdir is /after/lib <fossy>and %includedir is /after/include <fossy>but that dosen't meatter bc it's not even used iirc <fossy>there's a few more things to clear up but i'll do those when you PR, let's get this fixed first <stikonas>I was able to compile mes via bash and configure.sh but we don't have those... <fossy>stikonas: change the ln lie for x86 mes to <fossy>ln -s lib/x86-mes tmp/after/mes/x86-mes <fossy>now i'm getting unhandled exception: unbound-variable: (abort) <stikonas>let me try to see if error is still there <stikonas>well, that link change was one of the things I tried changing manually... <fossy>actually the link change worked for me <stikonas>well, yeah, the link is pointing to the new location <fossy>then that gets rid of the (("mescc: file not found: \"x86-mes/x86.M1\"")) error for me <stikonas>but I think mescc was still complaining about x86.M1 file at the top of exception list <fossy>what did i jus tdo to get rid of it lmao <stikonas>unhandled exception: error: (("mescc: file not found: \"x86-mes/x86.M1\"")) <stikonas>even though the link points to x86-mes -> lib/x86-mes <stikonas>but somehow I guess search is not from $PWD <fossy>this makes basically no sense to me <fossy>x86-mes/x86.M1 def exists... <stikonas>fossy: same error happens if I compile very basic file <fossy>just set that before any of the scripts are run <fossy>bc it was looking in /after/lib for x86-mes and co <stikonas>I think then I need to remove HAVE_CONFIG... <fossy>you only changed it for the first one <fossy>you need to set HAVE_CONFIG=0 <stikonas>yeah, well, I didn't change all of them yet because I was only testing... <stikonas>probably just copy to some folder in sysa <fossy>what's wrong with mescc.scm? <fossy>i have no problem with just commiting it into the submodule <stikonas>well, I just copied it into mes tree for now <stikonas>it might work but maybe will make updates harder <fossy>yeah, if it will work in the submodule <fossy>not sure if it tracks downstream commits in submodules <stikonas>I think in terms of outer repo, it's just a singled .gitmodules file <stikonas>and inner repo is just independent git repo, you can do whatever with it <fossy>unhandled exception: system-error: (("No such file or directory" . "lib/linux/x86-mes-mescc/mini.c")) <fossy>you'll need to update them for the changes <fossy>so, to find the sources, look in build-aux/configure-lib.sh <fossy>mes_kernel=linux mes_cpu=x86 compiler=mescc <fossy>lmk if you need more help with that <stikonas>argh, actually, I think I won't be able to push if I commit something to submodule <fossy>well my suggestion then would be just cp it in rootfs.sh <fossy>thanks for your help btw, very much appreciate <stikonas>I guess if we don't have gash later, we also need to write kaem for bash... <fossy>i'm going to be writing a lot of kaem scripts <fossy>also if you are going to do more after this please don't do tinycc, i've already started doing that <fossy>just so we don't dup work :) <fossy>also, FWIW, i don't want to use pregenerated configure scripts. they are too hard-to-read and are effectively binary code as they are not human readable <fossy>that means autotools before bash <fossy>because we won't be using configure for bash anyway <stikonas>well, pregenerated configure scripts are indeed not really source... <fossy>i'm doing a few things here which are quite unusual but i believe they are important <stikonas>pregenerated mescc.scm is at least readable <fossy>i don't mind pregenerated if it's readable <fossy>for example, i bascialyl generated mes.kaem using a script and just replaced the file names <fossy>but it's very readable because it's just something repeated <stikonas>and I guess at some point tar/gzip will be needed, to avoid submoduling everything... <fossy>basically my plan is: tcc -> bash -> patch -> tar -> gzip -> perl -> autoconf -> automake -> make <fossy>i'm not sure where i'm going to put the linux kernel <fossy>because this will manage kernels too <stikonas>well, in principle the place for kernel is not too important... <stikonas>for now we are using precompiled linux kernel anyway <fossy>yeah, but i want this setup to manage kernels as well, as once we have a smaller seed kernel it will not be able to complete the entire bootstrap <fossy>i will add all of this to the readme that i am wiritng <stikonas>yeah, seed kernel only needs to support a few syscalls