<oriansj>if anything isn't immediately clear, it is a bug and needs to be fixed immediately <oriansj>fossy: the building mescc-tools from the 357byte hex0 seed <plasma41>oriansj: The README in that repo says to run kaem.run without saying what directory it is in. Does that count as not immediately clear? <fossy>plasma41: for now, just know you need to cd into the directory for your archote <fossy>unless oriansj plans to fix it I will PR clearer instructions shortly. <plasma41>oriansj: The comments in the file x86/kaem.run says it can be run with kaem, but `find | grep kaem` of the root directory of the repo doesn't turn up a file simply named kaem. <NieDzejkob>AFAICS it's the 250 MB linux kernel, what gives? <fossy>plasma41: yeah its called kaem-optional-seed <fossy>to make it obvious its a seed <fossy>NieDzejkob: right now yes the linux kernel <fossy>in the near future a custom kernel written in m2-planet <plasma41>fossy: Well add that to the list of not immediately clear things. <fossy>in the long future on Knight hardware <fossy>Knight is an architecure that uses punch cards and no bootloader microcode or kernel <fossy>we really need a bootstrapping manual <fossy>kaem is the program, kaem-optional-seed is just the name of kaem in the repo <fossy>but thats no good because people wont understand that <fossy>oriansj, janneke: how would you feel about a bootstrapping manual created as a git repo that people can pull request to, hosted on bootstrappable.org? <fossy>alternatively, a wiki or something like that <fossy>but make it easy for people to contribute <fossy>plasma41: tbh there's a few not immediately clear things <fossy>and for those who develop them its pretty easy to miss those over as we might think they're simple but they actually arent obvious to someone who hasent used the repo <plasma41>What's the relation between hex0_x86.S and hex0_x86.hex0 <fossy>plasma41: I think .S is an implementation of the hex0 monitor in M0 assembly <fossy>and I think it only exists for verification purposes <fossy>ie you can compile the hex0 one and the M0 one and check they have the same hash <fossy>and then you only need to audit the simpler M0 one <plasma41>Why do some of the comments in those files use 0xc2b7 instead of 0x20 for space characters? <fossy>plasma41: which file and line number? <plasma41>Have I filled the not immediately clear quota? ;-) <fossy>ok time to rewrite the readme <fossy>can you read it for me and tell me how it looks <fossy>hey janneke; this "delta of 11 patches", where is that hosted? <fossy>do you have /just/ the patches not a tarball with the patches applied? <plasma41>The first sentence is all over the place grammatically. <plasma41>What all does the word 'creating' apply to? Does it apply to just the next word or to the whole rest of the list? Is it a list? There are multiple semicolons. <plasma41>In the second sentence of the fourth paragraph of your rewrite, you delineate the phrase "AMD64 for amd64 and x86 for x86/i386" with two dashes on the left and a comma on the right. It should be the same on either end or you should instead put the phase in parentheses. <plasma41>The last sentence of the fifth paragraph doesn't need that semicolon. However, it would benefit from being broken up into a few separate sentences. It's rather long. <fossy>lol im not good at grammar and the like <fossy>ill apply your comments shortly <plasma41>My elementary school English classes drilled grammar rules thoroughly into my brain. <plasma41>It's been around two decades since then and I still haven't managed to forget all those grammar rules. lol <plasma41>"Note that the kaem.run is split into two kaem files to make it simpler to grasp." This sentence should name the two files if it's going to mention they exist. <plasma41>Having the Phases of the bootstrap process stated explicitly in the README makes things a lot clearer. <plasma41>It's getting late in my timezone (CST), so I'm off to bed. I'll try to be back on IRC tomorrow afternoon to give further feedback. <plasma41>oriansj fossy: Thank you for your help and your great work. <janneke>fossy: no, the patches were created halfway between the tcc v0.9.26 and .27 release <fossy>janneke: thanks so much for all your patience with all my questions <janneke>previously that was more clear, the wip-bootstrap branch has switched to using the tarball with patches included <janneke>you are helping to make this process easiers for people that come after you <janneke>and finding problems/bugs or verifying what we have done -- very important <janneke>you can imagine that one wish i have is to build a released, patched tcc v0.9.27 and get most patches upstream, for example <fossy>but it seems upstream dosent want to help <janneke>well, i proposed waay more radical patches after i just succeeded to build tcc <janneke>so, when this bootstrap is really a thing, we will try again <oriansj>fossy: thanks for the patch it looks great <oriansj>although the idea of a git repo containing all of the collected knowledge of bootstrapping does sound like a good idea <oriansj> the relation between hex0_x86.S and hex0_x86.hex0 in the repo is partially dependent on location; for example ./x86/hex0_x86.hex0 and ./x86/Development/hex0_x86.hex0 should be the exact same file; where as ./x86/Development/hex0_x86.M1 is it in M1 macro assembly and ./x86/NASM/hex0_x86.S is it in NASM assembly <oriansj>The reason for the NASM directory is for those that don't trust THe hand written hex bootstrap or Mescc-tools (when built with GCC) but are willing to trust NASM built binaries; to verify all of the pieces in the bootstrap chain. <oriansj>The Development folder is to deal with the reality that even I don't write in hex, rather I write code in macro assembly and then when it is done and behaving how I like, then I convert to to hex (which is very painful process done rarely) <oriansj>plasma41: and the reason for 0xc2b7 in the files instead of space was entirely a side effect of copying from tig/less when doing the development (since writing hex* and M* for $ARCH is generally repetitive as many things keep repeating pattern wise) so those should be pulled. thank for pointing out that detail <oriansj>I guess I can blame whitespace mode in emacs for me not seeing that <oriansj>janneke: also, we can eliminate the need for the patches by simply implementing the missing features (which would put us closer to the goal of building GCC directly with MesCC ^_- ) <janneke>oriansj: yes, for most patches we probably could <janneke>mescc certainly deserves some attention and love <janneke>but that has no priority for me, atm <oriansj>especially given the current guix changes <oriansj>janneke: your wip to reduce the guix seed to 60MB <janneke>why not all? well, i would take patches to remove them all ;-) <janneke>a patch adds build scripts (we do not have make yet) <janneke>those build scripts, combined with patches to step-wise introduce long long, float and double, enable us to build tcc using a compiler that has no long long, float or double support <janneke>i would be happy to add support for that to mescc, but i don't think it's wise to add such features before fixing bugs we still have in the type system <janneke>i haven't attempted to build make using mescc, we could try that <janneke>but i'd rather have a simple make implementation in gash core utils <janneke>so many interesting and diverse tasks! <janneke>i hope that what we are doing is inspiring people to join <oriansj>scheme programmers to enhance MesCC and nyacc; test mes-m2. <oriansj>a C programmer or two to make my code even more pretty <oriansj>but I do kinda have dddddd who is doing a beautiful job on M2-Planet <oriansj>so I guess just more scheme programmers? <janneke>my plan to merge back mes-m2 just before your real crazy test+slow-lisp merge is to have shortcut/faster path to hex0->m2-planet->mes + guile-modules <janneke>now it may be that you get mes-m2 production ready sooner, that's great too <janneke>but i was projecting your route would lead to much cleaner and solid code, but take a year more or so... <janneke>once we have real guile modules in mes, it's easier to start recruiting scheme programmers <janneke>there are now still a lot of restrictions wrt the scheme you can write <janneke>but hey, that's just a plan; anything else is fine too <oriansj>janneke: well we could have real guile modules in mes-m2 this month <dddddd>Hello booters! Thanks for the kind words oriansj, but I don't think I did anything that pretty :) <oriansj>janneke: if I have good tests and a solid understanding of the primitives, they are easy to implment <oriansj>hence why mes-m2's display and write are entirely in C code and are smaller and faster than the scheme version in mes.c <oriansj>hence why I keep trying to figure out good primitives for guile modules <oriansj>because I can write those easily (assuming they are clear) and things that are easier done in scheme can be done is scheme <oriansj>I know you are excited janneke to be done but I don't want to release something that falls apart when people try to use it <oriansj>I want code easy to reason about at the heart of this bootstrap, so that everyone will be able to sit down in the morning and be done by lunch time; understading how every piece of it works. *janneke notes that quote <dddddd>I guess "everyone" is a bit too strong word. More realisticaly... what previous experience/knowledge is supposed in that context? <oriansj>look, I spent from August 16, 2018 to November 16, 2019 hammering on mes.c to make it M2-Planet happy. <oriansj>I've only been working on mes-m2, the way I think is correct for only a month and a half <oriansj>yeah, it is a little behind mes.c in regards to the number of primtives it has <oriansj>but for all of the primitivies it does have, they are drop in matches to what guile does <janneke>i agree, everyone should be able to understand this, to the level they feel they need to know <janneke>we have a working bootstrap, much more than a rope ladder already <janneke>we have time to make it simple and auditable <oriansj>I know that I perhaps spent a bit too much time in assembly and have to reload my scheme solution set. but that is what needs to be done in mes-m2. We need to look to guile primtives and figure out: is it easier to just do entirely in C or in scheme leveraging these: core:primitives that are well defined to do X <janneke>i think it's great/amazing what you are doing <janneke>with merging back after a year of your work, i am just hoping to buy us time by delivering an intermediate, big update <janneke>it's all guessing, planning wise for me <janneke>who knows, debian, arch, nixos, arm -- anything can interrupt <oriansj>debian has a massive and growing dependency blob they can't fix. Arch doesn't have any developers on it, nixos is drafting off of guix and arm doesn't care <oriansj>So stop feeling like we have to rush <oriansj>janneke: that last few years of rushing only makes it take longer <oriansj>every single shortcut we have taken, only delayed us more and more <oriansj>the time I wasted writing lisp in assembly <oriansj>the time I wasted writing FORTH in assembly <oriansj>the time you wasted trying to write binaries directly (and all of the debugging time too) <dddddd>Silly question, in the context of having a drop-in replacement of guile. Doesn't make sense to start from guile (and remove/adapt things as needed) instead of growing a new scheme from the "other side"? <oriansj>dddddd: not a silly question and at first glance that would make sense, except for one minor insignificant detail <oriansj>scheme programmers, write C code like it is scheme <oriansj>and why I spent 15 months getting mes.c into a resonable state before giving up <dddddd>And that's not good because... hard to understand? to change? is it slow? <oriansj>dddddd: tightly bound is the word I believe <oriansj>a minor change in one part causes bugs in dozens of other parts <oriansj>also tends to depend upon a scheme bootfile to do anything is insanely common and makes good low level tests impossible <oriansj>check the channel logs if you want to see me bashing my head against that wall over trivial things <oriansj>after looking at all the macros used in that C code, I reject that option based on direct first hand experience <oriansj>I would much rather have to do it entirely from scratch, than do that <oriansj>if someone else wants to pick up that task; I will happy go back to working on armv7l's bootstrap pieces. Which believe me would be a much more pleasant task <oriansj>but MesCC needs a proper Scheme and guile needs a proper scheme to be bootstrapped; so that is what I am doing, in the way I know will work and will be easy to use and maintain. <oriansj>If someone wants to take a crack at the guile route, let me know and I'll take a month long holiday at a warm tropical beach completely disconnected from everything digital <dddddd>What they used to bootstrap in the first place? <oriansj>dddddd: that is a good question, that I honestly don't have the answer to <oriansj>So until otherwise, since no one else wants to work on it; I'll be hammering on mes-m2. <oriansj>I honestly am still trying to figure out how janneke managed to write C code that gcc couldn't compile (and I spent 3 months before I got GCC to be able to compile mes.c) <janneke>what who used to bootstrap what, dddddd ? <janneke>oriansj: yes, that's something that really puzzled me too; i have always been using gcc to compile and debug mes.c; but somehow that did not work for you. <janneke>your efforts did help to cleanup mes.c enormously <janneke>before you forked mes-m2, mes.c used to #include all .c files, and we had snarfing of .i and .h files... <janneke>i merged all those changes back into mes-0.19, which made it much, much cleaner <janneke>dddddd: it has always been possible to bootstrap guile with a c compiler <oriansj>and faster ironically (without changing how the pieces worked) too <oriansj>it has always been able to build guile with a C compiler <janneke>i think that the psyntax bootstrap path has been lost <oriansj>so basically anyone's best guess is they just used another scheme or lisp to build what they used and as janneke said those details are now long lost <janneke>i think psyntax is was adopted by all current schemes <oriansj>probably because syntax macros are a bitch to get right <janneke>i spent about a month to get another version of that to run, and got nowhere <janneke>i decided to postpone this until we get some serious scheme hackers on board <janneke>meanwhile i did learn quite a bit about scheme and i just might have another look in a couple of years <dddddd>So, I'm confused. On one hand "it has always been able to build guile with a C compiler" but on the other hand "needs a proper scheme to be bootstrapped". What am I missing? <janneke>dddddd: we are talking about two different things <janneke>dddddd: do you know what psyntax is, how it works? <janneke>many schemes (chez, chicken, ...) need a binary version of themselves to compile themselves from source <janneke>guile can be compiled with only a c compiler <janneke>in guile-1.8, psyntax was optional and provided only syntax-case <oriansj>in short one needs psyntax support to bootstrap psyntax support, so guile just ships with a compiled version of psyntax that is just loads <janneke>in guile-2.0, psyntax was fully adopted/embraced and many "primitives" that were formerly written in C are now provided by loading psyntax <janneke>oriansj: i would avoid saying compiled, because guile now also is a compiler <janneke>guile ships with an expanded version of psyntax macros <oriansj>which are generated by guile and no human understands how it works <janneke>still scheme, but generated/expanded, so not "source code" <janneke>it would be amazing if we could implement andre van tonder's above syntax case and use that to bootstrap guile's psyntax; remove the expanded version from guile (and mes) <oriansj>janneke: it is just a matter of effort on our part. <oriansj>mes-m2 will be doing that as part of the be a drop in replacement for guile <oriansj>hence why I broke out all macro-expansion in mes-m2 to make it easier for people to reason about the various pieces <oriansj>dddddd: so yeah, it is just one of many subtle differences between buildable and bootstrappable; which we are addressing <janneke>do we have/can we think of a list of things that are still not "just a matter of effort?" <oriansj>janneke: everything ever achieved by mankind has only ever been a matter of effort. <oriansj>Now somethings require certain abilities but with sufficient efforts those too can be obtained in the modern market place <oriansj>aka a thousand illerate people can hire a single literate person and from that they all can gain the ability to read <oriansj>with enough effort (or money if that makes it easier for you to reason about) landing on the moon and solving global warming is entirely possible <oriansj>aka, if 8,000 people are willing to work 30 hours a week; they could solve global warming themselves <oriansj>not doing the bullshit efforts that they are doing now but rather something that might actually work <oriansj>hell if the equivelent amount of money was collected and spent carefully, global warming would be getting solved in a real way <oriansj>If doing what was always done doesn't produce the desired results, stop and think. Realize an alternate strategy is likely required. <oriansj>People need to stop being scared of problems that are too big and grab a freaking tool and get to work. <oriansj>You don't solve problems with speeches and public support <oriansj>You use the one thing that every human on the planet has; the ability to work problems <oriansj>and you keep hammering on it until it breaks <oriansj>I don't care if you only have a shovel. <oriansj>A single man literally cut through a mountain with just a shovel and anger <oriansj>so grab a freaking problem and do some fucking damage <oriansj>Fuck a single lady on Twitter raised $1M by just taking naked selfies to help address the fires in Australia *jelle thinks there is a comma missing <oriansj>We don't need dick picks for curing breast cancer, we just need more people putting efforts into breast cancer research and getting patents the fuck out of medicine *oriansj thinks jelle is probably right but rants do include errors <jelle>oriansj: just looked it up and found: "'My IG got deactivated, my family disowned me, and the guy I like won’t talk to me all because of that tweet. But f*ck it, save the koalas,' she said." <oriansj>jelle: exactly, every effort worth a damn comes at a cost <oriansj>Does anyone here think my wife is happy with the time I spend hacking on bootstrappability? <oriansj>when I could be cooking her dinner or building that new shed she wants <janneke>happiness is something you create yourself <oriansj>If you want to spend your time hanging out and drinking beer, just be honest enough to admit you want beer more than you want to solve global warming <dddddd>Meanwhile (and thanking you and your wife for the painful efforts), what about the list of mountainst to hammer? :) <janneke>i belief that people do drugs because they feel they cannot accept being part of this world and believe they are helpless to change it <oriansj>dddddd: I don't care to deal with that shit <oriansj>janneke: or because they need the drugs to feel normal <dddddd>I mean, not offtopic mountains, but bootstrapping ones (as I understood janneke's question) <oriansj>I honestly am hoping other people get their shit together, otherwise after I am done with bootstrapping; I'm going to have to start fixing global warming too <oriansj>dddddd: Knight in FPGA, Knight in TTL, Posix kernel buildable by M2-Planet, x86 in FPGA, x86 in libresilicon and after that I don't care; let the haskel people clean up their own mess <oriansj>I am willing to carry everyone else while I build to solve my own goals but I sure as shit am not going to try to save every dumb language. (looking at you Haskell and Standard ML) <oriansj>the pieces that I will off-handly help: porting M2-Planet+mescc-tools to new architectures (stage0 steps if the architecture looks clean) <oriansj>janneke has a good handle on MesCC and above; I trust him to do the right things <oriansj>just like I trust you dddddd to do the right think in porting M2-Planet to AArch64 <oriansj>I only have so many moments to solve the things I want solved in my lifetime <oriansj>So I focus on the things I think are the correct priorities <oriansj>and as you see, the impossible goal of bootstrapping GCC is literally not that far off at all <oriansj>and the goal of solving the kernel bootstrap problem looks easily solvable now <oriansj>we will do all of those mountains that I listed <oriansj>even if I have to carry everyone along the way <oriansj>one by one; not a single one will be left standing <oriansj>we can do anything we put our minds to <oriansj>So keep changing the world, nothing can stop us *oriansj takes a sip of water *oriansj goes to make breakfast *dddddd too; eating stops us :D <oriansj>I'll add anyone who wants to contribute to it <oriansj>currently adding contributors to talk-notes so that everyone in bootstrappable are able to improve existing bootstrappable documentation <oriansj>janneke: I invited you as a contributor to talk-notes, so you'll be able to push any changes you feel is wise <oriansj>that way all of the bootstrapping material is in one place for anyone to clone and run with if they want to do a presentation <janneke>oriansj: thank you; i am not using github but i might pull and send patches <janneke>i agree that we need something and i think we can use bootstrappable.org for that <oriansj>janneke: well bootstrappable.org would serve a different purpose than collecting all material that would go into a bootstrappable talk <oriansj>unless you mean the bootstrappable.org file in talk-notes <jelle>the reproducible-builds.org website has a section with talks btw <oriansj>jelle: none about bootstrappability (for example none of janneke or gio or rekado's talks) <jelle>oriansj: indeed, but this would be nice to host on bootstrappable.org I think <oriansj>jelle: true, I'll even write up a list with links <oriansj>I probably should include carl dong and mark jenkin's talks as they are quite related <janneke>oriansj: yeah, assorted notes/vague pointers should not be on the website and thus probably better in your talks archive <oriansj>janneke: although I can't find info on the FOSDEM 2018 talk; was that the one which guix had their own breakout? <janneke>oriansj: i did not give a talk in 2018 <oriansj>then why do I have slides for FOSDEM 2018 written on Feb 1, 2018? <janneke>did we put something together for rekado_ to present? *janneke checks the hash of their universe <oriansj>but I have that in a seperate folder <oriansj>and I can't find a time or date for when the talk actually occurred <janneke>it was only in april 2018 that i was able to compile a working tcc <janneke>so, in 2017 the dream was announced, after ~10 months of work <janneke>only to have something concrete to talk about late 2018 <oriansj>Wish I could find gio's slides from debconf 2019 to include <oriansj>(It looks much better in emacs org-mode) <gio>Problem is that it is a reveal.js presentatio, so it is HTML. <gio>But in theory there is an export to PDF feature. <oriansj>gio: mind if I incorporate it into talk-notes? <gio>oriansj: No problem, please do it! :-) <oriansj>and now 4 of the 6 talks about bootstrappability are all together; just to annoy carl dong and Mark Jenkin's for their slide sources <oriansj>well that is a crap load of pull requests <oriansj>just remember if anyone wants push rights to that repo let me know and I'll add you <fossy>oriansj does it have to be a talk <fossy>oriansj: youre right. doing things the right way now will save us time and energy in the future ***fosslinux is now known as fossy
<oriansj>dddddd: yep, bootstrapping is something absolutely counter to easy language development. <oriansj>fossy: non-talk bootstrapping related material can be added to the bootstrappable.org org-mode file in the repo. If you are to add a talk, include it in talks.org and if you wish to share your slide source material feel free to add a folder named $conference_$year for that material <fossy>hey janneke, I've found a bug in mes <fossy>Anyway, compiling i386-asm.c from mes-0.21 on your tinycc fork fails because it dosen't like the registers %si and %sil <fossy>These seem to be defined in the libc <oriansj>fossy: that would be the GCC source not the MesCC source <fossy>but i'm not using GCC at all in this TinyCC compile... <fossy>oriansj: how does GCC relate to it? <oriansj>M1 would treat %si and %sil as numbers then fall back to trying to resolve them as labels <fossy>ok, so where does the issue lie? <oriansj>GCC uses %si and %sil as register names for inline assembly <fossy>so why would mescc be thinking GCC/binutils is being used as the assembler when it calls M1? <oriansj>MesCC given that sort of inline assembly would either have to convert it to M1 macro assembly or use a different file with M1 macro assembly replacement inside <oriansj>hence why I am thinking the wrong source file was passed to MesCC <fossy>maybe something is wrong with bootstrap.sh <fossy>oh, i see from BOOT that janneke is using make rather than bootstrap.sh <fossy>except I don't have make at this stage <fossy>so I'll adjust bootstrap.sh to match whatever make is doing <fossy>well actually i can just get the commands make is doing and put them into kaem.run since eventually I'll have to refractor to ensure we aren't using a shell <fossy>because i don't want a shell in the bootstrap binaries for this project if i can help it <fossy>oriansj: do you know of a way to find what line number mescc is failing on <fossy>because it dosen't seem to be in the error output <oriansj>fossy: Well unfortunately, I never figured that one out myself <oriansj>hence why I suggested using guile to get the steps down first <oriansj>it tends to make the troubleshooting easier