<janneke>malina: yes, the past days (weeks) we closed the gap between stage0 and gnu mes <janneke>this has been anticipated and worked on for years by multiple people <malina>but there is still some 'binary seed' required but it has almost been removed entirely or? <malina>sorry I literallt came across this bit today when I coudn't find old edmund's notes in my computer anywhere. <janneke>there is the 357-byte hex0 binary, that has an ascii equivalent and thus can be blessed as source <malina>yes, I been sincd 2010 doing a ton of desing specs andw aht not and I keep discovering things , I kept either 'designing' mostly, and a bit coding (poor as I am hehe). find or been told of developments or stuff which I also had in the works <malina>I will try to find some time to delve into this because this is exactly what i implored edmund to give some thought at god knows when. he ddi say he would think about t, about bootstrapping further, and so on, but guessing it's busy at arm and ya. <malina>still now doesn't matter, ,given you guys had a go at this and here it is. <malina>this is amazing, because we CAN choose to trust this or that, but every user SHOULD in an ideal world , be able to develop that trust or at least bootstrap each piece of branch in his/her software <rain1>stage0 is able to build gnu mes! <malina>_especaillu_ becaise of how we are (mis)used <malina>from our metainfo created by using our software and so on <rain1>is that using the haskell bits? how was this done? <janneke>yes, m2-planet can build mes, the wip-m2 branch <terpri>also, if one is worried about the cpu microcode distributed by the maker of the cpu, one should also be concerned about the behavior of the silicon itself imo (intentionally enabling subtle sidechannel attacks, etc.) <rain1>is there a blog post or anything about this <janneke>rain1: i simplified the mes code base with many small steps to prepare it for m2-planet <terpri>i think some people (bunnie huang?) have proposed using fpgas instead of traditional cpus for that reason <janneke>rain1: at this time only an oot/tweet <janneke>it's very fresh, plus i've only built up to tcc right now <malina>terpri ok, but I mean, at tha level, even if possible, it's gonna be VERY expensive vs. being able to "TARGET" specific people or so.. I mean <malina>sure, so evil will always exist, anyway right, I mean sometimes we just have to call a spade a spade, and accept some minimal level of distrust ,) <janneke>(but we know that mes => tcc => <everything> is in production already) <malina>what I mean is like 2048 rsa signed microcode esily being delivered into cpu's is kinda taking the piss <terpri>yeah, i'm nitpicking a bit, i'm not personally concerned about that threat level <terpri>and yes, the (usually) updatable nature of microcode makes it different, i suppose <malina>well, it certainly allows far easier , manipulation by the foxes guarding the hen house <malina>but it's a different dicussion than this. <malina>getting a replicable source path from hand edited hex all the way up to the toolchain, is what is _always_ missing <OriansJ>malina: if one chooses to target us; then they will gain nothing. <terpri>(the line for me is probably somewhere between wanting libre microcode and firmware, and not having antifeatures like ime and psp) <malina>I keep reading gnu's claims of bootstrapping and I been deceived like hell past months <OriansJ>There is no secrets, no unreviewed changes <malina>always some binaries needed or prior compiler etc. and even if they aid it's ok to grab native ocmpiler first, and then expand, this was certianly not true. <OriansJ>everything in stage0 can be audited in a week from scratch <malina>ye, briliiant OriansJ / Janneke! <malina>presuming , seeing those names in the work I came across today, <OriansJ>737bytes for an init/shell and 357bytes for hex0 and everything else is source code with comments <malina>so 4 years with my crap coding I been struggling along to do various ideas, looks like guix, with way better coders and more users, are choo-chooing past I ponder if I should just give them a try <malina>yes, I will delve into this, and hopefully I can integate it into the bootstrap tools of my distro, I been waiting for these things <OriansJ>malina: we always need another set of eyeballs and a different perspective <malina>ya, truncate -s 10G guix.img;mkfs.ext4 guix.img; runiso (a qemu alias basically) iso img ; copy install done .. then I just boot it usually from my host partition if I need 'on the metal' testing <malina>ye I think I did a recursive clone of stage0, but not that one <malina>do they pull each other somehow recurseivle or? <OriansJ>malina: mescc-tools-seed is the POSIX port of the stage0 work <OriansJ>that one needs only: cd x86 && ../bootstrap-seeds/POSIX/x86/kaem-optional-seed to bootstrap all the way up to M2-Planet and mescc-tools <malina>right.. well I did find your readme in Linux/AMD64 or o <OriansJ>we also have AMD64 and AArch64 ports with virtually idential commands <malina>I'll log his convo into a note, and start delving into this in the coming week <malina>what a perfect start to the year ot come across this! :) <malina>I literally couldn't find edmund's notes from eons ago and was shufflign about looking in all kinds of partitoins and so on, then found his notes under some other guy's name.. which led to the hacker news comment or so about stage0 and ya, <malina>yes, how long time would it sake to get to m2-planet (which I understood is bootstrapped and ready to take on tcc, per janneke's ocmment) or? <malina>didn't know if maybe the.. I mean, don't KNOW yet where and what state and how long it would take ot get to a tcc compiler, right? :D <janneke>building mes may take 10min, building tcc make take 20min <malina>but if it's nominal, it just gets' EVEN BETTER <malina>ye great. good numbers, I mean, it even means even donkeys can bootstrap their own stuff inthe future.. like our children or so, and after all, might wanna have some sense of privacy or trust of their machines. it's a long shot but the principle is important. this is literally making me <janneke>malina: no, m2-planet compiles gnu mes, then mes can build the mes c library and lightly patched tcc <OriansJ>malina: dict and dict-gcide usually is a good combo <janneke>note that in guix, guile is used as a driver and is not a seed <Hagfish>tangentially related to reproducible builds, i find it really amusing that in Japan, "Satoshi Nakamoto" is written using katakana symbols, as if it's a foreign name <malina>except now and then when I am exploring my stuff, I sometimes ccome across them with similar stuff <malina>well, given the fact that the ONLY number which could be iffy , with bitcoin hash function <malina>it's ALL ok AS LONG AS S.N. did NOT have access to parameters, i.e. as long as it's not NSA/NIST hey, it's all good <malina>now, I personally am very certain if a single person was S.N. and had 4 billion btc or whatever it is, lol... they would bloody use it <malina>so I dno't really think it's really all that 'oh how is S.N.", as far as I am oncerned, it's pretty obbvious. <malina>I recall some yearsa go looking a tthe bitchain? trasnatcion ledger whatever it's called <malina>I don't normally check it, and the 5 most recent trasnactions totalled over 100 million euros <malina>now if THAT is the kinda money which keeps flowing through BTC, boy would it be nice if one somehow could have an extra eye on that aye? <malina>ye no, people have to be less naive. <malina>and my method is very nice and clean <malina>in my distro we have 'boot from any image file ' <malina>I support loop, qemu images etc etc <OriansJ>single partition is fine or if you like you can seperate /gnu onto a seperate partition too <malina>well yes, in 2010 I started 'spec desiging a package manager', turned out not long after, nix came out, ironically A. functional as I used to adore back at uni, and B. from one of my uni's hehe. <OriansJ>malina: that is just the standard store where all the built binaries are placed <malina>but instead I ended up doing lots of stuff and only in 2014 did I start nominally forking my old distro's package manager instead <malina>until 2016 , when I went more full in, and now I .. wish I had started from scratch I suppose (but I knew , it was a better chance to just work on pre-existing and even familiar) given I , as mentioned earlier. am not a good coder at all. <malina>I am writing in D slowly some backends <malina>which of course has a 'bootstrap' and not entirely opens ource ironically <malina>again thanks OriansJ , but a elaborate recipe for just slapping up a quick fs on a file , before booting <malina>is a bit over the top, but I see, once I know how guix ops work I can call a recipe and integrate it with booting from same root filesystems. Sure, thanks. It would take some time to move to guix. but I don't think I will be able to give up my own quite yet.. however, I did think t myself just yeserday that guix might be somethign i should try, since indeed, dunno, seems it might be the closest to many of my ideas. the only thing is, ya nvm <malina>im glad I actually slapped up 2 isos yest, to see something in kde and something in a kernel patch by ubuntu, <malina>so quickly looking at that recipe <malina>I don't need the bootloader right? <malina>I could define 'target' fs/mounts without a bootloder section? <malina>can be quite annoying when a lot of primitive installers can't habdle that, (tried voidlinux' as that second iso It hink) and the 'gui' can't continue if one wants dev/sda instead of /dev/sda1 :p <malina>although luckily it has 'remote root' option and can install that way <malina>nix though seemed to have more issues when I ask a rust dev with something like mozilla-hg which builds easily, so he left me kinda less convinced nix really is as 'ez pz' as it claims to be. <malina>may I ask, after I edit such a scheme, is it? how do I fire it up ? <malina>I didn't come here for another distro but what init does guix use? <malina>anyway package management design is a different topic. and very long one :d well ni ni. and thanks a lot for all the info. <yt_>I'm thinking of adding a C preprocessor pass to M2-Planet; should be fun, right? <yt_>janneke: this might be interesting for the readability of wip-m2 of mes <OriansJ>yt: well it certainly would be an interesting problem to solve. <OriansJ>malina: the example configuration is the sort of file you would download in the previously specified install steps for guix-system inside of a VM. <OriansJ>or if you wish to save yourself the trouble guix has a Graphical installer which should be quite easy to use. <OriansJ>as for what init guix-system uses that would be shepard (written in guile) <malina>ye, the first paste seemed to be what I prefer, so that's great. But if II have issues I will see if a installer helps. and ok, cool. the init here is called superman <malina>after superman[ager] obviously :) <malina>but even if the POC atm is in guile if I understabdn correclty, the same work can be replicated in other systems right? I mean, is guile or guilx part of the actual bootstrapping or is it the 'glue / infrastructure' one is referring to here? <OriansJ>malina: well one needs a scheme which can run MesCC, gash, gash-utils and guix to use the MesCC bootstrap route. <OriansJ>janneke's work is using mes.c to provide for all of those pieces except for guix <malina>right so currently it's written into the guix infrastructure. although still. that is still perfectly ok since as we discussed, usually one has some prior environment anyway, and one can, use the bootstrapped gcc say to cross into any other system. gash? is what, a guix shell or something? <malina>ugh, settting mtime to 1 for reproducibility. Unfortunately I work on that too and the stuff I see in other projects trying to make things ore 'reproducible' just makes thing anythign but imo. <malina>and then it will pullite the entire ecosystem by the time I have it sorted I bet. sigh, but that's the cost of being useless I suppose (as a programmer). ,) <malina>although that particular example isn't the worst (resetting mtime as such), I have seen other such things in gnu packages lately <malina>random timestamps stuff which just isn't a very good idea I think. Not that I have looked into it greatly lately but anyway. <malina>although generally, ye, I see guix as i say typically seem to try and achieve similar goals I always have in mind. <malina>this is what I kept noticing. so will have ot play with this. didn't think I would return to a public distro ever, but maybe I will. <stikonas>fossy: are the any requirements on kernel in your live-bootstrap repo? <stikonas>I tried my current x86_64 kernel and got segfault in cc_x86... <fossy>stikonas: i've been using my normal boot 5.9 kernel......... <fossy>can you provide your kernel somehow to me? <fossy>and if possibel the config it was compiled with <fossy>i must say that is really odd and should not occur <fossy>and especially at that point it should be completely reproducible <fossy>did you do git submodule update --init <fossy>sorry i need to write a readme <stikonas>well, I ran git submodule --init and git submodule update <OriansJ>stikonas: does cc_x86 crash when you feed it a file with the following contents void main() { return 42; } <fossy>that's hard to test in live-bootstrap lol <fossy>i bet it's the empty file problem <stikonas>[ 2.917375] cc_x86[88]: segfault at 9650000 ip 00000000080484a6 sp 00000000ffb14338 error 6 in cc_x86[8048000+4000] <fossy>wait can you try git submodule update --init --recursive <fossy>because mescc-tools-seed has submodules inside it so we have recursive submodules <fossy>yeah, didn't even need to test your kernel, if i do it without --recursive it segfaults because well ../M2-Planet dosen't exist <fossy>yeah, i've been procrastonating on it <fossy>i'll write one this afternoon <fossy>lmk how it goes, i really really hope it succeeds :D <stikonas>well, for now it's non-bootstrapped mes anyway <stikonas>but in general this live-bootstrap sounds really cool... There will be another alternative to guix... <stikonas> +> /after/bin/mes --no-auto-compile -e main /after/bin/mescc.scm -- -c -D HAVE_CONFIG_H=1 -I include -I include/linux/x86 lib/linux/x86-mes-mescc/crt1.c <stikonas>mes: boot failed: no such file: boot-0.scm ***coldtom6 is now known as coldtom
<OriansJ>I think I have a working typedef for M2-Planet <OriansJ>So M2-Planet is going to be requiring --bootstrap-mode for a bit until M2libc is up and running <OriansJ>but that flag is going to be super handy for flagging on features which cc_* doesn't support. <OriansJ>but that work is for tomorrow and hopefully I can finally give M2-Planet proper FILE support <janneke_>fossy: git blame doc/guix.texi | grep 'reach a fixed point' <janneke_>: cf4a912919e (Ludovic Courtès 2014-07-16 11:35:45 +0200 33135) reach a fixed point? That is an interesting question! The answer is ***janneke_ is now known as janneke
<janneke>good question, let's ask civodul when he's around <janneke>fossy: i think what civodul hints at is: 1> we build new bootstrap binaries with guix <janneke>2> we inject them into the guix graph <janneke>3> we build a new set of bootstrap binaries <janneke>=> repeat until there is no change; i.e., fixpoint <janneke>OriansJ: ah, so mescc-tools-seed could be renamed stage0-posix? *janneke just likes the first package build to be 'stage0' rather than 'mescc-tools-seed' esp. because it does not contain seeds anymore ;-) <janneke>fossy: note that this 'preparing for the bootstrap' phase is mostly obsoleted by the full source bootstrap <janneke>what could remain is: rebuild hex0-seed and bootstrap-guile <janneke>but bootstrap-guile is strictly only for guix (any guile will do) <janneke>and thus guix will hopefully never update any of it's bootstrap-guiles <OriansJ>well there is no way for the binaries to change in the guix build as these are all statically compiled binaries. <OriansJ>So in theory the fixpoint is zero rebuilds <OriansJ>janneke: yes we absolutely could rename mescc-tools-seed to stage0-posix; what do you think yt? <janneke>mostly i'm still a bit confused about "stage0" <janneke>initially, that was where hex0 lived <janneke>then, you created mescc-tools...okay...but that surprised me as i thought those were supposed to be part of stage0 <janneke>i created mescc-tools-seed, to contain prebult binaries of hex2 and M1 for guix <OriansJ>janneke: hex0 is just one step that can be done on bare metal or on a POSIX; originally the world was all going to be done on bare metal <janneke>but after i created a proper "%mescc-tools-static-stripped" package to build hex2 and M1 seeds within guix, the need for mescc-tools-seed went away (for me) <janneke>OriansJ: sure, but stage0 also has a stage{1,2,3} directories? <OriansJ>I never said communication was a great strength of mine. <OriansJ>nor that I wasn't shitty at naming things <janneke>so i'd like to get rid of my confusion here first, before acting upon renaming suggestions ;-) <janneke>in a way, a rose is a rose, but sometimes naming helps -- dunno? <OriansJ>I originally selected stage0 because it was going to start from the very beginning <janneke>let's at least have others here weigh-in with their thouughts <janneke>yes...that's why the name is so great (to me) <OriansJ>So as you complete a stage, you enter into the next stage of development; hence stage1 stage2 ... stageN <OriansJ>hex2 and M0 seemed like tools that would have really helped the development of MesCC as you were really struggling with those sorts of issues. So I copied them out and made mescc-tools for you <malina>for the record, if you are using gpl v3 I see here and there, I don't know if that is compatible with static binaries <malina>think you will have to use v2 no? <OriansJ>Then they slowly grew into just a sort of standard universal build tool of sorts <malina>and out of curiousity and further aunderstanding, what is the ~200 bytes extra in your seed vs edmund's original hex1 say? <malina>is it OriansJ ? Well, shows how little I know abotu GPL :) <malina>I use a custom disto licence normallybut ok <janneke>yes, it is convenient to have mescc-tool and m2-planet as standalone, gcc-buildable packages <OriansJ>malina: I support forward and backward references <malina>I was pretty sure GPLv3 wasn't ok with static binaries, but dynamic vs system libs <malina>or is that kinda part of the v2? <OriansJ>malina: you are not helping the conversation <malina>so why is it I seem to recall static binaries + GPLv3 being something something? NO sorry, I interjected. My apologies <janneke>so possibly i'm wondering: is stage0 dead, or what does it contain, what purpose does it have beyond/next to mescc-tools-seed + mescc-tools + m2-planet? <OriansJ>janneke: stage0 has a shitload more work to do after we finish the guix bootstrap from hex0 <OriansJ>as I am not stopping after a POSIX bootstrap <OriansJ>I am going to bootstrap on bare metal <OriansJ>and eliminate microcode form the TCB <janneke>ah, great...but in that case, can't mescc-tools-seed just be merged into stage0? <OriansJ>and use a design which everyone can audit <janneke>we only have the first steps of the real stage0 done, but hey <malina>when you say that orians, do you mean prior to a bootloader or? <OriansJ>malina: I mean there is no bootloader, no bios, no firmware, no microcode, nothing but circuits <malina>ye, ok, then I understood correctly. <malina>wow. good ol' days eh,p well, at least you won't run out of stuff to do <OriansJ>malina: eventually I will but it'll be years before then <OriansJ>but first I have to knock out the guix bootstrap binaries for good. <malina>ya, but if you are up for it, hey, why not. Genuinely think this hand edit -> back_to_modern_tolchains is part of comp history. <OriansJ>It doesn't matter if I am up for it; I'll do it anyway. <OriansJ>All of the impossible problems of bootstrapping will be revealed to be just boring work that takes time and patience. Then people will have no excuses going forward, except admitting it is just their own laziness. <malina>time and interest are not invariantly inversely proportional to laziness, even if often it can be. <malina>but I agree. having a bootstrappable normalied path , which can be used at least ONCE by any donkey <malina>then those who dno't care can do it anyway but it won't matter, it' snot that is they would know any better.. but to many of those who DO CARE, <malina>part of a necessary paradigm to ensure software integrity/veracity at least. then it's dealing with metal foxes and THEN maybe, well by then we have machiens killing us mostly, google renamed to skynet probably <malina>and what not but with a bit of luck, our comps will be decks and will be the onyl way to ensure no skynet binaries have kill circuits hidden on deck. o'hoi haha,sorry weekend blabber <OriansJ>malina: there are already kill bots; but it is the actual software that makes it so. Ironically they are the only people in the DoD that actually are paid full time to working on bootstrapping. <malina>oh ye, firmware trojans to deliver payload sby yanks always occured in the 50s as you might know, already then they blew up poor Russian oil plant after trusting to buy some computers fomr yanks <malina>llaugh on them, who would buy any computer equipment now from the US and expect it to be benign? LOL <malina>(the irony being of course, most do and don't care) <malina>and, it includes us because, there realyl isn't much choice, is there? <OriansJ>checkout the video "Field Stripping a Weapons System - Building a Trustworthy Computer" youtu.be/lI6u_fuAP1g <malina>that's why stuff like this, no matter what the driving force is, is crucial to some form of resistance and hope <malina>WIll do, but not gonna lie, as with everything in life, one cannot escape a closed system (if one can' it wasn't closed in the first place; just a restricted subset model).. anyway, it follows then, software can never escape the limits of the hardware (and not circumvent various <a,b,c,beep,N> ) things here. <malina>as long as cpu chips are black boxes, with the creators also holding control of the software(OMG LOL) (firmware/microcode updates) then ya, <malina>I never feel my computer is mine <malina>not with the countries like US around <malina>but anyway, again, it is what it is, and we can at least work on software part _anyway_ because IF politicans get their fists out of their arses, they MIGHT still do some very simple things, as they should have done decaes agp <malina>as I even implored them, and that is to <malina>ye libresilicon.. I mean, it's a shame it's even needed inthe first place <OriansJ>malina: it was always going to be needed <OriansJ>open cooperation always beats secrets and lies in the long term. <malina>dewepoaninsing human civilisation so that c*nts pardon my Gaelic, like the US instead had spent alll that money on say collaborative space exploration rather than illegal weapons etc, hell, we'd have moon bases now, no real need for libresilicon, and don't know, pretty pink elephants flying by in our dreams <malina>but alas, instead we're stuck hand coding hex or hand holding private bits, dunno, depends on how skilled one is. <OriansJ>malina: blame never heals the sin; it only makes one feel better for not commiting it and not doing what is required to heal it. <OriansJ>people always do what they believe is in their best interest. <malina>ye, I guess since I only live once and I did the wise stuff once, I'm past it now, now I am just a misanthrope, and if I lived in a lul country to g and buy guns out of the blue, I'd happily go on rampages too, I'd prolly go for politicans ahead of kids though but hey. <OriansJ>It is just sometimes their assumptions are wrong and thus their choices work against their best interests. <malina>^profund as hell too.. anyway j/k but no profoundness is needed anymore, just action. <OriansJ>malina: what do you think picking up a text editor is? <malina>until then, well, the faster humans are wiped out, the better I suppose. in the meantime, we can make pretty code, sure, we prolly are borg anyway so it doesn't matter, as we are justa poisonous pest. /sunday_rant_endish <malina>that's hat I pick up when I make notes :D <OriansJ>malina: I pick up emacs or vim and write code to address problems. <malina>but in past years, I been losing my pens, and so I use more and more 'text editors' . I don't feel it's anything near an actual notebook and hanfdwriting though. <OriansJ>malina: what does that have to do with bootstrapping or fixing problems other people have? <malina>fixing problems, I just implied it wil remain sympomatic rather than causal. The notebook was just a question and notice on digital vs analog notes. <OriansJ>malina: what work interests you the most? <malina>very broad question, not sure I'd have any concise answer actually. <OriansJ>malina: within the bootstrap problem space <malina>well, I moved from my old distro once and built my current one from LFS, so a 'bootstrap' from existing environment, i.e. my old distro. As I did rebuild cycles, I added more 'bootstrapper' binaries, which are either A. not open source or B. open source. however, the open source are not si _either_ unless we have a recursive base, i.e things like edmund showcased with selfcompiling compiler which could keep clogging on new bits and bobs. And <malina>then you guys came in, and all this to me, is essential in terms of the universe S, where only software is concerned. the path from 0 to hero, and being able to use our modern computers to bootstrap. <malina>I don't know and don't have a depper knowledge than that, I just wanted it a long time ago, spoke to Edmunds about it as I say, this was long ago long before this stuff it seems, but somoene came along and did the work, you guys. great! <malina>regardless of my bleek realisation of those things I no longer can consider myself part of or vice versa, matters not, we still plonker on in the hope that it helps and/or will do so. <malina>as long as we end up with a source-file.hex | printv/tr manually convert it to a hex binary or if one is content with said seed , then from there to get to a $cc whcih can build tcc -> gcc and/or whatever target CC one desires, one has succeeded in giving people the power and gif to verify their toolchain is valid, (disregarding any hardware hoodlums now) <malina>so ye, that's all. I just see it from my distro maintainer perspective and my desire that users should have this right, especially given evil/facist countries say, the right to do this. <malina>I don't have any 'concise' answer, personally, sorry, but that is what is my original 'desire'. <OriansJ>ok now think of it in terms of working together. What work needs to be done that you are willing to do. <malina>I am not sure, because if anything, I would have to look (and I am looking and trying to see what is the differences between your seed and the examples by edmund, I use first name because I always forget 2nd). <malina>anyway, I am not at all good at assembly and so on, so I have to look i tup again. but therein is the problem. unlike your notion of laziness, to me time is the main problem <malina>as a maintainer I always end up doing 90% maintenance and always lament how little 'developing' I get to do. <malina>I still need to look into your sources I haven't so far too much. <malina>I am waiting for some guy who first kept moaning yest to bring down some price on hardware just to say he can come today and what's your addy, phone, etc etc just to have him not show up <malina>making me possibly a bit cranky today <OriansJ>malina: ok so audit and review of the existing code. Good <malina>possibly. I do have a little question here if ok. What is the difference between the Edmund's bcc and your hex0-seed, is it some extra definitions, a kina continuation of bcc.. and what would I have to do to get from his bcc to your hex0-seed say? (or can I infer that from the .hex source file)? <malina>I probably can but just was curious. I haven't looked at such things in ages. <OriansJ>malina: ok in short: I read and write to files not just stdin and stdout. hex0 I support ; and # line comments. hex1 in addition I support forward and backward references. hex2 has no match in bcc because it is long label, absolute addressing and pointer relative calculations %label>label. M0 is radicially different than what is in bcc as it allow arbitrary line macro definitions and cc_x86.s actually compiles C code into M0 <OriansJ>assembly. Thus all of the later stages are entirely different. <malina>ok. thanks. ye, my aim would be to make sure I feel I grasp your seed by analysing it, once I do, then I belong to the intended audience of 70% and all is good :D <malina>I'll try this week to find time to catch up a bit on this <malina>hex0_AMD64.hex0 once I grasp and vet this, basically and I am happy to conclude it's the same as the seed, I'm good, right? <OriansJ>malina: well you might also want to check the kaem-minimal.hex0 if you want to not trust any shell <malina>I don't distrust my shell as such. Of course, the whole point of this is we can't know unless we manage to break free from the chicken/egg cycle. Now, if you _truly_ one day get to do that full metal bootstrap LOL.. then you'll have cracked it. Until then we will always be to some degree bound by the environment, so if kaem-minimal helps on minimisg that, ok sure. I'll work with it too. but I <malina>ye, my init is simple, but ye. What I love of a true foreshadowing/foresight, was Ken THompson's OWN note in his talk in '80s about don't trust the kinda compnaies hwo hire me <malina>and sure enough, heis hired by google now <malina>couldn't have hit the nail more sensibly, even if he worked at Bell then. <malina>also think it's a bit unfair by him to use it against Bell, even if he understood the potential, since they/him helped to bring Unix and thus linux to tge game. With that said, he might have known , seen it coming (NSA, CIA, Yaddi ya far more) than we do/did. <deesix>OriansJ, so the new M2-Planet flag is like the 1.x vs. 2.x series that you had in mind? I'm wondering which value should be the default (wrt breaking backwards compatibility or not). <deesix>As for the renaming, organization and [de]duplication of parts, I'm not sure what to do (I'm sure missing the big picture) but that'd help. <deesix>Maybe bootstrap-seeds is another candidate to merge up with stage0. <janneke>certainly not, the seeds should be nice and separate, imo <deesix>The seeds or the source code of them? <OriansJ>deesix: well I want to keep sources with binaries for easy proof <OriansJ>so we never distribute just binaries <AS196714hotmilf>hello everybody, i'am new to this channel, but a mindful reader since ~2 years <AS196714hotmilf>i do some documentation to the scripts / code to understand it better and <AS196714hotmilf>i have programs understanding the step 'mes.hex2' -> mes.bin -> mescc <AS196714hotmilf>or better: is there some documentation, how i can compile the patched 'tcc' with mes? <OriansJ>well janneke is the one most familiar with the MesCC bootstrap pieces including the building of TCC with MesCC <OriansJ>but I believe fossy has some work which should help with that understanding in progress <janneke>they would have to change their nick though <AS196714hotmilf>janneke: also thanks to you (and everybody here 8-) for this unbelievable project <AS196714hotmilf>my proposal is: make my documentation public, so i can better say what is missing in my opinion (hopefully its not missing, but somewhere lurking around and i have to dive deeper) <janneke>AS196714hotmilf: please change your nick <AS196714hotmilf>sorry, it's my nick in #DENOG (autonomous system 196714 admin) let me see, how i can change it channel-wise ***AS196714hotmilf is now known as gforce_de1977
<janneke>gforce_de1977: thank you, was already calling in ops for spam alert <janneke>i'm sure you understand if you're legit <gforce_de1977>OriansJ + janneke: thanks for the link, will read and ask what i can not understand! <OriansJ>anything not well understood is a bug that needs refinement <gforce_de1977>OriansJ + janneke: very good idea: i will make a talk in our local hackerspace and upload the slides. during talking one can have a better feeling what is not clear yet <deesix>Well, I don't know, but the current maze of directories, projects and submodules is hard to grasp. <deesix>stage0 contains mescc-tools-seed and bootstrap-seed as submodules <deesix>mescc-tools-seed contains M2-Planet, mescc-tools, mes-m2 and bootstrap-seeds <deesix>Then, the stage0,1,2,3 dirs; stage0/x86/ has stage0 and 1 dirs. <gforce_de1977>deesix: because of the maze i started to do my own documentation with the goal to share it. I still have not understand every bit and every step and at the moment have a working executable 'mes.bin' and have no idea which *.scm i should provide to compile the patched 'tcc'. so: how it the commandline? <deesix>Then, mescc-tools-seeds branches on arch dirs and no notion of stages... <deesix>gforce_de1977, I'm afraid that if I'm understanding something is M2-Planet and below. <gforce_de1977>everything including M2-Planet and below is ofcourse the most sexiest part. but in the end we want to bootstrap tcc -> gcc, so thats also important <OriansJ>deesix: well that is ultimately my fault. As I have been mostly been glueing pieces together as needed. <OriansJ>as people seem to have trouble following the steps git clone these three things to run. <OriansJ>So I use submodules to include everything at commits I know work together. <janneke>OriansJ: ah, mescc-tools-seeds is a git submodule of stage0 -- so my intuition that mescc-tools-seeds "is" stage0 is right <deesix>gforce_de1977, sure; just saying that I'm not very familiar with that part yet. <OriansJ>mescc-tools became seperated to satisfy janneke's needs for MesCC and getting a simple guix package. <OriansJ>M2-Planet was mostly an ugly hack done in 20 hours that kind of took off on its own. <deesix>Do you think there's a clear split at the "top level", what would that be? stage? arch? arch+kernel? triplets? None of the those? <deesix>I mean, not actually but in some "ideal" structure. <deesix>With room for future developments (more arch, etc) and reducing duplication as interesting goals (lacking a time machine, of course :P). <OriansJ>deesix: probably at its core: bare metal / POSIX / DOS / etc <OriansJ>then if needed individual steps (but probably not) <OriansJ>but once one gets to the M2-Planet + mescc-tools stage; everything else is universal <deesix>yt_, about your note in LOAD_W0_AHEAD. Maybe the rest of the set needs to be ldrsw too (and all of them being about the 64b register even if the data is just 32b). <yt_>OriansJ: in my opinion, I think the git submodules, especially in the stage0 repository, are generating confusion about what stage0 *is* and what its dependencies are (and even which things constitute a project or package on its own) <deesix>Most of the times we are loading absolute addrs with LOAD_W0_AHEAD, but there's a need for CONSTANT and other values, to be sign extended. <yt_>deesix: hmm, I'll have to look at it again, but it might be that we need separate LOAD_SIGNED_W0_AHEAD and LOAD_UNSIGNED_W0_AHEAD, or insert an sxtw after an (unsigned) LOAD_W0_AHEAD when necessary <yt_>For absolute addresses it doesn't ematter as long as you keep inside the lower 31-bit of the address space. Currently you need to stay inside the first 32-bit anyway, because we don't have 64-bit literals anywayy <deesix>yt_, yep, we're not supporting 64b for a couple of things. I need to check every use (plus the W1, W2... ones), to see if we need both kinds. I feel that other archs in M2-Planet are fine with one kind but, again, I need to compare. <deesix>yt_, thanks. About a preprocessor for M2-Planet, do you have a subset in mind? Are you going for the full thing (feels kind of huge). <yt_>deesix: I'll start simple, but I think I do want nested expansion <yt_>I'll probably leave out varargs, but that should be easy to add later without too much fuss <yt_>can you do float arithmetic in a macro #if !? yeah, no, not doing that :D <yt_>TBH, I'll leave out most except what you need to build some reasonable boolean expressions, e.g. #if defined(FOO) || defined(BAR) I want to support <deesix>Hmm, it seems that the constant expressions must be integral, indeed. <yt_>deesix: so thinking about LOAD_W0_AHEAD, could we just byte the bullet and add LOAD_X0_AHEAD and use 64-bit constants for absolute addresses? <yt_>entirely unintentional, funny enough <deesix>I did a similar joke in some doc, IIRC. I think, for LOAD_X0_AHEAD (plus a skip64) we need support from hex2 (maybe using the ? arch-specific symbol). Not sure if we need it but it's an idea to consider. <yt_>deesix: M2-Planet could generate the two 32-bit halves as separate constants, but I agree that true 64-bit constants in hex2 would be cleaner <yt_>and would benefit amd64, as well as mescc <deesix>yt_, oh... yeah, I think I saw some of your code doing that somewhere. <yt_>deesix: I may well have. mescc does it as well in some cases <deesix>Another thinkg on the back of my mind is to get rid of the "middle level" naming and go for direct asm-like naming. But in some places feels kind of nice to have (push/pop for example) but on the other is causing problems (push/pop too, ironically). <janneke>stikomas: you're looking at wip-full-source-bootstrap? <janneke>looks like it; yes there are problems with tcc-boot/tcc-boot0 and the new mes-0.23 (to be released) <janneke>stikonas: i have been looking at that all day and am about to push a revert and two terrible HACKs <janneke>we need a proper fix, but first i need to understand what's going on <yt_>deesix: it depends on what you're used to, I think. if you're coming from an Armv7 background, the push/pops make sense, but in Armv8 land you really want to see ldp/stp and sp/fp accesses with offsets <janneke>that may take some real time, which i don't have atm, so... <yt_>but all of that is difficult in M1 because of how those instructions are encoded and we can't insert those constants into the encoding easily <stikonas>well, no problem, I was just wondering if it's something that I'm doing incorrectly <janneke>stikonas: thanks for testing this and reaching out <yt_>my preference would be on least suprise for someone with an AArch64 background, but there are also a *lot* of people with an AArch32 background from the embedded world <janneke>i was very optimistic yesterday and only built until tcc-boot <janneke>everything onward is "known to work", right ;-) <stikonas>I also made the mistake of running guix gc before that, had to use ntp hack... <deesix>yt_, absolutely. The current situation is me not having that background at all, but with your developments I started to see that people knowing better would be at home with something more standard. <deesix>yt_, so those definitions reaching outside M2-Planet scares me a bit. <yt_>deesix: outside M2-Planet? meaning mescc-tools-seed? <deesix>yt_, yes; also I think OriansJ used them somewhere. <OriansJ>well janneke uses M1 defines in MesCC too; although he wrote his own to look more like GAS assembly <OriansJ>So honestly the goal with any DEFINEs you make is to save yourself trouble in 6 months when you forget what they previously ment. <OriansJ>as in 6 months you likely will forget because of other intersting work in the mean time. <deesix>The problem with the actual meaning is that its context is M2-Planet, but used elsewhere... <OriansJ>deesix: well not for long assuming I get M2libc in order <yt_>deesix: that would be my fault :) I used the M2-Planet AArch64 DEFINEs as inspiration for mescc-tools-seed <deesix>yt_, that's kind of expected, but I fail to realize it back then. <deesix>OriansJ, yes... M2libc seems to be an oportunity to standarize from M2-Planet up. <yt_>deesix: it was very helpful to get started with mescc-tools-seed for AArch64. Maybe decoupling the AArch64 DEFINEs between M2-Planet and mescc-tools-seed would be a good thing <yt_>(I haven't followed the M2libc discussion, so I don't know how that affects this) <OriansJ>not much of a discussion. It is mostly going "I hate having to fix things 5 times" So lets do most of the work in C and then I only have to fix it once. <yt_>OriansJ: I think I got that :-) I don't think I've grasped yet what that would look like in practice <deesix>yt_, yeeeah, During review I found it helpful to compare both sets, and expecting to be able to merge them. But decoupling started to feel more practical, somehow. <OriansJ>yt_: a git repo with mostly C standard libraries written in M2-Planet's C subset and 2 architecture specific files. <yt_>deesix: especially as M2-Planet's macros are going to have to evolve as more features are added (see the unsigned/signed arithmetic for example), while mescc-tools-seed should not have to go along with that <OriansJ>basically a unistd.c for all of the primitives that need to be in assembly and a bootstrap.c with only the bare subset needed for M2-Planet being bootstrapped by cc_* <OriansJ>probably with *.h files and externs to enable to GCC to also use the library when building M2-Planet <yt_> OriansJ: that looks a lot like what janneke has floating around in Mes already <OriansJ>yt_: indeed, it'll unify with janneke's libc needs too <pder>OriansJ, is the idea that any project would use M2libc as a submodule? <yt_>OriansJ: nice. could some of janneke's work there be reused for M2libc? <OriansJ>including blynn-compiler so we don't need to duplicate efforts <OriansJ>yt_: Possibly but I didn't have time to check yet. <yt_>OriansJ: fair enough, I did part of an AArch64 port of janneke's libc (still waiting for work to approve upstreaming :( ), and my gut feel here is that it is pretty much all you'd want M2libc to be already :) <OriansJ>send a pull request to M2libc and I'll merge it and we can go from there <yt_>hah, I would, but I'd have to clear it with work first <deesix>Hmm, """If the constant expression is required to be integral, its operands must consist of integer, enumeration, character, and floating constants; casts must specify an integral type, and any floating constants must be cast to integer.""" <deesix>Aha, but when casts are not allowed, the floats are out. <janneke>stikonas: i have updated wip-source-bootstrap; it's on its way building gcc-7.5.0 here, so i have good hopes that "it works" <janneke>as i earlier, i applied a revert and two hacks that i will need to cleanup some time <deesix>Output from stage0/stage2/cc_aarch64.s is the other usage of AArch64 M1 macros I was thinking of. <deesix>Even stage0/stage2/High_level_prototypes/cc_aarch64/* <yt_>deesix: I'm not sure what those are used for <deesix>yt_, that was 2nd Ed. of K&R A.7.19 and A.12.5 <deesix>yt_, AFAIU, those are the Knight version of cc_* for AArch64 and a prototype of it in [I guess the M2-Planet subset of] C. <deesix>So, to properly answer: ANSI C but not directly the standard. <yt_>deesix: gotcha, looks like the committee draft I'm looking at is roughly C99, which seems to restrict to integer constant expression: """The expression that controls conditional inclusion shall be an integer constant expression except that ...""", which is good enough for me :) <deesix>yt_ indeed, and for gcc is a solid thanks but sorry, no: floating constant in preprocessor expression ***nore_ is now known as nore
<stikonas[m]>ok, I think this time wip-full-source-bootstrap guix branch will work. It's now building gcc-cross-boot0-7.5.0... ***ChanServ sets mode: +o rekado
<janneke>stikonas[m]: i found the problem with the updated tcc-boot0, so i'll be resetting wip-full-source-bootstrap (saving the "working version" of course ;-)