<OriansJ`>fossy: thank you, I'll be checking shortly <OriansJ`>markjenkinsznc: I'll be doing a full manual audit and then commit the work for M0-compact.hex2; with only the addition of your name as a coauthor and a few minor whitespace tweaks <OriansJ`>fossy: the line: require(token->next == NULL, "INVALID set COMMAND\nABORTING HARD\n"); you probably ment NULL != token->next as require exits failure and displays the string if the expression is false <OriansJ`>and the comment about M2 above it is no longer required <OriansJ`>and NULL != token->next->value right below it <dddddd>The ../M2-Planet/ references are still in the makefile, it doesn't build. <dddddd>cc1: warning: unrecognized command line option "-Wno-int-conversion" <dddddd>gcc (Debian 4.9.2-10+deb8u2) 4.9.2 <dddddd>... and lots of warnings that I guess are the ones that flag hides: <dddddd>warning: assignment makes pointer from integer without a cast <OriansJ`>hence why it hasn't been propagated to savannah yet <dddddd>Are you rewritting to get a buildable project on each commit or just adding new commits with fixes? <dddddd>Why not wait to fossy to fix it in a proper way? <dddddd>This sloppiness only lead to more sloppiness, and lack of motivation to do the things the "right" way, IMO <OriansJ`>I guess I am a little too kind to people learning, with the hope they notice what I fix and learn from it <dddddd>fossy, to be clear. I'm not criticizing your work but the integration. <OriansJ`>ok, kaem.c now builds correctly and passes its own tests <OriansJ`>ironically gcc needs fflush(stdout); to get matching behavior. <OriansJ`>so dddddd I guess we will need to make a fflush function for M2-Planet <fossy>dddddd, OriansJ`: I would much prefer to solve these issues myself... otherwise ill never learn <fossy>dddddd: I broke kaem out into a folder because I plan to split it out into multiple files <fossy>and kaem was passing its own tests.... <fossy>I didnt test with gcc 4, I was using gcc 9. <fossy>I was just about to fix the m2 planet things and rewrite history <fossy>I still think kaem should be in its seperate repository <OriansJ`>fossy: indeed; I'll create one and give you permissions <fossy>we still have the kaem repo on gh <fossy>but nothing on Savannah afaik <OriansJ`>hopefully you look carefully at my fixes <fossy><dddddd> fossy, to be clear. I'm not criticizing your work but the integration. <fossy>integration, do you mean like commits etc? <dddddd>fossy, the way your commits are accepted (or not) and integrated in the repo. <dddddd>OriansJ`, which file fflush would belong? <fossy>generally when I make a larger PR I make fixing commits and then when told that the tip is acceptable rewrite history <fossy>I dont know if thats a good approach <OriansJ`>fossy: invite sent; you'll need to export the commits to kaem.c from mescc-tools to get it back up to date but it'll be a good learning experience <fossy>dddddd: just to be clear are you saying the code is fine, just the process of review, merge and my use of VCS is done poorly? <OriansJ`>dddddd: I am guessing test/common_*/functions/file.c <OriansJ`>fossy: you need to be more careful how you commit <OriansJ`>the idea is that all commits should be to the best of your ability having the code in a known good state <dddddd>fossy, that approach is great IMO. About your code, I didn't looked at the details and the ideas behind the changes so I can't talk about it. But not merging several commits where the build is broken feels wrong. <dddddd>You wasn't given the opportunity to fix it (and end your process). <dddddd>All said, it's true I don't know if OriansJ` was aware of your plan. <dddddd>OriansJ`, I'll add a just "return 0" fflush for all archs. <fossy>yes, I need to treat my PRs as I would master <OriansJ`>fossy: remember everything is for the world to review and know the quality of your work. <OriansJ`>You have to walk the line between consistently making improvements while still reflecting on is this work good enough for others to see. <fossy>I tried to do that in my code <fossy>Didnt pay enough attention to git thougj <fossy>so OriansJ`, will kaem be removed from mescc tools? <OriansJ`>not exactly, it'll just become a submodule <OriansJ`>but I want you to get the kaem repo into shape first <OriansJ`>I want full history of changes to kaem.c imported <OriansJ`>You can fix your commits and rebase them too if you like <OriansJ`>but I want that repo as perfect as you can make it <fossy>so in future, OriansJ`, things that were in that commit you made I really would appreciate you including in your review to me, because all of those commits are logicial <fossy>but I like to have the opportunity to learn by fixing <OriansJ`>ok; first a code smell involving token_head <OriansJ`>it means that what you really should be doing is creating a local struct Token* n in those functions to iterate over the list <OriansJ`>you probably are doing the same thing with env_head and env_tail <OriansJ`>if you are iterating to call to a function which updates it; pass that single node to the function to update <OriansJ`>It only appears because you are doing a weird list thing in collect_token <OriansJ`>What you should be doing in there is calloc a string, create an index (int i) and set it to 0; And only if the index is greater than 0 create the new token, set its value, next and pos then update token with it <OriansJ`>in fact; I want you to do it only using single-link list <OriansJ`>you can copy M2-Planet's reverse_list from cc_reader.c when the token list is done and you need to reverse the order of the tokens <fossy>OriansJ`: thank you for the pointers <fossy>This is the first time I have used a linked list as a primary strucutre in a program <OriansJ`>struct Environment will be replaced by it too <fossy>Environment dosent need pos though <OriansJ`>It can either have a name set or a position set <OriansJ`>tokens_to_array and env_to_array are to be replaced by a function that works for both cases; using a list passed to it <OriansJ`>collect_token is to return an int; that int being the last char read <OriansJ`>the calling function is to check if it is an EOF and thus know if the script is done <OriansJ`>To eliminate the need for the global variable you currently have <fossy>but imo the code should terminate at the end of main, if successful <fossy>that was my idea with that global <OriansJ`>run_script should return when you hit the EOF <fossy>function prototypes, why remove those <fossy>OK, now we have only kaem.c in the repo <fossy>Now lets integrate the other repo back in <OriansJ`>fossy: why have function prototypes at all? <fossy>OriansJ`: in case a function is called by another function thats code is before it <OriansJ`>and your do whiles are wrong; you mean while(NULL != n) as those loops will do the wrong thing if the list is NULL and you'll get a segfault <OriansJ`>fossy: yes but that doesn't exist in the current code <fossy>But there is no good reason to have them in the current code <OriansJ`>fossy: the future will include things you don't know about yet <OriansJ`>So plan for being wrong by default, it makes getting into a happy state easier <fossy>Why did I put so many fix commits in <fossy>why are my commits so undescriptive <OriansJ`>change cd to recieve the full token list and move the requires into cd; then you can have all of your builtins as just a list. <OriansJ`>You'll need to copy M2-Planet/gcc_req.h to get gcc to like it <OriansJ`>Then you can create an init function for the builtins to allow easy expansion <OriansJ`>fossy: also give flycheck-mode in emacs a chance <OriansJ`>The only thing that it says you can ignore is the scope reduction of variables (it is good advice for GCC but doesn't quite work for M2-Planet) *fossy does not use emacs <OriansJ`>also you don't need to copy strings, just copy the pointer <OriansJ`>for example in variable_substitute_ifset <OriansJ`>you could have 2 char* (name and alternate); walk the string until : and set to 0; then have the alternate point to after the - <fossy>please if you find more things keep going even if I dont respond <fossy>with correct blaming and the lot <fossy>OriansJ`: thank you for putting up with my not excellent code ^-^ <dddddd>I'm always a bit confused about what libc means in M2-Planet context. Is it 'test/common_*/libc-core.M1'? What about 'test/common_*/functions/*'? If that's the case, why is the libc in a testing dir? And 'functions/' on the root dir? If all this is historical, what would be a sensible way of organization? <fossy>^^^ IMHO test/common* should be moved <fossy>dddddd: you too thanks for putting up with my not super excellent code ^-^ <dddddd>fossy, no problem... my pleasure! I saw it as WIP code and, as such, is not always the best. We all learn from each other, thank you! <OriansJ`>dddddd: well technically M2-Planet has no libc; because it doesn't need it. The libc-core.M1 is just to setup the stack for main and call main and be done. ARMv7l includes division and modulo because it lacks the instructions required for a proper host of M2-Planet <OriansJ`>but you are correct, putting them under test might not have been the wisest of ideas <OriansJ`>perhaps it is time for M2-Planet to create a folder called libc and move functions into it <OriansJ`>probably with an explaination file expressing that M2-Planet doesn't actually have a libc but rather a set of commonly useful functions <dddddd>Sound like a plan. I can move out of testing all those files and adapt the scripts. I'll probably ask about some details about what of 'functions/' should be moved/copied, if any. <fossy>OriansJ`: also, maybe create symlinks in test for backwards compatibility? <fossy>because a TON of our scripts use test/ <dddddd>If M2-Planet is the only user, I guess we don't need that. What's using M2-Planet files directly? <fossy>mescc tools seed certinaly s <fossy>and many other m2 planet programs use them, but they may be mirrored in the repositoey <OriansJ`>dddddd: all of the functions will need but we will need seperate folders for the different architectures inside of the libc folder <OriansJ`>we probably could keep the structure and use sed to fix all of the references <dddddd>No doubt OriansJ`, about dirs per arch, with non-arch files on their own. <OriansJ`>we can fix mescc-tools-seed when we update its submodules <dddddd>So, not being really a "libc", should we call the dir just "lib" (or something else)? <OriansJ`>probably put the common functions just at the root; unless you think a folder called common would work better <OriansJ`>lib is fine; library if one wanted to be verbose <dddddd>This and your recently test realocation shapes a beautiful future release, I think (: <dddddd>I'll think about root v. subdir for the common/non-arch files. <fossy>OriansJ`: once I fix kaem and it is in a much nicer state in the new repo how do you feel about a v1.0 for kaem? <OriansJ`>I think fossy it'll be ready for v1.0 when it can be used to build make <fossy>I think it does deserve some kind of release though <OriansJ`>but yes, it is certainly worthy of a release <fossy>with a wide variety of new featurea <OriansJ`>how does kaem v0.8.0 sound to you fossy? <OriansJ`>as its version was synced with mescc-tools <OriansJ`>and mescc-tools sync'd its release with support for new architectures or breaking changes <fossy>how do you think about VERBOSE outputting post variable expansion? <fossy>^ that is the git log right now <fossy>diff is the same between mescc-tools master and kaem master <fossy>next up: testing each commit is buildable, and if not, fixing that, and making commit descriptions better <fossy>stage 3: code fixing and rebasinng <fossy>hmm, OriansJ` dddddd which order would be better to do those in? <fossy>just trying to figure out the best way to approach this <dddddd>fossy, for those 7 commits, I guess I'd make new commits from a squash, adding descriptions and checking the build as I go (git stash is your friend here). Like your final "rewrite" step for PR, I think. <dddddd>Not sure it's a good idea to touch earlier commits. <dddddd>fossy, what's the plan for OriansJ` fixes? Are those in the seven commits already? <fossy>dddddd: I plan to touch my envar commits. <fossy>Everything earlier I have verified is A-OK <fossy>dddddd, OriansJ`: git log with lots of descriptions ttps://ttm.sh/EA6.txt <fossy>now OriansJ` would you prefer to be to force push the rewritten history before i do all the code fixes <fossy>or would you prefer just one force push <fossy>Whats the general consensus with bugs found from fuzzing? Should I add those into a test to prevent regressions or not? <fossy>oh hey janneke havent seen you for a while <OriansJ`>fossy: I honestly don't know if there is a consensus on with bugs found from fuzzing beyond, fix the bug and add asserts (require in our case) to prevent it from occuring. As tests are more to ensure the expected behavior occurs <OriansJ`>so janneke what fun have you been having? <janneke>OriansJ`: i'm having fun with the Hurd atm <janneke>and the new python dependency in the bootstrap <janneke>and, of course, getting the scheme-only bootstrap in core-updates to be really OK over other architectures <OriansJ`>why add python as a dependency in the bootstrap? <janneke>*please* -- please, please, pretty-please -- ask that the glibc folks <janneke>or better, rewrite their python scripts in (guile) scheme ... <janneke>i haven't dared to really look into this yet <OriansJ`>So nothing needed before we have GCC 4.7.4 <OriansJ``>but something we will need to address for the Hurd port <OriansJ``>is micro-python an option for MesCC to compile? ***OriansJ`` is now known as OriansJ
<janneke>indeed, it's waay off our current radar <janneke>which means that raising awareness now can help <janneke>python-on-guile starts to look real nice too <OriansJ>unless MesCC expands its functionality to be able to compile guile and GCC directly <OriansJ>or janneke we could move all the steps prior to GCC 4.7.4 onto a seperate POSIX so that we don't have to deal with those platform level problems <OriansJ>or expand mes-m2 to also run python-on-guile <OriansJ>which would make mes-m2 the most important program in the entire bootstrap in a big way <OriansJ>as it would become the guix's permanent bootstrap binary <OriansJ>as it would have to support most of guile2.x's functionality; which is a tall order. <OriansJ>I guess the question becomes, how ambitious do we feel? <janneke>hehe, asking that question is answering it :) <OriansJ>So stop me if I get this list wrong but we are planning to: <OriansJ>1) bootstrap every hardware platform in existence and will ever exist <OriansJ>2) Do so with only the handful of people here who are able to put in a few hours at a time <OriansJ>3) write a brand new POSIX buildable via M2-Planet that works on all of the above <OriansJ>4) do custom hardware with the above <OriansJ>5) port the work onto every operation system we can imagine <OriansJ>6) solve a dozen or more open problems in computer science while doing the above <OriansJ>7) get it done within the next couple of years <OriansJ>somehow I feel ambitious is just too modest of a word to describe it <OriansJ>oh and 8) bootstrap all software in Gnu and the free software world <OriansJ>Which will include bootstrapping Guile, Haskell, SML and more than a dozen other programing languages along the way <OriansJ>I've got dddddd helping me on Slow-utils and M2-Planet, fossy helping me on kaem and janneke helping me on MesCC and guix. We need more help <OriansJ>possibly bauen1 to help with a POSIX, markjenkinsznc is helping me with the stage0 bits <OriansJ>I have mescc-tools/cc_*/M2-Planet ports to do for RISC-V, PowerPC, z80 and 6502 <OriansJ>Oh and I have to add in optimizations into M2-Planet to enable it to fit on 16bit targets... <OriansJ>Which means I will have to reengineer M2-Planet to enable that to be done in 16bits with a seperate programs to act as specializers <OriansJ>janneke: perhaps we should approach this in a more conservative way. <OriansJ>janneke, dddddd, fossy, markjenkinsznc and bauen1: tell me if you are onboard with the following next tasks. <OriansJ>janneke: first goal reduce guix down to just guile; then you are to work on the scheme side of mes-m2 (if the C side needs features or changes I'll deal with them) <OriansJ>dddddd: Finish slow-utils so that janneke can eliminate all binaries except guile from the guix bootstrap <OriansJ>fossy: keep progressing kaem until it is able to build make and then we can discuss your next goal (unless you wish to help on another piece) <OriansJ>markjenkinsznc: once the stage0 disassembler.py is ready, I'll pull the extra null padding from the stage0 binaries and we can start figuring out how to get cc_* to fit in 16bits <OriansJ>bauen1: if you are still up for it, get your POSIX able to run the steps in mescc-tools-seed and then make your POSIX buildable via a single GCC command. I'll help with the M2-Planet conversion after that. <OriansJ>questions, comments or things I stupidly missed? <janneke>i'm still working on finishing the scheme-only bootstrap <janneke>next up is removing mescc-tools and mes from the guix bootstrap by using the mes-m2->mes merger that i prototyped last october; where i branched mes-m2 just before slow-lisp was added <janneke>when that is done, focus will shift to the scheme side of things <OriansJ>janneke: you will not need that step <janneke>and also to seeing what to do with your mes-m2 work since october (that is really giving me headaches :-) :-) <OriansJ>mes-m2 is going to grow into a full guile2.x drop in replacement <OriansJ>So getting guix down to just guile will make mes-m2 a single step replacement job <janneke>i understand, but i like to stay on the current stepwise approach for the guix side <OriansJ>guile needs to be properly bootstrapped and mes.c just can't do it <OriansJ>mes-m2 is going to have to be a much larger scheme than originally planned to solve the guile bootstrap problem <janneke>indeed -- but we only use guile for the builder and for running gash <janneke>as soon as gash runs on mes, guile is only used for the builder <janneke>that means that fossy or the nix bootstrap do not have to introduce guile <janneke>but...things may look different next week, of course :) <OriansJ>So you want a shorter time frame for mes-m2 to be able to run MesCC and gash? <OriansJ>The only big stopping issue for mes-m2 to run MesCC is support for modules <OriansJ>once that is done, the rest is rather trivial to do <OriansJ>It is possible to add that support entirely via scheme, since mes-m2 supports keywords <OriansJ>I was planning on converting mes-m2 to Continuation-passing style form this week; at which point it would be at virtually feature parity with mes.c <OriansJ>as you can see it is rather trivial to add primitives and major architectural features to mes-m2 <OriansJ>so if you wanted MesCC to run on mes-m2 by the end of this month, it is possible for us to be done with that <OriansJ>Not on the C side but on the scheme side <OriansJ>as you are a much stronger scheme coder than me these days <OriansJ>figure out the minimal version of module support needed to solve the MesCC problem and I'll build any C hooks you need to get it done. <OriansJ>I'll be adding more primitive tests to mes-m2 to ensure all the other primitives you leverage will exactly match that of guile in everyway <janneke>OriansJ: no, there is no hurry and i don't have much time to spend on this <OriansJ>then how does spending time on code that M2-Planet can't build; help the situation? <OriansJ>if it is because that is the work you were paid to work on, fine; I get it, everyone needs to be able to pay their bills. <OriansJ>But unneeded duplicate efforts are not exactly something a resource constrained project benefits from. <OriansJ>Please help me understand because honestly I don't see the benefits given the assumed goals <OriansJ>Which where outlined and numbered above <markjenkinsznc>I need to test it with a proper script against all known binaries, but so far manual testing shows success with disassembling and re-assembling stage0_monitor and M0-compact, so that's a pretty good sign <markjenkinsznc>That disassembly shows how aggressive my disassembler is, even at the end of binaries where long instructions no longer fit, my disassembler keeps looking for short instructions, hence the 'wrong' ADD.CI R0 R2 R0 R0 at the end and some 'wrong' NOP in this current output sample. :) <markjenkinsznc>Like GNU objdump, I'm going to have cmd line options for marking certain memory areas as data. <markjenkinsznc>After finishing testing I'll have more to do to achieve feature parity with disasm.c, options to have addresses in output, string detection, original hex as a comment, more inteligient choice of format for immediate values. <markjenkinsznc>My disasm.py is kind of long and ugly, but partially because I've laid groundwork for adding those features, they'll be pretty easy to add from here. <markjenkinsznc>Not very hard either to add support for disassembling hex0, hex1, and hex2 files as well and retaining the human elements of labels, use of labels and comments <OriansJ>I guess we can plan on shaving off the extra nulls in the knight bootstrap sooner than I thought possible <OriansJ>that will simplify M1 across the board <dddddd>janneke, what do you mean by "this"? *dddddd re-re-reads the log <OriansJ>dddddd: I am guessing the work of getting MesCC running on mes-m2 sooner <janneke>yes, hard-core bootstrap work as explained above; i'm hoping to return to that in one or two months time <dddddd>+1 on clear view of the duplicated efforts, to avoid them. What are we working on that is not really required? <janneke>i'm doing integration/guix work, otherwise guile-related consultancy and need to secure my funding for the past year and the coming year; which means some broadening, experiments atm <janneke>mes-m2 is moving too fast atm for me to put in the little time i have to spare; it always takes quite some effort getting into <janneke>on the other hand i'm very happy to see a better and more clear road forward than ever; even since last year <janneke>the hurd work is partially a digression/distraction but it could become interesting/important to play with reducing the trusted computing base <janneke>getting guix to run on the pinebook pro is important to help with the arm port; that needs to go into guix soon too <janneke>so, the past half year the project has really been broadening for me <janneke>then, there's also life happening over here <OriansJ>janneke: I get it; life sometimes gets busy and it is hard; believe me I really get it. <OriansJ>dddddd: guix if I remember correctly <dddddd>It seems important as IIUC it's the only thing that will require guile --so maybe the _only_ reason for mes-m2? *dddddd sometimes feels in a maze of twisting little passages <dddddd>My only hope for the time being is that at least you both (OriansJ, janneke) are [mostly] in the same page. <dddddd>But, honestly, I don't know if you are... or if conflicts of interest are in play, or whatever... <OriansJ>dddddd: well syncronization between people is a big open problem; so sometimes janneke and I get out of sync or we simply disagree about the best path forward and that is ok <dddddd>I'll keep doing my best and having fun, but I prefer to spend time on things that will be *used*. Of course the future is unknown but if things are changing from week to week... well... <dddddd>If that's the Hurd, so it be. Or that python/glibc port to scheme, or slow-utils, or RISC-V for M2-Planet, or mes-m2, or helping fossy, or Arm stuff.. but please let's optimize (as far as possible) for the priorities. <dddddd>What's the common ground, might be the question. <dddddd>If so... feel free to cherry-pick. <fossy>OriansJ: I am more than happy to help with that. <fossy>I am also more than happy to help continue to progress mescc-tools-seed <fossy>because those are the two things which I feel are important enough to be used <fossy><OriansJ> dddddd: I am guessing the work of getting MesCC running on mes-m2 sooner <fossy>Imho this should be a primary goal <fossy>But mes m2 is not my area of expertise <fossy>rationale behind kaem is primarily to help me in mescc tools seed <fossy>I find a cross platform bootstrap extremely important, that is not limited to GUIX <fossy>but I agree with dddddd. We need to be optimising for the priorities <fossy>We are literally on the last stretch here, and I personally would like by the end of the year to be able to git clone && kaem-seed and end up with a complete toolchain <fossy>but no offence OriansJ but I feel like getting kaem into a state that can run gash easily (which is almost done) is more important than getting it to compile make <fossy>there are many side paths we can take and other peoples time is not mine but I am very interested in a working bootstrap <fossy>and isnt that our real goal? <fossy>heres a duplicate effort I see: getting kaem to run make compilation and getting mes-m2 to run gash <fossy>since gash can now compile make <fossy>I dont think 2 Years is a reasonable timeframe for all those gosls <fossy>2) 3) 7) 8) I reckon we can do <fossy>but having the solid foundation of a bootstrappable C base I think is doable <fossy>but guys we arent in a great rush <fossy>our bootstrap map is complete we just need to automate it and fill in the missing links. Once again we are on the home stretch here! Then we can focus on hardware stuff, Haskell stuff, etc. It dosent matter if it takes a month or two years, just that we are making progress on each of those missing links. <fossy>Yes, we are being ambitious. But who would have thought that janneke and OriansJ could have done as much as that had in the past 4 years? We are moving closer slowly but steadily, lets keep on that track and keep communicating. <bauen1>OriansJ: i've just got kernel "threads" working, so after that i need to actually run threads in userspace and implement syscalls but i'm getting there ... <bauen1>i'll probably try to get tinycc to run, compile a program and have that program work correctly ("hello world") to iron out any bugs in my kernel <bauen1>and then i'll see how to proceed <bauen1>tecnically i would need sh, make, binutils and tinycc to fully bootstrap the kernel <bauen1>but it should be possible to remove everything but tinycc <fossy>tinycc + binutils -> M2-Planet + M1 + hex2 <fossy>but removing everything but tinycc is a good first step <fossy>lemme know if there anything you need from kaem <OriansJ>fossy: completely fair the kaem work is more of a hedge against having other platforms having to depend upon guile for their bootstraps <OriansJ>bauen1: sounds like a reasonable short term goal <OriansJ>fossy: also, in case you didn't realize. gash is a bash replacement written in guile. So kaem can't run it directly but only spawn it through a scheme interpreter. <OriansJ>I finally fixed the nested set! bug in mes-m2 and solved the similiar version which was not previously known about in quasiquote <OriansJ>Finally got to line 2005 of Ur-Scheme's compile.scm <OriansJ>it would be the line: (assert-equal (parse-string ";hi\n(;hi\n)") '()) <OriansJ>getting the error: ERROR: not equal (#\newline ()) <xentrac>apparently that string is parsing as a character literal for a newline <xentrac>or as something that starts with one <xentrac>does "(\n)" parse as an empty list as it should? <OriansJ>well the above test: (assert-equal (parse-string "\n()") '()) passes <OriansJ>yeah (assert-equal (parse-string "(\n)") '()) passes just fine <OriansJ>give me one minute to clean up the current patch of mes-m2 and you can verify it yourself <xentrac>does it also parse ";hi\n" as #\newline? <xentrac>I guess the standard Scheme terminology is "read"