IRC channel logs
2020-02-23.log
back to list of logs
<markjenkinsznc>I'll rework this test fully into makefile and make it generic so it can test all of M0-macro.hex2 with vm and vm64 , various prototype and C versions, and the future M0-macro-compact.s against vm16, vm, vm64 <markjenkinsznc>I'll also pay attention to where test artifacts should go as per other tests in makefile <markjenkinsznc>No one should worry that I'm potentially distracting Jeremiah with common interest in knight stuff when there is still work ahead on bootstrapping GNU/Linux on Linux+small ELF binary.. <markjenkinsznc>but the good news for anyone so inclined is I'm also learning scheme and prototyping my own implementation in a high level language (not yet published) and will be coming up with lots of feature test cases, including ones relevant to mes and guile compatibility <OriansJ>ooh markjenkinsznc you'll like this; current prototype .s version can do cc_x86.s in under 32KB <markjenkinsznc>if I move on this fast enough I'll be in time publish and share test cases that mes-m2 can use <markjenkinsznc>as I recall simple tests being desired as guile's suite is said to get hairy fast <OriansJ>markjenkinsznc: plus I have kernel work to do if bauen1_ is up for it <OriansJ>and a proper POSIX kernel bootstrappable from M2-Planet is a higher priority than mes-m2 anyway <markjenkinsznc>I have a bit more interest in mes-m2 as I'm not really that far from being able to run M2-planet and M2-planet compiled mes-M2 with knightpies on some exotic platforms with python not otherwise supported by M2-planet and cc_*.s family <markjenkinsznc>Though I have my doubts performance feasability while interpreting mes while interpreting knight while interpreting python, three kinds of interpretation at once could be crushing <markjenkinsznc>the other reason I see kernel work as being further in my future is I'm enamoured by the tccboot hack of 2004 (uses linux 2.4), so on any stack where there's just the means to compile tcc then there's the means to make an escape and create self-bootstrapable x86 media for Linux 2.4 compatible old comps <OriansJ>as when mes-m2 can run MesCC; it will solve the Linux bootstrap problem on all platforms upon which mes-m2 can run. <OriansJ>but that is why I feel the bootstrapping of a POSIX kernel is a higher priority. It gives more time for kernel development such that it'll be ready to build GCC and Linux by the time MesCC is done. <OriansJ>also these diversions are a productive use of my time as I am still stuck on a few points for mes-m2 (so I might as well make progress on other fronts) <OriansJ>fossy: string behavior in scheme (which I found in Ur-scheme as previously posted about here) and module support (Which could be done in scheme or C and I can't decided which is a better option) <fossy>you just beat me as the page loaded :P <OriansJ>you'll see the error message "(lambda foo ... expressions are not valid scheme\n" which you'll need to fix in Ur-scheme on line 1880 by replacing (lambda cmd (... with (lambda (. cmd) (... <OriansJ>(you can use guile to verify this change doesn't change the behavior of Ur-scheme, only making it more mes-m2 compatible) <OriansJ>I need to figure out a trivial enough test to replicate that error; so that I can properly solve that bug in mes-m2 <OriansJ>most of the issues in mes-m2 is just a lack of proper tests for the scheme primtives; which requires one to play scheme language lawyer to justify the behavior is correct (or guile is correct) then it is rather trivial for me to fix mes-m2 to comply with the easy to verify result <OriansJ>and it is honestly not very fun work to make those tests for me. <OriansJ>it is another reason why slow-utils is important (a series of useful scheme programs) to the success of mes-m2 because it gives me half-steps on the path to MesCC <OriansJ>and let us be honest here; mes-m2 is effectively the biggest leap in the bootstrapping process. (JUMP from a minimal C compiler to a feature complete compatible for guile, guix and MesCC in one binary) <OriansJ>We certainly will get there but like eating an Elephant, you have to do it one bite at a time. <OriansJ>we will probably have to force some minimalism on guix (or atleast its early bootstrapping stages) <OriansJ>markjenkinsznc: I would like to thank you for the M0 enhancement <OriansJ>because cc_x86.s just got built with --memory 8K <OriansJ>ok for M0 to build M0-compact is 64KB <OriansJ>So it is in the scope of a 16bit processor (barely) <OriansJ>Now to tweak regular M0 to reduce memory requirements and enable it to run on 16bit and 64bit targets as well <OriansJ>(mostly needed by hex2 and not M0-compact) <xentrac>OriansJ: protective coatings are often a good idea <xentrac>"(lambda (. cmd) ...)" looks like an ill-formed S-expression to me; is it in fact well-formed? <xentrac>I'm glad Ur-Scheme is usefully contributing to m0* even though my body is not succeeding in doing o <OriansJ>xentrac: would (lambda (. rest) ...) look ill-formed too? <xentrac>it does to me; I'm surprised at the implication that Guile can parse it <xentrac>even more surprising, Ur-Scheme evidently parses it too <OriansJ>well (. rest) is the most natural s-expression form for variable arguments <OriansJ>and the leading dot makes it obvious what should be done if more than 1 argument is passed <OriansJ>now if what was ment was only 1 argument should be passed, that would be a different problem <xentrac>heh, I'd forgotten about the minimal quasi-OO mechanism I used in Ur-Scheme's parser to be able to read from strings while also supporting ungetc <OriansJ>markjenkinsznc: and I added some examples of variable different behavior inside of instructions depending upon register size <xentrac>given that it is an S-expression form, it is indeed the most natural form for purely variadic arguments <xentrac>but it surprises me that it *is* an S-expression form <xentrac>including according to the parser I wrote! <OriansJ>it comes for free when you add support for dot forms <xentrac>unless you do it wrong, as I apparently did in my brain <xentrac>I wouldn't advise depending on Ur-Scheme for information about the correct way to handle strings in Scheme. I mean it's probably compatible with the Scheme interpreters I used to bootstrap it — for the limited set of situations encountered when running Ur-Scheme itself <xentrac>(the correct way to handle strings or anything else) <OriansJ>xentrac: well finding good scheme tests has been one heck of a problem for me <OriansJ>because without them, progress in mes-m2 is slow at best <OriansJ>believe it or not xentrac your Ur-Scheme did result in a half-dozen functional improvements in mes-m2 <OriansJ>and as I work through the remaining issues, I either will have a bug report for you or another feature enhancement for mes-m2 <OriansJ>It is just sometimes slow to pick out exactly what interaction is resulting in different behavior between guile and mes-m2 <OriansJ>speaking of mes-m2; fossy patches are up <fossy>OriansJ: have Ur-scheme tests been used in mes-m2 yet? <OriansJ>but I guess working through those is probably a good idea <xentrac>they're kind of crappy; they just test that Ur-Scheme produces the same output as $refscheme (see file urscheme/runtests) <OriansJ>xentrac: crappy is better than entirely missing <xentrac>they were enough to get Ur-Scheme running <OriansJ>now test.closure.scm has a lambda form which I never would have considered <OriansJ>because in mes-m2 (define (foo a) (display a)) is simply converted to (define foo (lambda (a) (display a))) <OriansJ>but ironically mes-m2 already supports it <OriansJ>with identical output to guile none the less <xentrac>what's the lambda-form you never would have considered? <OriansJ>(define (deeper a) (lambda (b) (lambda (c) (display a) (display b) (display c) (newline)))) <xentrac>heh, the only comment in that file is wrong <xentrac>Ur-Scheme implements define in the same way, but I think it's converted to (%define ... <xentrac>%define being a primitive define that doesn't support the (define (fn args) ...) syntax <OriansJ>I guess I have a rash of tests to integrate tomorrow (if you don't mind xentrac) into mes-m2 (with your copyright header of course) <xentrac>sure, go ahead; do you need me to relicense them or anything? <OriansJ>just permission to make them part of mes-m2 under gplv3+ is great <xentrac>sure, but you don't even need any explicit permission for that; that's the purpose of the GPL <OriansJ>I'll mostly toss on a license header with your info, change them to write to standard test locations, verify the results with guile and then add the expected test results into test.answers. <OriansJ>xentrac: if you can think of an order for the tests that simplifies debugging; let me know (I'll probably do an early start tomorrow on it) <xentrac>hmm, it might make sense to run them in the order that they were created <xentrac>specifically crufty quadruple remainder quotient number2string fib simplestring stringlength cons negative variadic assq implicitbegin predicates quote memq niladic tailcall emptyvariadic cond definelist let stringappend eqv case or map closure equal <xentrac>literalchar list2string reverse charwhitespace symbol2string string2symbol charint charalpha set write doubledefine string2number firstclassops fib2 funkyid lessthan append <xentrac>This ordering brought to you by darcs changes --xml-output "$x" and ElementTree. <xentrac>I think I kept adding crap to crufty though <xentrac>so at first it was just display, newline, begin, define, and an IIFE <xentrac>later I added arithmetic, recursion, and multiple arguments (but no decimal conversion for output! the fibonacci function produces output in unary) <xentrac>then quadruple quadruples a number by adding it to itself and tests it with another recursive unary number representation function <xentrac>hardly a surprise that the next things I added were "remainder" and "quotient"! <xentrac>the next step was number->string in number2string (but number->string is defined in the compiler, not the test) <OriansJ>xentrac: thanks, I'll give that a try <OriansJ>I guess it is time to add make-string to mes-m2 <OriansJ>now for a little language lawyerish for (define meh (make-string 8)) (string-set! meh 0 42) is guile correct in setting byte zero to null <OriansJ>or for the more questionable: (string-set! meh 0 "huh") <dddddd>guile 2.0 -> ERROR: In procedure string-set!: Wrong type argument in position 3 (expecting character) <OriansJ>2.2.6 -> "\x00\x00\x00\x00\x00\x00\x00\x00" <dddddd>That's meh after make-string alone <OriansJ>dddddd: oh because 2.2.6 simply makes a string of null chars <OriansJ>feel free to change to (make-string 8 #\a) <dddddd> -- Scheme Procedure: string-set! str k chr <OriansJ>first set of tests are in and new primitives have been added (patches are up) <markjenkinsznc>Morning OriansJ, thanks for your recent stage0 commits, a gift from the gods so to speak that will lead to a 0.4.1 release of knightpies with a more comprehensive test suite <OriansJ>markjenkinsznc: M0-compact is just 200bytes more than regular M0 <markjenkinsznc>M0-compact is awesome, I think it still makes sense to bootstrap from M0 as makefile is currently doing <OriansJ>and the amount of effort of converting M0-compact into hex2 is well within the bounds of a single day's work <OriansJ>that way M0-compact can be built directly from hex2 which already works on 16 and 64bit hosts <OriansJ>markjenkinsznc: completely fair hex2 work is annoyingly slow and hard to debug <markjenkinsznc>you mentioned before that you do hex0/1/2 from scratch instead of starting from automated output and adding whitespace and comments and then using tooling to verify produces same binary <OriansJ>markjenkinsznc: it keeps me honest as it forces me to want to minimize the amount of work auditors would have to do with such code <markjenkinsznc>the direction I was going would have a minimally changed M0-macro-16.hex2 and M0-macro.hex2 that works on 32 and 64 <OriansJ>markjenkinsznc: less hex? files to audit the better <markjenkinsznc>indeed, I guess by doing M0-macro-compact.hex2 which has runtime instead of build time support, we get to stay with one hex2 file <markjenkinsznc>another consideration in my mind was that someone auditing during their bootstrap would only audit the hex2 files they use specific to their arch and not the ones they don't use <markjenkinsznc>what's starting to make sense for me is that M0-compact binary is only slightly larger and the runtime support with READSCID isn't as complicated as I originally imagined <OriansJ>1696 bytes is an 8 hour audit (approximately) <OriansJ>(less if they don't check the offsets) <OriansJ>hex2 at 1006bytes is a 4 hour audit (approximately) <markjenkinsznc>All that being said, if I could salvage M0 in the way I've attempted, a shorter audit, even if just a bit, and less instructions used <OriansJ>hex1 at 472bytes is a 2 hour audit (approximately) <markjenkinsznc>shorter that is if auditer just audits hex2 they're using for their arch and not the other one <OriansJ>well 200bytes is only approximately 30-60Minutes of auditing time <OriansJ>So it might not be worth the effort on our behalf <markjenkinsznc>auditing M0-macro-compact.hex2 is less work if all hex2 files, those to use and not use have to be audited (as my approach produces M0-macro.hex2 and M0-macro16.hex2) <OriansJ>and you are free to do as you see fit markjenkinsznc <markjenkinsznc>but if you're going to write M0-macro-compact.hex2 anyway.. I might as well get it started for you <OriansJ>markjenkinsznc: well it would certainly save me some effort if you are willing to do that work for me <markjenkinsznc>You do amazing comments beyond just the instructions in these hex files, but I can at least copy the style in terms of whitespace and get a comment with the instruction name on every line <OriansJ>markjenkinsznc: if you look at them in emacs with whitespace-mode on you'll notice something else <markjenkinsznc>and I've just realized that the hex comments after the instruction are mostly the same as the .s comments, so if I do the hex2 it could be pretty close to what you'd do <OriansJ>markjenkinsznc: I look forward to seeing it <markjenkinsznc>the other thing I I'm seeing with emacs in X mode (haven't tested term mode) is some lines are highlighted in a special way in their entirity, just realized this is lines longer than window width <OriansJ>markjenkinsznc: so kinda ugly relative to my later style (which can be seen in mescc-tools-seed) <OriansJ>Ideally I'd go back and update the files in stage0 to reflect that later style (as it is much nicer) <OriansJ>But I really haven't had the chance to do so <OriansJ>mescc-tools-seed/x86/hex0_x86.hex0 lines up the # assembly and lines up the ; higher level comments <markjenkinsznc>Doing the hex2 conversion is probably the best way I can thank you for M0-compact.s as well. I'm accepting that my attempt to shoehorn M0 as is into 16 bit compatibility has already failed to pass the test of minimal effort, changing as little as I could get away with didn't get a quick result <OriansJ>markjenkinsznc: well we learned the hardway repeatedly every process of a quick result usually results in alot of wasted effort and delays. <OriansJ>every single clever trick ended up biting me square in the ass eventually <OriansJ>Heck 2 bytes in M0-compact.s are because of the hack currently in use by High_level_prototypes/disasm.c <OriansJ>markjenkinsznc: one thing stage0 NEEDs is a proper disassembler <OriansJ>then I can remove the backwards compatible pieces for padding to 4byte alignment <markjenkinsznc>Might attempt it in python, would also be a high level prototype <OriansJ>markjenkinsznc: well it being entirely in python is fine <markjenkinsznc>I'll probably need a good dissassembler when I start doing hex2 support for other archs <OriansJ>as we don't need it to be implemented in the bootstrap stages <markjenkinsznc>even when I did hex2tobin.py (or was it M1.py) I was doing funky stuff with hexdump -C and diff to figure out what was wrong with my binaries <OriansJ>getting knight into rasm2 would be cool but probably alot of work <markjenkinsznc>M0-compact.s will be especially nice if anyone builds a chip minimal knight 16 and wants to do further work from assembly instead of the efforts to make M2 style C happen <markjenkinsznc>I'm guessing much of the historical 16bit C world involved memory that wasn't byte addressable to get past 64K and/or bank switching and stuff... <OriansJ>well 64KB is very tight for a C compiler and one would have to shove in alot of optimizations to ake it fit <markjenkinsznc>someone building 16 bit knight might want to do a memory mapped IO device that's a DRAM <markjenkinsznc>but the programming techniques to do bank switching or memory mapped IO to a bonus memory device would be super hairy <OriansJ>hence why knight has the HALCODE support <OriansJ>to allow one to define hardware primtitives that software can leverage <OriansJ>It allows one to keep the entire memory space clean with no special cases <OriansJ>but makes hardware implementation harder <markjenkinsznc>so if anyone is reading this chat in future years, the standard warning applies.. before you start glueing chips together or FPGA hacking to get a knight16, they should implement any hardware memory tricks in emulator and built out fancy software using that first... <markjenkinsznc>And they should live with the hardware cost of doing any fancyness with HALCODE <markjenkinsznc>all right, has been a fun chat, now off for some hex2 work off and on for a few days <OriansJ>When one has a MMU, HALCODE is also designed to allow the kernel to use its own memory space to provide extended functionality <OriansJ>so ParadoxUndone what fun are you working on? <ParadoxUndone>well thinking of using this spare ex facebook blade i have sitting here doing nothing <ParadoxUndone>yeah a stuck it on the grid computing platform years ago and it performed quite well and it has been powered down ever since <OriansJ>ParadoxUndone: not to mention the salt, unless you prefer the fresh water bodies <ParadoxUndone>well marina with a lock and a run off into it from a river so rather brackish but still <ParadoxUndone>i wouldnt ever wish to live anywhere else but a boat on the sea in a marina or otherwise again <OriansJ>ParadoxUndone: personally I find surfing and life on the beach more enjoyable but to each their own. <ParadoxUndone>i don't think surfing is for me but i understand,as a youngster i used to swim out to sea and back nearly every day <ParadoxUndone>our community had it's own private beach next to it and next to that was the nudist beach which used to spill over occasionally <ParadoxUndone>took me 5 mins to figure out what was wrong with my friend i had brought home from school one day while i was blowing the tender up on the beach <ParadoxUndone>turns out he was in shock, some ebony beauty had spilled over, I was used to it and thought nothing of it but my freind was froma quite village in the middle of nowhere so <OriansJ>must have been one heck of a stunner <dddddd>That's what I'd call a proper bootstrap <OriansJ>ParadoxUndone: usually a sign of youth <ParadoxUndone>a bunch of the boaties here do it, as soon as they get out there off come the clothes and they sailing naked <ParadoxUndone>head the pier head telling them to pop some clothes on once or twice coming in though the lock <OriansJ>what adults do to themselves is their own business; so long as it doesn't result in the harm of others. <markjenkinsznc>I've verified with multiple tool chains that M0-compact.hex2 still matches the binary M0 + hex2 initially produced and will continue to keep that so <markjenkinsznc>The main audit I want to do with M0-macro-compact.s and M0-macro-compact.hex2 side by side after getting the formatting nice is that the comments match and no comments from the .s were lost <markjenkinsznc>Seeing how M0-macro.hex2 was produced with a from scratch process and it made the initial binary for M0-compact that ended up in test/SHA256SUMS and that final binary is unchanged, there is a bit of transitive trust one can place in M0-macro-compact.hex2, another reason for M0-macro.hex2/s to live in some kind of legacy pieces dir when all is said and done <markjenkinsznc>Another audit I could do to the .hex2 is use find replace in my test editor to match the instruction prefixes with names and do a side by side comparison to the comments after comparing the comments in both files <markjenkinsznc>I hope you're not disappointed I started with machine gen as a starting point with audit being second <markjenkinsznc>oops, ignore that M0-compact-hex2 branch, there's an obvious flaw in the comments part of the output that I realized as soon as I went outside for a walk <markjenkinsznc>much better starting point that is, more to do to make this pretty, audit it against the .s well and style points to learn from newer hex2 files <xentrac>I feel like guile is the kind of thing that should throw an error if you (string-set! meh 0 42) <xentrac>although allowing that to pass might be reasonable behavior for a different Scheme <OriansJ>Because once knight has a "good enough" disassembler I can drop the extra NULL padding from strings and generate some smaller binaries for stage0