IRC channel logs

2021-10-03.log

back to list of logs

<stikonas>yeah, a lot of 32-bit constants
<stikonas>although, I wanted to eventually generate them on the fly anyway
<stikonas>rather than using precomputed constants
<fossy>stikonas: ok i had a look at it i see no reason for that not to work but you need oriansj's change i think
<stikonas>yeah, I'm testing it
<stikonas>just noticed that bootstrapped sha256sum is a bit broken
<stikonas>it's computing some hash but not sha256
<oriansj>stikonas: it is sha256
<stikonas>oriansj: normally yes
<fossy>but you want to return *before* env is initalized and not after like is currently at "avoid empty arrays" otherwise it will def break
<stikonas>but risc-v version is miscompiled
<stikonas>the constants in the lookup table are mangled
<oriansj>stikonas: does RISC-V support PC relative loads?
<stikonas>well, AUIPC
<oriansj>So what if we do the load jump immediate pattern?
<stikonas>and store number between those?
<oriansj>as AMD64 and AArch64 don't have these sorts of issues with constants
<stikonas>i.e. load 8 bytes ahead jump, constant, rest of the instructions
<oriansj>yep
<stikonas>oh, but let me thing about loads
<stikonas>hmm, I guess it's fine
<stikonas>just load address with AUIPC and then do LD on that address
<xentrac>oriansj: cool, thanks!\
<xentrac>you probably want to addi 4 or something, don't you, stikonas?
<oriansj>for the jump
<stikonas>yeah, of course
<oriansj>or you could jump and load backwards
<oriansj>jump immediate load
<stikonas>well, the other alternative is to emit more complex immediate load code in m2-planet using shifts
<xentrac>I guess LD has an immediate offset too
<stikonas>yes, LD has offset too
<xentrac>no addi necessary
<oriansj>also I have no idea why the LUI+addi combo are causing issues
<stikonas>I think it's only supported up to 32-bit signed integers
<xentrac>yeah
<stikonas>although it's a bit confusing
<oriansj>the code base (except for the new RISC-V support) has already been over 64bit architecture support and 32bit integer behavior
<stikonas>why 32-bit unsigned integers fail to work
<stikonas>and basically work only up to 31-bit
<stikonas>well, it's LUI limit
<stikonas>not M2-Planet's limit
<xentrac>LUI sign-extends
<xentrac>IIRC
<oriansj>so does AMD64 and AArch64 signed instructions
<oriansj>does RISC-V not have an unsigned version?
<xentrac>no
<oriansj>that doesn't sign extend
<xentrac>no
<stikonas>that's why binutils also does some hacks when you have those larger numbers
<stikonas>it uses AUIPC/addi initially
<xentrac>for pointers
<oriansj>So I guess we need a check in M2-Planet to not produce LUI for 32bit values
<stikonas>sorry lui/addi
<xentrac>yeah
<stikonas>well, yes, and probably do something else
<stikonas>I think it actually breaks at 2^31-1-2^11
<stikonas>(that 11-th bit that we add due to sign extension of lower ADDI bits)
<oriansj>something like 0 != ((0x8 << 28) & value)
<oriansj>well that is why we have: https://github.com/oriansj/mescc-tools/blob/master/hex2_word.c#L101
<stikonas>oriansj: can you merge https://github.com/oriansj/mescc-tools/pull/26/files
<stikonas>it works now
<stikonas>argh
<stikonas>wait
<oriansj>and https://github.com/oriansj/mescc-tools/blob/master/M1-macro.c#L627
<stikonas>copy-paste typo
<stikonas>well, you can add range-check in M1-macro.c
<stikonas>but I guess logic of dealing with large immediates in M2-Planet
<stikonas>I guess 0xFFFFFFFF works by accident...
<oriansj>hence the logic in primary_expr_number
<stikonas>hmm, although, it didn't work in hex2_word.c
<stikonas>well, if 0 != ((0x8 << 28) & value maybe we shouldn't return
<stikonas>but rather deal with it
<stikonas>ok, https://github.com/oriansj/mescc-tools/pull/26 is now ready
<stikonas>works fine and I think this version is uncontroversial now
<stikonas>it just moves zero-length check up
<stikonas>as fossy instructed
<oriansj>works for me
<oriansj>merged
<stikonas>yes, it works for me too, just rechecking checksums
<stikonas>and will make stage0-posix update
<stikonas> https://github.com/oriansj/stage0-posix/pull/50
<stikonas>all binaries except for sha256sum seem to work correctly
<stikonas>strange, now sha256sum seems to work too...
<xentrac>yay determinism!
<xentrac>;)
<stikonas>well, code was changed...
<stikonas>although, I'm a bit confused by that...
<xentrac>sometimes in those situations it's worth changing it back to see if you can reproduce the error
<xentrac>that way you can avoid having it pop up again in two months after another apparently unrelated change
<stikonas>ok, it doesn't work
<stikonas>it ended up in my PATH
<stikonas>so I was running bootstrapped sha256sum both times
<stikonas>anyway, that's probably a good testcase for M2-Planet's large immedates support
<stikonas>once it matches GNU sha256sum, large numbers are fixed
<oriansj>so RISC-V is almost (so close) to being ready for a stage0-posix release
<oriansj>just need to sort out the large number support in M2-Planet and are there any other bugs we need to crush?
<stikonas>well, ideally kaem-minimal's crash detection
<stikonas>right now I don't understand why it fails
<stikonas>but it's new code
<stikonas>other arches use older syscall
<oriansj>and we have some bugs to shake out
<oriansj>the more it is pushed the better.
<oriansj>I'll begin fuzzing shortly
<stikonas>waitpid was not available on risc-v
<stikonas>so I had to use wait4
<stikonas>maybe messed up some flags
<stikonas>ok, I think it's not instruction that doesn't work
<stikonas>it does bitwise not
<stikonas>where as I have 4 in the register
<stikonas>and want 0
<stikonas>hmm, I think M2-Planet's implementation also doesn't work for random integrers (it's 1 bitwise xor number)
<stikonas>oriansj: ok, I fixed it in GAS version
<stikonas>I'll try to encode it now and make a PR
<stikonas>so then only big numbers are left...
***Noisytoot is now known as GPLv3
<stikonas>oh yes, I forgot to fix kaem checksums for other arches...
<stikonas>oriansj: kaem-minimal fix https://github.com/oriansj/stage0-posix/pull/51
<oriansj>umm I just pushed a fix for kaem checksums for the other archs
<stikonas>yeah, I saw
<stikonas>anyway, kaem-minimal wouldn't affect other checksums
<stikonas>just removed one instruction and replaced beqz with bnez
<stikonas>and it works now
<stikonas>well, actually inserted nop to avoid recalculating jumps
<oriansj>fair, that shit is tedious
<stikonas>but I think M2 implementation of logical not only works for 0 and 1...
<stikonas>so might hit similar problem...
<oriansj>wait what????
<oriansj>logical NOT only makes sense for TRUE and FALSE
<oriansj>which are #defined
<oriansj>So if you are doing a logical NOT on anything else, it is probably a bug
<oriansj>bitwise not however will work with anything you want
<stikonas>oh ok...
<stikonas>I was basically trying to implement this: https://github.com/AssemblyScript/musl/blob/aad50fcd791e009961621ddfbe3d4c245fd689a3/include/sys/wait.h#L52
<stikonas>#define WIFEXITED(s) (!WTERMSIG(s))
<oriansj>So !(a == b) will be correct as ==, !=, >, >=, <, <= only return TRUE or FALSE
<stikonas>and s was equal to 4 when crashed
<xentrac>when I read that macro I always thing "excited" is misspelled
<xentrac>*think
<stikonas>well, at least gcc defines logical NOT for all integers
<stikonas>e.g. return !4 actually returns 0
<oriansj> https://github.com/oriansj/M2libc/blob/main/amd64/Linux/unistd.h#L58
<oriansj>wait4 with struct rusage *ru set to NULL works fine for wait
<stikonas>and return !4 exits with 5 in M2-Planet binary
<stikonas>yes, I did set rusage *ru to NULL...
<stikonas>anyway, just some difference between gcc and M2-Planet...
<oriansj>return !4 in C isn't exactly proper C and could be considered undefined behavior as I understand.
<oriansj>Now if TRUE was #define TRUE 4; it might be valid
<stikonas>hmm, it seems C standards do define it...
<stikonas>The result of the logical negation operator!is 0 if the value of its operand comparesunequal to 0, 1 if the value of its operand compares equal to 0. The result has typeint.
<oriansj>so 4 would not be in the defined input set
<oriansj>unless one interprets comparesunequal to 0
<oriansj>in which case !4 would just return 0
<stikonas>I think defined input set is ints
<stikonas>4 is not 0, so result should be 0 according to C specs
<oriansj>So why write !4 when it just becomes 0?
<stikonas>anything that is not 0 goes to 0
<stikonas>and 0 goes to 1
<stikonas>and source is here http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1124.pdf
<stikonas>this is some technical draft as for final ones I think you have to pay
<stikonas>negation is 6.5.3.3
<xentrac>my protesting against news-media bigotry against "hackers" is being seriously downvoted: https://news.ycombinator.com/item?id=28732879
***GPLv3 is now known as Noisytoot
<stikonas>ok, large ints are working
<stikonas>will do PR tomorrow
<stikonas>once I check checksums and tests
<oriansj>uh a compare and a jump just to do a logical negation
<oriansj>although I guess one could uset setne and sete to achieve the same result
<oriansj>xentrac: media isn't looking to use the correct term in the technical sense nor the cultural one either.
<oriansj>I believe there is even a term for how inaccurate newspaper articles are. Something about an expert being horrified by the amount of errors when covering the subject they are experienced on. Then promptly forgetting that fact when they turn the page and read an article on politics or some various other topic.
<oriansj>but yes I agree xentrac that it should be argued for the RMS meaning of the word hacker. https://stallman.org/articles/on-hacking.html and I do find it ironic that it isn't embraced on a site called hacker news. However that might be a sign of the shift in culture away from technical brilliant ideas and hacks and towards one chasing the VC dollar.
<oriansj>looks like M2-Planet has a few new crashes possible on bad input
<oriansj>nice
<oriansj>but M1 is looking crash free
***ChanServ sets mode: +o janneke_
***ChanServ sets mode: +o janneke
***deesix_ is now known as deesix
<stikonas>oriansj: I've now made a PR to get large immediates working in M2-Planet
<stikonas>it might not be the most optimal way...
<stikonas>but it works
<stikonas>(and sha256sum on risc-v now works correctly)
<stikonas>but cc_riscv64 does not support large immediates, maybe we can just leave it like that...
<oriansj>stikonas: cc_* doesn't have to be perfect, just functional enough to build M2-Planet
<oriansj>and it looks quite functional enough without large immediate support
<oriansj>also merged
<stikonas>so basically for risc-v we just need to update m2-planet in stage0 and recalculate checksums
<oriansj>provided that clears out the remaining bugs. Then yes and we will do an official release
<oriansj>as adding support for RISC-V is a major achievement and worth marking
<stikonas>I'm now redoing checksums
<stikonas>good that I have aarch64 hardware...
<stikonas>makes it much less painful
<oriansj>thank you
<stikonas>risc-v is still slow since its in qemu...
<stikonas>but at least it's not 15min...
<stikonas>anyway, it's probably good enough for release...
<stikonas>if we really need something more later, one can do another bugfix release
<oriansj>well bugfixing is part of the routine update process anyway
<stikonas> https://github.com/oriansj/stage0-posix/pull/52
<oriansj>but yeah, if the pieces seem to all be working well. It is good enough
<stikonas>well, sha256sum output now matches what gnu coreutils does
<stikonas>(make test_*) runs both
<oriansj>yep
<oriansj>merged
<oriansj>and I will tag for release
<stikonas>and then at some point janneke should update mes's kaem files with new blood-elf option
<stikonas>or we make PR for him
<stikonas>well, it's email there...
<stikonas>sorry wait a moment
<stikonas>I think I have temporary files in checksum
<stikonas>oriansj: we need to remove first 3 lines from riscv64.answers
<stikonas>argh, probably already tagged...
<oriansj>removed tag
<oriansj>and I'll do a git reset HEAD~1
<oriansj>so you can do a clean up and recommit
<oriansj>and I'll merge
<stikonas> https://github.com/oriansj/stage0-posix/pull/52
<stikonas>ok...
<stikonas>those slipped through via make Generate_test_anwers...
<oriansj>understandably.
<oriansj>It is why I alway do make clean test-* to prevent such things
<stikonas>I think I was running this while doing M2-Planet checksums...
<stikonas>but forgot for stage0-posix
<oriansj>these things happen
<stikonas>ok, things seem to work after make clean too now
<oriansj>and a new official release for mescc-tools is now up (now that the RISC-V isssues have been largely cleared)
<oriansj>and an M2-Planet release
<stikonas>still a few components left...
<stikonas>I guess bootstrap-seeds need a new release, then mescc-tools-extra and stage0-posix
<stikonas>oh and M2libc...
<oriansj>sure I
<oriansj>I'll do some tagging
<oriansj>although I don't think mescc-tools-extra needs a release
<oriansj>as nothing new has been added
<oriansj>but I'll give it a minor tag (v1.2.1)
<oriansj>ok, I'll incorporate all that and do a proper release for stage0-posix with it all included
<oriansj>fuzzing for hex2, M1 and M2-Planet all looking good now
<oriansj>stikonas: what are your thoughts on instead of the output being in bin but rather bin-${ARCH}
<oriansj>as it would allow all of the architectures to be built in parrallel
<oriansj>fossy: thoughts?
<stikonas>hmm
<stikonas>that's true but maybe it's better to do
<stikonas>${ARCH}/bin
<stikonas>I think at least from scripting perspective in live-bootstrap that might be easier
<stikonas>then ${ARCH} is basically self-contained...
<oriansj>good point
<oriansj>ok all checksums updated, now to enable parallel builds/tests
<stikonas>should also email to bootstrappable mailing list...
<oriansj>probably after I finish the parallel change
<oriansj>and I'll finish after I deal with the wife interrupt
<stikonas>sure, I also need to help wife with dinner...
<oriansj>I'm going to make the parallel builds for the next release cycle as it has grown in complexity to include build artifact inspection and audit. Which will make for a good next release but too much to do in a single day.
<deesix>Hello, oriansj. That reminds me the testing unification I left half finished... sorry for that. I need to catch up a bit with (months of) logs and the work on riscv that I've been seeing flying by these days. Congratulations, stikonas!
<xentrac>glad to see you, deesix!
<deesix>As for "how can two different people running the exact same static binary produce different behaviors", I saw qemu having bugs that make some runs to segfault but works sometimes (for code that runs 100% fine on the metal).
<xentrac>I don't think catching up on months of logs is necessarily a good idea
<deesix>xentrac, hello! Thanks :)
<stikonas>deesix: thanks. Yes, risc-v is now done at stage-posix level
<stikonas>well, at least 64-bit version, but I don't think I'll do 32-bits. Somebody else can pick it up
<stikonas>of course there are other stages, not just stage-posix, e.g. mes does not yet have any risc-v support
<oriansj>deesix: well it is finished for everything except knight and test1000; which honestly made the RISC-V test creation sooo much easier and faster. Thank you for that
<deesix>xentrac, sure... all of it is a lot, but a bit of reading to get/recover some context might help, I think... (also to make silly questions again and again :P).
<oriansj>we got RISC-V tests integrated in only a few minutes but it took a little bit to get them all to pass.
<oriansj>deesix: we do encourage questions and improved shared understanding.
<deesix>xentrac, *not* make
<fossy>I also think $ARCH/bin is fine
<stikonas>well, that's the point of tests, if they had been passing initially, then they wouldn't have been that useful
<deesix>oriansj, great... so they were integrated. Thanks!
<stikonas>now we were able to use them to find a few bugs
<oriansj>stikonas: and mailed out to mailing list the release announcement
<stikonas>you also did quite a bit of work yourself, you were mostly mentioning me... Anyway, it's good to finally have release out
<xentrac>questions are good
***ChanServ sets mode: +o oriansj
<oriansj>stikonas: well I believe in praising people when they do good work.
<oriansj>and you absolutely did great work that should be pointed out
<xentrac>I think it's great too