IRC channel logs

2021-07-27.log

back to list of logs

<fossy>i am doing new linux kernel instead of trying to get linux 2.6.16 to work. too new software, too little syscalls, etc. many problems, too much work for smth to be replaced anyway
<fossy>looks like new kernel should be trivial with new make i am hoping
<fossy>if nothing else, the linux kernel build system has got WAYY more managable over the yr
<fossy>years
<xentrac>how so?
<siraben>Melg8: ping
<Melg8[m]>siraben: yea?
<stikonas>ekaitz, oriansj: I have some proposal for hex0 optimization, so would like to see what you think
<ekaitz>stikonas: shoot!
<stikonas>in particular yesterday, I was looking at the code that reads hex characters
<stikonas>so my optimization is a bit longer in terms of instructions 17 -> 30, but it's still 17 unique instructions, 13 are duplicate and it eliminates most branching/jumping there, so much easier to encode from .S->.hex0 or for somebody else to check
<ekaitz>when I did it, I worked first on assembly and translated later so it's probably a good idea to simplify it
<stikonas>so the binary might be a bit bigger, but simpler to inspect
<stikonas>let me show some example
<ekaitz>please
<stikonas> https://paste.debian.net/1205773/
<ekaitz>let me open the original to see the difference
<stikonas>to make it clearer, think of it as (a0 - 47) * (a0 > 47) * (a0 < 57) + (a0 - 54) * (a0 > 64) * (a0 < 71) + (a0 - 86) * (a0 > 96) * (a0 < 103)
<ekaitz>is this easier to inspect?
<ekaitz>i mean, there are less jumps but it's a little bit mindbending
<stikonas>well, that's what I am asking...
<stikonas>it's more involved in algebra
<GeDaMo>If you want to check if a character is in a set, you might use a bitset
<stikonas>but it does not use those mangled B and J instructions
<GeDaMo>(1 << char) & bitset
<ekaitz>stikonas: that's a great point, yeah
<siraben>Melg8: I was wondering how to replicate your bootstrap experiment on Nixpkgs. How do i get a shell up?
<ekaitz>stikonas: if the jumps are short as in this case, how is the wierd B/J format bit mangling affect? does it really change the order of the bits or it just affects to the highest bits?
<ekaitz>i mean, we may just be using a subset of the jump field that may be left untouched, I don't remember the format
<stikonas>format is here https://secureservercdn.net/198.71.233.189/a1a.c8e.myftpupload.com/wp-content/uploads/2020/12/word-image-1.png
<ekaitz>I just checked the J instruction's lowest 11 bits are left untouched
<ekaitz>well "untouched"
<stikonas>GeDaMo: but what bitset would we use for aphanumerical characters 0-9 A-F a-f?
<ekaitz>I mean, we can accept J instructions with 11 bit jumps, but the B's are still a pain in the ass
<stikonas>yeah, B are especially nasty
<stikonas>and I thought hex0 code without jumps/branches are much closer to hex2 code :)
<ekaitz>4 bit jumps are basically a skip :D
<GeDaMo>The bitset would have 1s set for each of the acceptable characters
<ekaitz>I agree, but there's one last question: are we going to need to tackle the jumps in any stage?
<ekaitz>it's just me thinking if we are just delaying the pain
<stikonas>ekaitz: I think hex1 will be calculating jumps
<stikonas>oriansj: had some proposal
<stikonas>well ,at some point we have to tacke jumps anyway
<ekaitz>that is my point, if we make this step's algorithm harder to read just to avoid the jumps and we have them later we didn't solve anything
<stikonas>oriansj thought the easiest way to implement all these riscv instruction types is implementing XOR in hex1
<ekaitz>how you mean?
<stikonas>e.g. have 000000CD . 00BF0000 A0000000 => A0BF00CD written out
<stikonas>. 000000CD . 00BF0000 A0000000 => A0BF00CD written out
<stikonas>so . means XOR this thing into the next 32 word
<ekaitz>right
<ekaitz>just combining together several values
<stikonas>yes, then in M1 definition file you can define
<stikonas>registers for each of the position in rd, rs1, rs2
<ekaitz>the main issue I see here is we're still having to reorder the bits... which is bad for readability
<stikonas>well, it's little endian...
<Melg8[m]>siraben: main thing which do the job is https://github.com/melg8/cit/blob/feature/BootstrapNix/bootstrap_nix/bootstrap_seeds/default.nix here. It doesn't depend on anything except local files. Maybe you could use it as starting point. But it is not up to date with latest live-bootstrap changes so first would need to update it. I've tested it just like "nix-build ./default.nix" from bootstrap_nix/bootstrap_seeds/ folder. Other files not used
<Melg8[m]>in default - is for test of recreating live-bootstrap style of build (whithout using nix at all) so i don't think that would be needed in nixpkgs
<ekaitz>stikonas: I mean the addresses are not clear in the value as the bits must be reordered according to the B/J formats anyway
<stikonas>oh I see, yes, that's true...
<ekaitz>is it that difficult to implement the bit mangling in hex1?
<ekaitz>probably, right? we would need to add some extra stuff :S
<stikonas>well, we might still have to implement bit mangling...
<stikonas>although, it would be easier to do in M1 level than hex1
<siraben>Melg8: thanks I'll look through this tomorrow
<siraben>I think i got an error about how derivation doesn't take a git argument, afk right now but I did try nix-shell
<siraben>in that folder
<GeDaMo>stikonas: https://ideone.com/F6hJ6P
<stikonas>GeDaMo: thanks
<ekaitz>GeDaMo 's idea could work to reduce the amount of jumps
<GeDaMo>You could hardcode the bitset, I just couldn't be bothered figuring it out manually :P
<stikonas>yeah, it's fine to hardcode bitset in my opinion, especially if it's shown in the comments
<stikonas>how to do that
<stikonas>algorithm is even more convoluted though...
<stikonas>but might result in fewer instructoins
<GeDaMo>Which algorithm?
<stikonas>your isHex stuff
<stikonas>ir maybe I'm just more used to algebraic stuff
<GeDaMo>It's two shifts and two ANDs
<stikonas>since I did maths at university
<GeDaMo>And a memory fetch
<stikonas>but yes, I think in terms of instructions it will be shorter than both of the other versions
<ekaitz>if you have issues with it I can help you tomorrow stikonas
<stikonas>no, it shouldn't be too hard to convert to assembly
<ekaitz>where can I read a little bit more about M1? I'm unable to find it. We have too many subprojects :)
<ekaitz>(i'm trying to find the best solution for the J/B formats)
<stikonas>ekaitz: probably source code in stage0-posix repo
<ekaitz>i can't find it there... :S
<stikonas>there is no high level prototype...
<ekaitz>ugh!
<stikonas> https://github.com/oriansj/stage0-posix/blob/master/x86/M0_x86.hex2
<stikonas>ekaitz: oh, there is one for knight
<stikonas>ekaitz: oh, that's because there is a C version in mescc-tools
<stikonas> https://github.com/oriansj/mescc-tools/blob/master/M1-macro.c
<GeDaMo>stikonas: in your paste, is the character being tested in a0?
<stikonas>GeDaMo: yes
<ekaitz>too many projects :))
<stikonas>ekaitz: C versions is probably a bit bigger than the one written in hex2
<stikonas>usually C version is cross-platform and can deal with all architectures
<ekaitz>yeah! I just need a high level view of it, I still lack of context sometimes
<stikonas>ekaitz: yes, I also didn't look too much into implementation of M1, was just using it
<ekaitz>(now I realize I've been reading this file this morning... sigh)
<GeDaMo>stikonas: something like this, I think https://ideone.com/ttVsSa
<stikonas>hmm, riscv64 code probably doesn't need that much modification to run on riscv32 too...
<stikonas>mostly elf header and occasional constant
<stikonas>GeDaMo: thanks. So that's also "2 jumps" like in my paste but indeed significantrly fewer instructions
<stikonas>and bitset can probably be explained in the comments
<ekaitz>stikonas: rv64 and rv32 are basically the same
<ekaitz>the only difference is on the word vs doubleword opcodes
<ekaitz>for this case, I'd say we won't need to change that much
<xentrac>there are a few other minor differences
<stikonas>well, addresses are 64 vs 32 bit length
<stikonas>so some code will need e.g. different shifts
<stikonas>but indeed minor differences
<xentrac>some of the opcodes are different
<xentrac>in a poorly-considered way
<ekaitz>afaik they are the same all the time
<ekaitz>RV64 is just an extension with extra opcodes
<xentrac>nope, not at all, it's a different, incompatible instruction set
<xentrac>although most of the opcodes are the same, which can trick you into thinking that
<xentrac>but it's not true
<xentrac>there's a FAQ about it
<ekaitz>can you show me the FAQ?
<ekaitz>the book I'm following doesn't mention that, in fact it's only providing the differences between the two in the RV64 part
<ekaitz>I recently made a JIT library migration and I didn't notice this... maybe it's just in some specific cases?
<xentrac> https://groups.google.com/a/groups.riscv.org/g/sw-dev/c/mUUaoxjQy6Q
<ekaitz>the only difference I see in the opcode is SLLI and such
<xentrac>I think loading and storing to memory may be another big thing
<xentrac>storing to memory with the same opcode as on rv32 will clobber another 4 bytes
<xentrac>i think?
<ekaitz>not really
<ekaitz>32 bit uses sw, and 64 bit uses sd
<xentrac>huh, guess I'm just wrong then. what nice news!
<ekaitz>so the only difference is in the SLLI SRAI and such
<xentrac>thanks for correcting me!
<ekaitz>but still! for our work it doesn't really matter that much, the code can be mostly copy-pasted from RV64 to RV32
<ekaitz>:D
<ekaitz>we just need to change the doubleword instructions to word instructions and they should work
<ekaitz>at least in hex0 / hex1
<stikonas>yeah, that shoudl be enough in my opinion. Well plus elf header
<ekaitz>the elf header only changes the flag from 64 bit to 32
<ekaitz>it's the e_machine I thing
<ekaitz>think*
<stikonas>ekaitz: e_machine is actually the same for 32-bit and 64-bit
<stikonas>some other fields in the header are different
<stikonas>machine class is 32 bit but also some addresses have to be 32-bit rather than 64-bit
<stikonas>so header length is different
<ekaitz>there's just one
<ekaitz>e_ident[EI_CLASS]
<ekaitz>that has to be set to indicate 64 or 32 bit and that should be everything
<stikonas>ekaitz: stuff like e_phoff is different
<stikonas>or e_entry
<stikonas>also program header has some addresses
<stikonas>so e_ehsize and e_phentsize will also be different
<stikonas>but we can probably steal those from x86
<ekaitz>lol so hex0 might be working by accident because i didn't really payed attention to that
<ekaitz>i copied from AMD64 anyway :D
<stikonas>well, for rv64 it makes sense to copy from amd64 but for rv32 copy from x86
<stikonas>I wonder if elf32 will run on rv64...
<stikonas>or at least one qemu...
<oriansj>ekaitz: mescc-tools/examples/M0-macro.c
<ekaitz>oriansj: yes thanks! I already found and checked it
<stikonas>oriansj: and do you have any opinion on alternative hex character checks (with fewer branches)?
<stikonas>I guess there are some pros and cons with with each of them
<oriansj>stikonas: hex0 should optimize for less bytes and easier understanding.
<ekaitz>oriansj: which one is more important?
<stikonas>well, fewest bytes option might be hardest to understand
<oriansj>branches and jumps are fine to do by hand if you properly label the addresses at each label.
<oriansj>ekaitz: understanding is most important. Less bytes is the second priority.
<stikonas>well, then maybe just leave current version...
<stikonas>although, currently addressess in hex0_RV64 are not properly labelled
<oriansj>The binary hex0 is the only thing that needs to be inspected byte by byte and less confusion the better. Hand toggling in the binary is the second factor which we would want to optimize for (less bytes)
<stikonas>well, in risc-v branches are a bit annoying even with labels with addresses but it's not complicated to understand, just a bit more work to check everything
<stikonas>you basically have to inspect it bit by bit... rather than byte by byte
<oriansj>stikonas: well there will only be 3 programs where jumps need to be calculated: hex0, hex1 and kaem-minimal.hex0
<stikonas>that's true
<oriansj>but yeah RISC-V is a horrible architecture for bootstrapping
<stikonas>yeah, I didn't know risc-v would be that horrible...
<stikonas>assembly language itself looks alright but machine language is horrible...
<stikonas>and of course we'll later have all those problems with old gnu software not working on risc-v
<oriansj>well whenever Instruction Set design decisions start with the assumption of simplify hardware at the cost of making programmers do more work. It usually means it is a dumpster fire of short-sighted decisions.
<stikonas>ekaitz: by the way, any reason why you open file with append in hex0?
<ekaitz>no reason at all
<stikonas>I would think O_TRUNC would make more sense
<stikonas>easy to change though...
<ekaitz>yeah, surely
<stikonas>just need to use mode 00001000 there instead of 00002000
<ekaitz>yeah, change it if you prefer. I think it's better with O_TRUNC
<stikonas>at least this is easy to encode into .hex0 file
<stikonas>just one line change
<stikonas>(plus comments)
<stikonas>I can make a PR then...
<stikonas>hmm, speaking of PR, .S file should probably go to stage0-posix...
<oriansj>although it looks like RISC-V's hex1 is likely going to be bigger than ALL other architectures' hex1 because word manipulation needs a bunch more state.
<stikonas>does it even make sense to write hex1 then?
<stikonas>rather than going for hex2 directly
<stikonas>if hex1 is quite big already, adding multiple label support might not make it much bigger
<oriansj>stikonas: well that will be 100% your decision and once I get word behavior in hex2 done'ish you'll have a more informed decision to work with.
<stikonas>ok, I made O_TRUNC pull request to bootstrap seeds...
<stikonas>although , I think .S file should be moved to stage0-posix/RV64/Development
<stikonas>since that's what we do for all other arches
<ekaitz>stikonas: yeah, good idea
<stikonas>oriansj: maybe you can move it rather than me making another 2 PRs?
<ekaitz>I added it to the bootstrap seeds because I wanted it to serve as documentation, but it doesn't really make any sense to keep it there if others are in stage0-posix
<stikonas>maybe README file then?
<stikonas>.hex0 files are actually duplicated among both repos
<stikonas>actually, if we follow other arches, maybe .S should be moved to stage0-posix/RV64/GAS/