IRC channel logs

<stikonas>ok I see it. By the way bashisms are explicitly allowed in live-bootstrap too...

<stikonas>anyway, I should probalby go back to my kaem prototype work...

<mihi>ah ok, I only remember some discussion about bashisms in this channel, not sure about which project they were.

<stikonas>I think somebody tried to convince us not to use them in live-bootstrpa

<mihi>I guess the easiest bandaid fix is in https://github.com/mirror/sed/blob/cba6413f07c39d757bafcfff411588e3c0631627/sed/utils.c#L290 to check whether utils_fp_name(stream) starts with /dev/ and ignore errors...

<stikonas>but given that bash is the only shell we have (besides kaem), it makes little sense to restrict ourselves

<mihi>About kaem, UEFI and UTF-16: Did you check whether UEFI shell's editor can work with both ascii and UTF-16 files?

<mihi>honestly I don't know...

<stikonas>mihi: I think it can't

<stikonas>it produces UTF-16 files

<mihi>so probably kaem should be able to read them :)

<oriansj>and not need any conversion at all

<stikonas>well, right now I read ASCII files

<stikonas>possibly we can add another optional binary

<stikonas>to convert UTF-16 into ASCII

<oriansj>sure, we can add it to mescc-tools-extra

<stikonas>it might have to be written in hex0

<mihi>is it hard to skip all nulls when reading the ASCII file? Which would work for our cases.

<stikonas>maybe we can do it on reading

<oriansj>well in assembly we can just not look for nulls

<stikonas>we need to skip 00 and FF and FE

<stikonas>file starts with UTF byte order mark FFFE that helps to determine endianness

<stikonas>well, it's just a couple of if's to skip if we read those...

<stikonas>also not sure what to do with line breaks...

<stikonas>either I just don't do them

<oriansj>as 4C 00 4F 00 4C 00 isn't any different than 4C 4F 4C to logic looking only for 0A and 20

<stikonas>since we only need them from m2-planet and before kaem

<stikonas>maybe I should paste what I currently have

<stikonas>hmm, debian pastebin thinks that kaem is spam...

<oriansj>we can do pascal strings or just ensure 4 nulls at the end

<mihi>yet another 4 letter acronym :P

<oriansj>read only words and if the last word is all zeros

<stikonas> https://paste.ubuntu.com/p/vkNtDZfmyV/

<stikonas>right now I think I've got load_image to work but not yet start_image, that one exists with non-zero exit code without launching child

<stikonas>I guess else if(c == '\n') { should also have || '\r'

<stikonas>and then add some if c == 0 to skip over...

<stikonas>so this kaem doesn't have to do so much tokenization but on the other hand UEFI API is somewhat bulky

<stikonas>and we also need to do the same fix in hex0, hex1, ...

<mihi>For info, I just tried with VirtualBox's UEFI shell's editor, when you save file with F2 it is Unicode, but when you press F9 before F2 it will become ASCII.

<stikonas>oh yes, I missed that File type thing

<stikonas>so I think we just assume that files are ASCII

<mihi>so probably if it is easier, restricting to ASCII files should be fine :)

<oriansj>siraben: umm it might be simpler than you think

<stikonas>well, it will save us some bytes...

<oriansj>stikonas: https://unicode.org/roadmaps/bmp/

<stikonas>?

<oriansj>fgetc function can just drop the nulls and read another byte

<oriansj>as long as we stick to ascii

<oriansj>all the bytes we will be reading are 00 ##

<stikonas>oriansj: we don't have fgetc though...

<stikonas>I didn't put it in another function...

<stikonas>we might but we'll see...

<stikonas>I'll first get something working, then we can tune C code

<oriansj>stikonas: fgetc is just what I call the function that reads a byte in kaem/hex0

<stikonas>ok...

<oriansj>in your C code, it would just be a function called anything you want which does the fgetc call

<stikonas>yeah, it's fin->read(fin, &size, &c);

<stikonas>where uint64_t size = 1

<stikonas>oriansj: and what about line breaks?

<oriansj>so yeah, a little function with a goto if c is zero

<stikonas>do we need them?

<stikonas>we can't easily use \

<oriansj>I don't know if we need them in UEFI

<stikonas>because all paths use that \

<stikonas>ok, I think let's not add them

<stikonas>even in POSIX we don't use them that much

<oriansj>just to make long commands pretty

<stikonas>ok, got it to run hex0 (was a minor bug with child image handler, should be passed by value)

<stikonas>now I need to fix command line arguments that are passed to child and I think kaem prototype will be done

<oriansj>nice

<stikonas>well, it's an easy part...

<mihi>stikonas, when you use MEDIA_FILEPATH_DP as device path, you should be able to avoid actually loading the file yourself.

<stikonas>converting it to hex0 will be harder

<stikonas>mihi, yes I know, but I'm not sure if it's much smaller

<stikonas>in terms of code

<mihi>you might be right. :D

<stikonas>mihi: cause I need to copy the file location string

<stikonas>and I still need to allocate_pool for file path string

<stikonas>so I thought I can just as well read the whole file into it

<mihi>for the filename you could probably get away with a static buffer. Also I believe you could free the device path and maybe even the executable image between Load_image and start_image.

<stikonas>mihi: static buffer is not that good in hex0

<stikonas>your binary will grow by that size

<mihi>so buffer allocated at startup with either allocpool or allocpages.

<stikonas>yeah, I think freeing earlier does work

<stikonas>oriansj: I think I've got it working now: https://git.stikonas.eu/andrius/stage0-uefi

<stikonas>at the moment there is an assumption that children don't check loadoptionssize field

<stikonas>it might be a good idea to fix that even if we don't use it

<stikonas>but other than that it seems to work

<stikonas>3.5 KiB when compiled with C compiler, but hopefully will be smaller when we hand-write it

<oriansj>stikonas: well assuming PE format doesn't include a bunch of required crap, we definitely should be able to trim that down by 4x

<stikonas>well, looking at those binaries, there are a lof of gaps filled with either 0x00 or 0xCC

<stikonas>oriansj: I'm thinking a bit on how to go from C to hex0...

<stikonas>I guess we first need to try to target some off the shelf assembler

<stikonas>but I don't think nasm works...

<stikonas>so maybe I should just use llvm-as as it natively supports PE binaries that UEFI uses

<oriansj>stikonas: does UEFI support return codes?

<stikonas>oriansj: it does

<stikonas>oriansj: I think it supports even more of them than POSIX

<stikonas>up to native integer length, so uint64_t on 64-bit system

<stikonas>oriansj: and it also supports returning arbitrary text data

<stikonas>but I haven't coded that in caem

<oriansj>here is my idea: a hex2 binary which just returns 42

<stikonas>in kaem

<oriansj>shouldn't need any more than a few assembly instructions and should cover the basics of the headers needed

<stikonas>yeah, we can start with that...

<oriansj>well it doesn't look horrific: https://upload.wikimedia.org/wikipedia/commons/thumb/1/1b/Portable_Executable_32_bit_Structure_in_SVG_fixed.svg/2980px-Portable_Executable_32_bit_Structure_in_SVG_fixed.svg.png

<oriansj>but will probably take some experimentation to discover what we can just NULL and what needs to be carefully set

<oriansj>the checksum bit has me concerned how to do that in hex2

<oriansj>but we might be able to ignore it and put nulls if it isn't actually checked.

<stikonas>there might be some differences in 64-bit version...

<oriansj>hopefully we don't need the DOS stub program (unless it ends up making things simpler for us)

<oriansj>ooooh, as COM binaries are a property of PE files; wouldn't that mean UEFI supports COM binaries with PE wrappers?

<oriansj>or too hopeful of a thought?

<stikonas>hmm, no idea...

<stikonas>oriansj: DOS stub can be zeroed

<stikonas>at on QEMU it's still happy

<stikonas>not sure if we can completely skip it

<stikonas>but at least if it's zeroed, then it's not too bad

<oriansj>I was thinking something a little more aggressive

<oriansj>as the DOS_STUB is just a COM file, if we put a useful program in there, would UEFI actually run it? As then we can skip virtually all of the PE file format entirely

<stikonas>oh I see

<stikonas>we can try...

<stikonas>though maybe UEFI would just jump to normal entry point

<stikonas>and ignore dos_stub

<oriansj>indeed, we don't yet know

<oriansj>but we certainly will have some fun figuring this puzzle out

<stikonas>checksums are already zeroed in my test file

<stikonas> https://stikonas.eu/files/bootstrap/test.efi

<stikonas>I didn't even have to do it

<stikonas>so probably most of the fields can be at the very least zeroed

<stikonas>this can be built with fasm: https://stikonas.eu/files/bootstrap/test.asm

<oriansj>thanks, that'll be handy for testing

<stikonas>(I just put it in Development/rootfs directory in stage0-uefi repo and then tried to manualy run it

<stikonas>not sure if this applies to UEFI but https://stackoverflow.com/questions/553029/what-is-the-smallest-possible-windows-pe-executable

<stikonas>The smallest x64 PE executable is 268 bytes.

<stikonas>I wonder if UEFI can run 32-bit binaries

<stikonas>that could cut the size by a lot

<oriansj>well we have to abuse UEFI to find its limits

<stikonas>well, be careful with that. Best to make sure it's within specs, otherwise some things might work in one implementation but not the other

<stikonas>though I guess we can test on a few different computers and see if things generally work

<oriansj>well the problem with specs is they don't always reflect reality

<oriansj>but yes, generally we should be confirming to the specification to give us the greatest compatibility as possible

<oriansj>^confirming^conforming^

<muurkha>possibly of interest to some here: https://www.offlineos.com/

<muurkha> http://b5l4xazoe5xjswxrptga65mbcq4o5ism2isee2kc76gohmrdf3o5jdqd.onion/

<Hagfish>have all the UEFI tests been done in qemu so far? (sorry, i haven't been following, but it seems like there's another big achievement ahead or recently passed)

<stikonas>Hagfish: yes, only in qemu

<stikonas>there wasn't much to run until recently

<stikonas>and it's much quicker to boot qemu

<Hagfish>yeah, i'm just glad i haven't missed the big moment :)

<stikonas>there isn't that much right now yet

<stikonas>it's just two C programs

<Hagfish>sure

<Hagfish>but it's like booting the first computer, in a way

<Hagfish>it might not do anything, but it's sort of a clearly defined start point

<Hagfish>the set of all bootstrapped software is currently empty, but soon that set will grow and grow until it includes all software

<Hagfish>(although i guess the definition of "bootstrapped" will change over time)

<Hagfish>does it count if you didn't solder the transistors together yourself? etc.

<muurkha>this was a continuous source of controversy on the old homebrew computing newsgroup. comp.homebrew? I forget

<muurkha>which was for people using existing chips but not existing boards, basically

<Hagfish>yeah, that's an interesting dividing line

<Hagfish>and i think it's perfectly valid to talk about "bootstrapped within qemu", that's still an amazing technical achievement and rules out all sorts of attacks

<muurkha>maybe, I'm not sure it rules anything out against an active adversary

<stikonas>well, qemu or real hardware shouldn't really metter

<stikonas>*matter

<muurkha>it matters a lot

<Hagfish>an attacker wouldn't target qemu if they know a defender can just run the same code on real metal

<muurkha>real hardware isn't vulnerable to the compiler with which you compiled the kernel under which qemu is running

<muurkha>or with which you compiled qemu

<muurkha>Hagfish: not unless they've compromised the metal, but that only matters once we can just run the same code on real metal

<stikonas>qemu was built with gcc, uefi was probably built with gcc too...

<stikonas>even on baremetal

<muurkha>probably

<Hagfish>muurkha: ah yes, if they've compromised the metal, then it would make sense to hide the same behaviour in qemu, for consistency

<stikonas>yes, but you can compromise firmware

<Hagfish>true

<stikonas>you don't need to compromise hardware

<muurkha>and even if they haven't compromised the metal or the firmware, they could compromise qemu in an undetected way until you actually did compare the results

<muurkha>and then only if you respond to a discrepancy in a very persistent way

<Hagfish>comparing the results seems cheap, but i guess there's a question of which machine you do the comparison on :)

<muurkha>it's cheaper once the bare-metal results exist :)

<Hagfish>true :)

<stikonas>in any case if metal or fimrware is compromosed, it's unlikely to be trusting trust attack. It will probably be just backdoor (kind of like management engine)

<muurkha>yeah

<muurkha>still, barer metal would be great

<stikonas>well, builder-hex0 is closer to bare metal, we have that

<stikonas>but it's slow and somewhat hard to use (since it's close to baremetal)

<muurkha>in Accelerando there's a fun line about this which involves bootstrapping on a clock-radio microcontrololer

<stikonas>on the other hand stage0-uefi could probably be automated almost to the extent of stage0-posix

<stikonas>but if you want to do real baremetal bootstrapping with no firmware, e.g. on some arm system or riscv

<stikonas>then builder-hex0 is a good starting point

<stikonas>though you have to replace bios syscalls with minimal hardware drivers

<muurkha>yeah

<muurkha>you could presumably get a pretty usable system running an an 8-bit AVR talking to encrypted external memory

<muurkha>ARMs usually seem to have a lot of annoying HAL attack surface

<muurkha>but to a certain degree this is counting angels on the head of a pin, given the enormous progress we've already made

<muurkha>(I say "encrypted external memory" because the choice of the right authenticated encryption mode could ensure that hypothetical malicious external memory can't do anything worse than a DoS)

IRC channel logs

2022-07-09.log