IRC channel logs

2020-12-27.log

back to list of logs

<OriansJ>I could probably also eliminate the need for lseek if I knew of another way to know a file's size efficiently.
<bauen1>OriansJ: how many are approximately needed ? read, write, open, exit, mmap, clone/fork ?
<deesix>Around 15, I think.
<deesix>From AArch64 M1 definitions: BRK CHDIR CLONE CLOSE EXECVE EXIT FACCESSAT FCHDIR FCHMODAT GETCWD LSEEK OPENAT READ UNAME WAIT4 WRITE
<OriansJ>bauen1: I can tell you exactly: exit (syscall 1 on x86), access(syscall 33 on x86), chdir (syscall 12 on x86), fchdir (syscall 133 on x86), waitpid (syscall 7 on x86), execve (syscall 11 on x86), read (syscall 3 on x86), write (syscall 4 on x86), lseek (syscall 19 on x86), open (syscall 5 on x86), close (syscall 6 on x86), fork (syscall 2 on x86), getcwd (syscall 183 on x86), brk (syscall 45 on x86), chmod (syscall 15 on x86) and uname
<OriansJ>(syscall 109 on x86)
<OriansJ>So exactly 16 syscalls
<OriansJ>(with wait4 on AArch64 and AMD64 instead of waitpid but they are used the same and can easily be harmonized if it makes your job easier)
<OriansJ>absolutely nothing else is used syscall-wise as we don't format the disk or need a proper shell.
<OriansJ>only need the bootstrap kaem as init and we can do the rest from there
<fossy>getcwd can be easily dropped. chmod can be if the kernel has an option to default to executable files, fchdir and chdir can also be dropped
<fossy>if nessecary
<plasma41>fossy: You are correct. I didn't do my due diligence before I commented. Running `checkbashisms -f rootfs.sh` produces this output: https://paste.debian.net/1178461/
<siraben>OriansJ: with such few syscalls we could do this bootstrap on the MMIX architecture
<siraben>which GCC targets and newlib implements some syscalls for
<siraben>fossy: thanks, looks interesting
<siraben>oh i didn't know we can get rid of the shell dependency as well
<fossy>huh til about push/popd
<fossy>being bashisms
<siraben>TIL about checkbashisms
<siraben>nix run nixpkgs.checkbashism -c checkbashisms
<plasma41>fossy, siraben: checkbashisms is included in the devscripts package in Debian, btw.
<fossy>cool
<siraben>Am considering an inline directive for blynn-compiler
<pabs3>shellcheck also does some checking for POSIX sh compliance
<OriansJ>siraben: well yes. The idea is everything in mescc-tools-seed could be done on any arbitrary hardware that supports atleast 1 input and 1 output method. It is only on later stages do things get ugly.
<bauen1>that is very doable
<bauen1>i would really prefer to implement the more modern syscalls (i.e. mmap instead of brk, clone instead of fork), but if necessary i can just implement a compatibility later for brk / fork later on
<bauen1>i just don't really have enough time currently (as always) :(
<bauen1>fossy: you don't need chmod if you never implement access controls
<bauen1>i assume that waitpid uses an exact pid and not one of the special (0, -1, ...) values ?
<OriansJ>bauen1: waitpid is currently only used in kaem and mes.c (and future mes-m2) for spawning of a process and waiting for it to complete (then looking at its return code)
<OriansJ>(as MesCC spawns blood-elf, M1 and hex2 depending on the compile options)
<OriansJ>bauen1: we also would be more than happy to use the more modern syscalls if you wouldn't mind assisting us in knowing what needs to be done to switch to them; so that we can update accordingly. (what we don't know, we can't fix)
<janneke>for example, are these "modern" system calls just as portable as the "old ones"?
<janneke>think: bsd, the hurd
<janneke>we probably don't want to go modern if that means supporting *both* old and new?
<nimaje>seems like freebsd implements waitpid via the wait4 syscall
<OriansJ>nimaje: it isn't just FreeBSD; it is *all* POSIX systems. Including those a college student might write for themselves on arbitrary hardware they made for themselves.
<OriansJ>The root of trust needs to be as diverse as theoretically possible.
<OriansJ>and because all of the early stages should be able to cross compile *ALL* other targets; it forces any trusting trust attacker to compromise *EVERY* hardware platform including those made by college students with $300 worth of parts
<OriansJ>When we find MacOSX and Windows Developers; we will have them port the work there as well.
<OriansJ>So unless an attacker compromises *ALL* operating systems on *ALL* hardware platforms; we will detect trusting trust attacks 100% of the time.
<OriansJ>with sub-300 byte roots of trust (that can be verfied in a day)
<nimaje>well, that was an example of a system that doesn't have a waitpid syscall, maybe linux has a special syscall for waitpid or it is also implemented via wait4, but the previous discussion said there was a waitpid syscall and not a normal function
<OriansJ>nimaje: fair and that is one of the reasons why I am starting to engineer https://github.com/oriansj/M2libc
<OriansJ>To harmonize not only the libc and definitions with MesCC but also provide portable functions that wrap syscalls to make porting easier.
<OriansJ>(I have alot of work ahead of me)
<nimaje>seems like linux has for most of the wait* functions an special syscall instead of implementing the functions in terms of each other
<siraben>I use macOS but wouldn't be able to do the low level stuff on it
<OriansJ>nimaje: well Linux woman won by making it easier to port to Linux from Commercial Unixes
<janneke>OriansJ: on mes' wip-m2 branch, the mes c library can be compiled with m2-planet
<janneke>obviously that is needed to prototype the full source bootstrap
<janneke>but it's good enough to build mes
<janneke>(or it was, it may have bitrotten -- i'm planning to resurrect it the coming weeks)
<nimaje>and a linux wait(2) manpage (that I found online) says the clone syscall is unique to linux ("using the Linux-unique clone(2) system call")
<mihi>if you want it portable, you should replace fork/exec by posix_spawn: https://pubs.opengroup.org/onlinepubs/9699919799/functions/posix_spawn.html. Fork on the Hurd is no syscall but "emulated" by glibc, and posix_spawn is faster. Also modern glibc uses clone or vfork instead of fork syscalls if they are available (Linux 5.3 for clone, Linux 2.2. for vfork). Some non-POSIX systems like MinGW don't implmement fork,
<mihi>but implement posix_spawn (via Win32 CreateProcess API)
<nimaje>pretty sure posix only specifies which functions should be there not how they should be implemented (pretty sure freebsd implements posix_spawn via vfork and exec)
<mihi>main reason why to use vfork or clone on Linux is that it reduces memory/swap requirements when memory overcommitment is disabled (which you want to do on performace critical systems to avoid the dreaded OOM Killer)
<rain1> https://www.reddit.com/r/ProgrammingLanguages/comments/kkrmm9/bootstrapping_case_studies/
<rain1>just some random chat
<OriansJ>rain1: why don't you karma mine with a link to mescc-tools-seed. As "from hex to cross-platform C compiler, scheme interpreter and Haskell Compiler" should grab some attention.
<rain1>:P
<OriansJ>I don't care to do PR work but someone needs to do it or people will think stage0 is dead or something.
<OriansJ>^PR^Public Relations^
<OriansJ>(I'm happy to deal with Pull Requests)