IRC channel logs

<oriansj>minor git worktree notes for anyone who is unfamiliar: https://paste.debian.net/1240912/

<doras>oriansj: nice. I also use "git worktree remove" often.

<oriansj>rickmasters: did you look at the GFK filesystem proposal?

<oriansj>because I have a stupid idea

<rickmasters>oriansj: sorry, no, i mean i saw the repo, but i didn't see the details until i just looked at it now

<rickmasters>oriansj: you mean this page?: https://github.com/oriansj/GFK/blob/main/GFK%20standard.org

<oriansj>rickmasters: correct

<rickmasters>oriansj: what the idea?

<oriansj>well as that is a filesystem that could be written by hand in hex0 without much trouble. if your builder kernel did do disk writes in that format, then the C kernel could just be a build product written like any other file

<oriansj>kaem can be moved to /sbin/init and a kaem.run file populated and it could continue the rest of the way to Linux

<rickmasters>so, the next kernel can be in m2-planet, if that is what you mean by C kernel, and then a m2-planet executable could be written to read the next kernel and kaem and write the back in your file system format, then switch to new kernel

<rickmasters>... avoiding more hex0 work

<oriansj>good point

<oriansj>better idea honestly

<oriansj>as everything is just in memory at that point right?

<rickmasters>yes, all files are in memory and can be read by any executable

<rickmasters>i was thinking of supporting the ability to flush files back to any partition https://github.com/ironmeld/builder-hex0#manual-build-instructions

<rickmasters>if that helps, but one could append files together and flush them as one disk image currently

<rickmasters>catm /dev/hda bootloader

<rickmasters>catm /dev/hda1 newkernel kaem nextbuildscript etc

<rickmasters>something like that maybe

<rickmasters>whoops, forgot the file system

<oriansj>I'll have to think about that over dinner but it sounds like a good idea

<rickmasters>convert-to-GTK newfilesystem newkernel kaem initscript more-source.tar

<rickmasters>catm /dev/hda1 newfilesystem

<rickmasters>i dunno, i'm a little fuzzy on how it should work

<rickmasters>to clarify, the first arg of convert-to-GTK is a binary file that it writes to that would contain the new file system

<rickmasters>a GTK capable bootloader could load the kernel and jump to it

<stikonas>s/GTK/GFK/ I guess?

<rickmasters>... or just leave the kernel outside GFK

<rickmasters>yes GFK, sorry

<rickmasters>i think in the old days there was a /boot filesystem sometimes FAT that holds the kernel, but i'm no expert, its changed over the years

<stikonas>well, these days for UEFI boot there is ESP partition that has to be formatted as FAT32 (other fs can work but are not mandated by standard). Usually it is mounted on /boot/efi and can contain either bootloader or kernel

<stikonas>for non-UEFI BIOS systems it's either booting from active partition (for MBR table) or a bit more complicated for GPT (you to raw create bios boot partition, I guess a bit like ESP but without filesystem structure)

<oriansj>well I guess my first task is to create a create and unpack tool for GFK images.

<oriansj>with a special --master-boot-loader flag as the MBR is kind of special for x86

<oriansj>then get a bootloader written that can prompt the user for the filename to load into memory and execute

<stikonas>that tool would go to stage0-posix?

<oriansj>indeed as a submodule

<oriansj>we build it, the new bootloader and new kernel then use the tool to build the disk image and catm /dev/hda1 image.img to create our bootstrap filesystem

<oriansj>and just halt the system and then on boot type in the name of the new kernel filename (/boot/kernel)

<oriansj>(or honestly anything we want to call it)

<stikonas>yes, that's sounds reasonable

<stikonas>at least would make transition from builder-hex0 to next bootloader / kernel easy

<oriansj>then after we build Linux, we just reboot with the new init setup and tell the bootloader to use the Linux file we made

<stikonas>although I guess transition from builder-hex0-mini to builder-hex0 would stay manual

<oriansj>(or build grub and install a fresh filesystem on a second disk)

<muurkha>rickmasters: /boot was sometimes put at the beginning of the disk to make sure the BIOS could load from it, but also it was handy to have two root filesystems in case you screwed one up

<muurkha>same kernel, same /home, maybe same /usr, but different /lib/ld.so and the like

<rickmasters>muurkha: yeah, keeping within bios limits makes the code a lot easier, except switching cpu modes, that took me some time to get right

<muurkha>yeah, it's a pain in the ass

<muurkha>and a totally unnecessary one

<muurkha>I mean, it's necessary if you have to use the hardware you have

<muurkha>but it wasn't necessary for the hardware to impose that pain on you

<muurkha>it could have been designed better

<rickmasters>yeah, i thought i couldn't use 32bit memory without giving up bios access, but eventually found a way

<rickmasters>i should post that code to the osdev wiki

<muurkha>oh wow, a lot of people will appreciate that

<muurkha>even if most new machines are UEFI

<rickmasters>yeah, its worth doing, but most people aren't bootstrapping and will just use 32bit ATA drivers

<rickmasters>a lot more code but its out there

<muurkha>there are other cases where you want to invoke BIOS functions besides just reading from a disk tho

<rickmasters>muurkha: i hear ya. checking out, goodnight

<muurkha>goodnight!

<muurkha>it'll be a big contribution to the osdev wiki I think so I hope you do

<rickmasters>i'll plan on it

<oriansj>hmmmm, I'll probably want to eliminate the need for the superblock to be on the last block of the drive

<oriansj>and just make it a recommended standard when hand creating the file system.

<oriansj>and probably shift the leadblock slightly to allow builder-hex0 to fit on the disk before it would appear

<oriansj>hmmm actually, no need

<fossy>doras, stikonas: xbps is tar with extra metadata. by permissions and location, it means that it will not change them, i believe. what is missing around reproducibility with file permissions?

<fossy>adding noops isn't a big deal for bootstrap kernel IMO, but adding eg. gettime functionality is not the best route, removing it is way better

<fossy>stikonas[m], doras: parallelism was not something that I originally had in mind while live-bootstrap is incomplete. it introduced more problems than it solved, i found

<fossy>live-bootstrap is maturing well right now, though, so i wouldn't have problems with introducing parallelism

<oriansj>fossy: well that is on janneke to decide on the bare minimal syscall support for Mescc

<fossy>ultimately yeah

<oriansj>the question becomes if later I should add GFK support to the Linux kernel or let someone else do it

<fossy>gfk = ?

<oriansj> https://github.com/oriansj/GFK/blob/main/GFK%20standard.org

<oriansj>it is a file system standard that I wrote up to enable a filesystem to be something trivial for someone with a notepad and a hex editor to create/update/fix

<oriansj>Tonight, I am hacking together a minimal basic tool for generation of images files that conform to that spec

<oriansj>so that M2-Planet can build it, and a kernel and it can be written to disk to complete the rest of the bootstrap

<fossy>a

<fossy>ah

<oriansj>let us hope lseek is working correctly or things will get kinda funny fast

<oriansj>I think limiting binary programs to under 1GB should be reasonable for a GFK filesystem being built with M2-Planet on builder-hex0

<littlebobeep>oriansj: what do you mean binaries? I thought filesystems have limits on filesize of any file

<oriansj>and I don't think we will be needing to build a system image with over 65K files

<oriansj>littlebobeep: yes you are absolutely correct there

<littlebobeep>what is the downside for having higher limits?

<oriansj>littlebobeep: more comlexity in implementation

<oriansj>that being said GFK supports adjustable blockpointer size

<oriansj>16bits means 2^16 volume blocks supported

<oriansj>it also supports adjustable volume block sizes (default is 4096bytes)

<oriansj>I'm using a pointer of 32bits out of the gate as the default

<oriansj>as (2^32)*4096 (16TB) should be sufficient when trying to bootstrap Linux

<oriansj>2 levels of indirection would be (512*512*4096bytes [1GB])

<oriansj>(or 4 times that if one turns off the checksumming merkle tree)

<oriansj>but I don't image we have any files that large prior to bootstrapping Linux

<oriansj>and if we hit a need for another level of indirection support when generating GFK images; I can just add it

<oriansj>actually

<oriansj>nope, can't avoid having to calculate where to put the superblock in generated images

<oriansj>ok, it is 4AM and I need sleep. I've commited the progress I made

<oriansj>just need to add the dumping to disk

<doras>fossy: I want to allow bootstrap with a different umask.

<doras>If you'll change your root user umask to 0002 and run the bootstrap, you'll see that you don't get the same hash in tarball and XBPS packages.

<doras>In chroot mode, I mean.

<doras>This is because the file permission metadata is determined by the permissions of the files at packaging time, and if the umask is different, file permissions are different.

<doras>I'd rather not have to change the umask externally to the bootstrap process to keep it self-contained.

<doras>If mes supported the umask syscall, I would have set it to our expected value even earlier in the bootstrap.

<doras>In the rootless bootstrap mode, the umask is inherited from the user running the bootstrap. This is where I hit the hash diff.

<doras>stikonas: I think there is an issue with the make/gzip swap. I get a failure during bootstrap.

<stikonas[m]>doras: can you paste the error

<doras>I see that you change the "gunzip" to "gzip -d" manually in our scripts after your change, but it seems that it effectively means the "gunzip" command is never created.

<doras>However, later stages of the bootstrap requires it.

<stikonas[m]>Oh did I miss it?

<stikonas[m]>Where is it needed?

<stikonas[m]>I thought it ran to completion...

<doras>I'll paste my failure.

<stikonas[m]>And sysc did not have gunzip even before

<doras>stikonas: https://gitlab.gnome.org/-/snippets/3477/raw/main/automake-1.10.3-build.log

<stikonas[m]>Yeah, I see. I'll readd symlink then

<stikonas[m]>Or maybe deal with amhello

<doras>Any reason to avoid creating gunzip?

<doras>I like gunzip :)

<doras>Oh I see, our previous gunzip was a fake gunzip. It wasn't even passing the "-d" option as expected.

<doras>Usually it's a shell script wrapping "gzip -d".

<stikonas>doras: looks like automake hardcodes gunzip itself

<stikonas>doras: gunzip is always fake

<stikonas>I don't think you evne need shell script

<stikonas>it just looks at how it was invoked

<stikonas>(similar to busybox)

<doras>I'm familiar with the concept, but it seems Fedora packages a shell script.

<stikonas>anyway, I'm actually thinking of maybe just rebuilding gzip properly before automake 1.10 (with autoconf)

<stikonas>if autoconf that we have can deal with such old configure.in

<stikonas>that is present in that gzip version

<doras>Maybe build a newer version.

<stikonas>yeah, I can go for newer version too

<stikonas>just need to check what are autotools requirements

<stikonas>right now we don't have gzip "package" at all

<doras>Sounds like the best approach. Having a "full" gzip would be nice regardless.

<stikonas>because it was built so early

<stikonas>doras: I think we can go for gzip 1.3.12

<stikonas>1.3.13 "needs" automake 1.11

<stikonas>it doesn't build on my modern system... Though that shouldn't be a problem for live-bootstrap

<doras>Oh well. gzip from 2007 must be better than gzip from 1993 :)

<stikonas>I'm actully not sure what changed in gzip...

<stikonas>even ungz from stage0-posix seems to be able to do its job

<stikonas>fossy: remember you patched old gzip not to include timestamps?

<stikonas>in commit 016b9511d2fd6bac25a42ea15c298451a6daad30

<stikonas>1. we probably don't need to do that anymore since packages were switched to bzip2

<stikonas>2. there might be simpler way to remove timestamps from all earlier packages. Use musl stubs for time related functions

<stikonas>(s/musl/mes)

<stikonas>rather than building real mes libc functions

*littlebobeep is not sure what build dependencies are different between gzip, bzip2, and xz

<unmatched-paren>littlebobeep: i believe xz is written in C++?

<littlebobeep>unmatched-paren: https://git.tukaani.org/?p=xz.git looks like C, or?

<littlebobeep>this is interesting too: https://github.com/vasi/pixz

<littlebobeep>I think p7zip might be partially C++ which also implements LZMA2 like xz

<littlebobeep>some people talk about zstandard online but I never looked into it much: https://facebook.github.io/zstd/

<littlebobeep>weird its an RFC, I wonder how many RFCs came from Facebook 0_o https://datatracker.ietf.org/doc/html/rfc8478

<littlebobeep>it might be faster than LZMA2, but I don't know anything that can get better compression than LZMA2

<littlebobeep>haha hmm OpenBSD says "g" in their gzip stands for 'gratis' https://man.openbsd.org/gzip.1#HISTORY

<unmatched-paren>littlebobeep: oh, i see, i think the problem might be that it's build with cmake? which I know _is_ C++

<unmatched-paren>s/build/built/

<unmatched-paren>although i guess you could always write an ad-hoc makefile

<unmatched-paren>there was _something_ that made xz annoying

<unmatched-paren>i remember that there was some GNU project that only distributed xz tarballs for a while

<unmatched-paren>which made using it in bootstrapping difficult

<unmatched-paren>i think it was noted on bootstrappable.org

<littlebobeep>unmatched-paren: I see no reference to cmake in the Guix 'xz' package definition, would it be there?

<unmatched-paren>ah, there's also a configure.ac that i missed, my bad

<unmatched-paren>looks like it can be built with both cmake and autotools

<littlebobeep>This claims zstandard is 1300% faster at decrompression than xz but has no technical benchmarking details so meh: https://archlinux.org/news/now-using-zstandard-instead-of-xz-for-package-compression/

<oriansj>littlebobeep: that which is stated without evidence can be rejected without evidence.

<littlebobeep>XD

<oriansj>like my two year old who seems to believe I only need 3 hours of sleep because he wants to play at 7am. The conclusion was made long before compiling evidence to support its justification.

<unmatched-paren>oriansj: i remember seeing there being some reason why a crucial bootstrapping package only being provided in xz form was annoying; am i imagining things?

<mihi>unmatched-paren, I also remember that there have been some issues to build xz-utils with tinycc, which is why I built <https://github.com/schierlm/xzdec-min>

<mihi>but I also don't remember the exact issues (even at the time when I committed it and wrote the README for it)

<littlebobeep>mihi: Why does github say 5,3% C++ ?

<mihi>littlebobeep, when you click it it tells you that the repo has not even been indexed for programming languages.

<mihi>Probably some heuristics that guessed wrong (I see this alot with Modula and Oberon code)?

<littlebobeep>uhmmm why does it show language statistics if it hasn't scanned?

<mihi>I can assure you that it builds fine with tcc

<littlebobeep>mihi: Cool project, but it says only glibc?

<mihi>The included config.h is for glibc. If you want to contribute config.h for meslibc or musl, feel free to do so (should be easy to generate if you have working autoconf)

<mihi>maybe it even works unchanged

<unmatched-paren>if it's from autoconf, surely it's not a suitable file for bootstrapping? since it's machine generated

<littlebobeep>unmatched-paren: why does machine-generated matter?

<littlebobeep>if it is not binary

<mihi>it's hand-vetted and commented

<oriansj>littlebobeep: it absolutely matters to certain projects

<unmatched-paren>littlebobeep: that's fair enough, and I have made this argument (regarding the case of literate programming systems, where verifying that it matches the original document is trivial) but seemingly any generated file is unacceptable for live-bootstrap

<oriansj>for example live-bootstrap and stage0* generate files are explicitly forbidden.

<oriansj>^generate^generated^

<unmatched-paren>although i am unsure, in that case, how hex0 and kaem can ever work, since there's not really a way to go lower than a hex or binary assembler without e.g. punched cards

<oriansj>unmatched-paren: I hand toggle bytes into memory but yeah I get your point that most people don't want to go that far

<oriansj>and why bootstrap-seeds exists

<unmatched-paren>it's trivial to verify, so long as you have a trusted way to view the binary, that the hex and resulting binary correspond

<oriansj>and why the GNU project rightly rejects bootstrap-seeds; as binaries are not ever going to be considered source to them. Despite in that special case it was human written and the hex0 was just documentation.

<unmatched-paren>although now that i think about

<unmatched-paren>*it

<unmatched-paren>how do you view the binary as 00100010011 etc

<unmatched-paren>without software

<oriansj>leds attached to memory

<oriansj>(light emitting diodes)

<unmatched-paren>ah, ok

<oriansj>you turn the clock off for the CPU and just toggle the memory address you want

*unmatched-paren is not a hardware person :P

<oriansj>but make no mistake, there are *VERY VERY* good reasons why bootstrap-seeds starts with: *NEVER TRUST ANYTHING IN HERE*

<unmatched-paren>*unless you have checked that the hex0 code corresponds exactly

<littlebobeep>oriansj: I do not understand what the issue is with code produced by a machine, if it is in readable source form and can be modified by a human?

<oriansj>unmatched-paren: except you can't be certain it is a valid check unless you are able to remove software from between you and your inspection of the binary.

<unmatched-paren>littlebobeep: invisible unicode right-to-left text modifiers say hi

<oriansj>littlebobeep: it is trivial to generate a 1Mloc artifact and bury a subtle backdoor that even the best auditor would never find.

<oriansj>even if you look at human written C programs: https://www.ioccc.org/ there are a great many ways to hide bad behavior

<unmatched-paren>littlebobeep: for one example of a hard-to-find vulnerability in plain, seemingly readable text: https://trojansource.codes/

<oriansj>unmatched-paren: although stage0's sin will warn about every single non-ascii core byte https://github.com/oriansj/stage0/blob/master/High_level_prototypes/sin.c

<oriansj>note that: I was explicit on only allowing a subset of ASCII characters to be considered valid in source code files

<unmatched-paren>that makes sense, yes

<bauen1>unmatched-paren: you can hand toggle the SPI protocol to read from SPI NOR FLASH, been there done that (but with a provided clock because I don't have a debounce circuit)

<unmatched-paren>sorry, that sentence is a bit alien to me :) what's the "SPI protocol"?

<bauen1>unmatched-paren: https://en.wikipedia.org/wiki/Serial_Peripheral_Interface actually it's more of an electrical interface specification i guess

<unmatched-paren>so basically a protocol for electronic components to communicate?

<bauen1>unmatched-paren: yes, specifically a lot of small flash / eeprom / other sensor chips

<oriansj>wow, I was so tired last night I thought I pushed the partial GFK creation tool to github but I guess not

<mihi>oriansj, littlebobeep, unmatched-paren: I guess we can debate forever whether copying .c file names from a build log into a kaem file (while vetting the file names) is more or less hand-generated than copying single #defines from autoconf output (after verifying that glibc indeed does or does not provide that functionality) into a commented config.h file. Anyway, if you don't want to use config.h,

<mihi>you can build your own one, or just take the autoconf machinery from original xz-utils to build one for you which is 10 times as long :)

<mihi>(I also verified whether those defines are used inside the decoder code and removed all that are not, that is the second reason why the file got shorter)

<oriansj>mihi: it is ultimately up to the person doing the work, what they will or will not accept

<oriansj>and I fully support *EVERYONE* to do what they believe is correct.

<mihi>unmatched-paren, see https://www.youtube.com/watch?v=BA12Z7gQ4P0&t=0s for how to build an eeprom reader and programmer from a breadboard and a few resistors and LEDs :)

<mihi>oriansj, I fully agree to that statement

<oriansj>a debate on the subject is at most an effort to convince those who are undecided to accept a certain level of ideological purity and a certain level of practical reality.

<unmatched-paren>oriansj: that statement reminds me at some level of the free software movement as a whole, and the whole debate about hardware and BIOS

<oriansj>which they are free to choose for themselves without judgement but must understand that others may or may not choose to accept using your work.

<oriansj>unmatched-paren: well yes because that is literally what we are doing to a degree.

<mihi>"This program is distributed in the hope that it will be useful" :)

<stikonas>unmatched-paren, littlebobeep: well we bootstrapped xz in live-bootstrap, so I don't think shipping just xz now is problematic

<oriansj>we depended upon Linux/BSD/POSIX kernels we couldn't yet bootstrap for years because we didn't have a better option *YET* but now that rickmasters has given us a better option we can move past that. We will do the same for BIOS/FIRMWARE/MICROCODE in time but we have practical results now that are real progress compared to what we had weeks ago.

<unmatched-paren>absolutely :) and ultimately CPUs

<oriansj>and all secrets and magic will be revealed to all that wish to learn.

<stikonas>doras: I'm struggling to compile those gzip versions from 2007 or so. For now I'll just create gunzip symlink

<stikonas>or maybe just cp since ln is not available that early

<stikonas>if we want newer gzip, we can just build it in sysc...

<oriansj>oh and if you are bribed (say in the terms of $$$ Millions) to subvert the bootstrap, please do so in a fashion that also helps expand the bootstrap closer to the final goal. ^_^

<unmatched-paren>HAHA FOOLS! i used the bribe money to buy parts for a homemade CPU! :P

<unmatched-paren>actually buying parts for a bootstrapping cpu may not be the smartest idea...

<oriansj>unmatched-paren: don't forget to share the design documents so that others can make and audit it too

<oriansj>unmatched-paren: it is fine until we are able to make our own parts

<oriansj>once we have do it yourself lithography, then yes buying off the shelf parts might not be the most secure way of building a CPU but until then store bought is fine

<oriansj>morning rickmasters

<unmatched-paren>oriansj: to be clear, i was joking (if you're being sarcastic/etc. i'm sorry, but it wasn't very obvious)

<unmatched-paren>it kind of looked like you were taking me seriously :)

<rickmasters>oriansj: good morning, hope you got some sleep

<oriansj>unmatched-paren: oh I am quite serious. If you are offerred $$$ Millions to subvert the bootstrap, I want you to take the money. I just ask that you still help with the bootstrap and not do 100% subversion all the time because you'll be detected quickly and then they might want the money back.

<oriansj>rickmasters: only 3.5 hours because my kid decided he wanted to play the second the sun came up.

<oriansj>it'll still be a net win for all of us if you get bribed that amount.

<oriansj>if someone offers you less than retirement amounts of money to subvert the bootstrap say, "No one with the ability to do what you want will do it for less"

<oriansj>think of it as a collective bargin we all agree to that converts that risk to a net gain for all of us.

<unmatched-paren>presumably the person bribing would be technically illiterate, so i could pretend to add a backdoor involving unicode and they wouldn't know that hex0 disallows it :) then i could take the money

<oriansj>also a valid option in that case.

<oriansj>and if threatened do it poorly so that it is immediately detected and claim that we are just super good at detection and you needed much more time to do the work (years even)

<unmatched-paren>hmm... 0x0000 is the noop instruction, right? i could also pretend to smuggle in a noop with a pull request :)

<unmatched-paren>s/pretend //

<unmatched-paren>s/to //

<oriansj>and money to fund yourself to get the time required to do the creation.

<unmatched-paren>and say it's a super h4xx0r instruction that destroys the whole bootstrap

<oriansj>well 0x0000 is add byte [eax], al for x86, andeq r0, r0, r0 for armv7l, and other things depending on the architecture but yeah in bootstrapping 0x00000000 is probably just being used as a NULL value

<unmatched-paren>ah, i though it was NOP

<unmatched-paren>but the fact that it isn't makes it even easier to obscure the fact that it does nothing!

<oriansj>an easy way to flag your commit is check in a unicode character (even whitespace)

<oriansj>we have sin after all

<oriansj>and if you detect someone doing subversion in the bribed manner, just provide patches that fixes the subversion as if it is any other type of bug. If they do it in the blackmail manner, raise big red flags all over the place.

<oriansj>That way the bribed person stays a productive member of the community (who just needs someone to do clean up on their work)

<oriansj>but a blackmailed person is immediately flagged for a full review of everything they have ever submitted.

<oriansj>That way blackmail becomes a non-starter

<oriansj>but bribes become the only valid attack vector

<unmatched-paren>we could also have some kind of secret comment that looks like something technical but actually doesn't make any sense that marks a subversion point: something like `# NOTE: possible TLREGXX overflow in the HTML firewall` :P

<unmatched-paren>hypothetical scenarios are fun to think about

<oriansj>again, obvious subversion is an immedate blackmail flag and needs to be made into a *BIG* issue immediately.

<oriansj>subtle subversion is just a fixit ticket as a github issue or a bug report. (ideally with a patch)

<oriansj>the details should not be discussed online to prevent third parties from knowing what they are besides the vague obvious/subtle distinction which provides them nothing useful to work with for knowing which you are doing.

<oriansj>anyway back to getting GFK filesystem generation implemented.

<Hagfish>i really like that video of manually setting the binary values in an EEPROM. i think the circuit would benefit from a binary counter and 16 push buttons, though

<Hagfish>that way you could speedrun flashing the whole 2k memory by hand

<oriansj>or toggle switches

<Hagfish>i'm thinking about how to make the writing process as convenient as possible

<Hagfish>choosing one of the 16 buttons should enter that value into the memory, and automatically increment the address

<oriansj>oh, I was thinking of how to make the electrical circuits as simple as possible

<Hagfish>yeah, that's a different (and maybe more important) concern

<oriansj>but yeah that would probably be cool, like 1 hex char per button

<Hagfish>right, that's what i'm thinking

<oriansj>then typing a byte just becomes 2 button presses, yeah that'll definitely enable hex0 bare metal speedruns

<Hagfish>2 presses?

<Hagfish>couldn't the circuit increment the counter automatically?

<Hagfish>the timing would be really tricky, admittedly

<oriansj>4bits is 16 different values; 8bits is a byte

<Hagfish>oh, sorry, yes

<oriansj>so first press for first 4 bits, second press for second 4 bits

<Hagfish>yeah, it would be typing nybbles

<oriansj>unless you wanted 256 buttons

<oriansj>say in an 8x8 matrix; would allow really quick entry of binary programs

<Hagfish>not impossible to wire, although learning the keyboard would be tricky

<Hagfish>a base64 keyboard would be cool

<oriansj>Hagfish: we already have those

<oriansj>they are just called keyboards

<Hagfish>heh, i mean the implementation would be cool, but yes

<oriansj>well I guess if it was in that sort of low level base64 to deposit bytes to memory it would probably be neat

<Hagfish>right

<unmatched-paren>a keyboard with `0` and `1` only :)

<oriansj>that already is a thing: https://johnmu.com/2021-binary-keyboard/

<Hagfish>i was thinking that one meaningful level of purity/security would be to use one of those 6502-based computers (like Ben Eater's channel explains) with a hand coded EEPROM firmware

<Hagfish>people have written sdcard drivers for the 6502, so you could use it (and the keyboard) to write an sdcard

<Hagfish>which could then (through an sdcard to ide converter) be used as the basis for a 286 or something

<oriansj>Hagfish: well some sdcards have z80s inside of them, so with some cleverness you could have the SDcard generate the contents of the SDcard in a bootstrap fashion.

<Hagfish>that still involves trusting the bios, sadly, but if that bios is really a ROM (written decades ago) then that gives a certain type of confidence

<Hagfish>i don't think you could get the sdcard to accept writing from a simple circuit unless you could rewrite its ROM

<Hagfish>but yeah, just thinking about that makes me imagine the Simulation Hypothesis and The Matrix

<oriansj>well unfortunately I don't have good answers for that problem. But I do believe that if you live your life like it actually matters, the worst that happens if you have a life you find meaninful.

<oriansj>^if^is^

<Hagfish>yeah, i think that's about as good an answer as philosophy will allow

<oriansj>dongcarl: what do you think the cryptocurrency community would do if they heard my stance on bribes?

<doras>stikonas @stikonas:libera.chat: sure. Thanks.

<stikonas>doras: seems to be working fine now, I already pushed the change

<stikonas>last time I think when I removed gunzip I ran live-bootstrap till findutils...

<stikonas>which is just 2 packages short of the failed one...

<doras>And here I am being careful and running full bootstraps before each MR :)

<doras>Even multiple ones!

<stikonas>doras: yes, I usually do run multiple bootstraps before each MR...

<stikonas>but yesterday you made some MR, so I switched to running yours

<stikonas>anyway, fixed now. An I'm now testing a fix for zero size bzip2 package (https://github.com/fosslinux/live-bootstrap/issues/156)

<stikonas>and then more of your MRs...

<doras>Thanks :)

<doras>I think getting https://github.com/fosslinux/live-bootstrap/pull/165 in would be nice since it changes quite a few hashes.

<doras>By the way, I think most of the hash changes are due to "--numeric-owner --owner=0 --group=0" and not "--mode=go=rX,u+rw,a-s".

<doras>I mostly needed the latter, but went along with the command that was suggested here.

<stikonas>doras: I have a question regarding https://github.com/fosslinux/live-bootstrap/pull/163/files#diff-bb3655625ec3a00693f39b72442d345b962ebafafe06d36cb6cff6dc6ba4e566R372

<stikonas>wouldn't that skip rm step if /dev/null exists?

<stikonas>I think that might break things

<doras>It skips only if it's a character device.

<stikonas>oh, maybe test -c is good enough

<stikonas>yeah...

<stikonas>there is some bug (in bash?)

<stikonas>so that after musl's configure script runs

<stikonas>then /dev/null is replaced with normal file

<stikonas>but I guess test -c wouldn't be happy with it

<doras>I thought it was because we created tarball packages before we mounted /dev/null, and fossy added some redirects some output to /dev/null at this stage, which doesn't exist.

<doras>At least this is what "git blame" suggested.

<stikonas>no, this was before packages

<stikonas>fossy might have adjusted something

<stikonas>but I remember that musl's configure was breaking /dev/null since I first added it

<doras>Hmmm...

<doras>I'm referring to at this commit: https://github.com/fosslinux/live-bootstrap/commit/968d2ea7d1b9361e43366ac49b7ac4614560a27d

<doras>I haven't seen /dev/null being removed anywhere else, but I also haven't looked hard.

<doras>And this commit definitely added that line.

<stikonas>it might we had non-fuctional broken /dev/null

<stikonas>well, normal file that was swallowing redirects instead of char device

<stikonas>but the problem was before that

<doras>I see.

<doras>My 3 open MRs are now fully tested and ready from my POV.

<doras>After those I'll still need the mes update though.

<stikonas>yeah, I remember about mes too

<stikonas>mes updates are slightly annoying though, as all early checksums change and have to be recalculated one by one (--update-checksums doesn't work on them)

<doras>Oh. I wasn't aware it doesn't work with them.

<doras>stikonas: that "# Manually install bzip2 (cannot replace binary while running)" comment still doesn't make sense to me.

<doras>fossy as well.

<doras>It was added here: https://github.com/fosslinux/live-bootstrap/commit/f1600467a7455a306b37bd5d67990d2e368d6eb8#diff-a0a2a53267a9978c5b5d7f947b814e76695989b2d9dcc6fea6586a03f56c20b1R16

<doras>It's also not clear which binary it is which is running that cannot be replaced. The only binary running at that point should be bash (and "install" itself).

<doras>But regardless, "install" should be perfectly capable of replacing a running binary without any issues.

<doras>I assume we already do this with bash itself during the bootstrap, too.

<Hagfish>yeah, that does seem strange

<Hagfish>it's nice when you find that someone has commented their thinking, but sometimes a comment opens more questions than it answers :)

<stikonas>doras: bzip2 does not like to be replaced

<stikonas>it's not install that does that

<stikonas>it's bzip2 unpacking .tar.bz2 package that contains bzip2

<doras>Perhaps it's worth mentioning the unpacking part in the comment.

<doras>But more so to backport whichever fix needs to be backported for this issue. It's quite ridiculous.

<stikonas[m]>doras: I don't think there is simple fix

<stikonas[m]>bzip2 version we used is modern

<stikonas[m]>What I can do is to move bzip2 to other path

<stikonas[m]>Before unpacking bzip2

<doras>That would be a more reasonable workaround. Rename it to "bzip2.old" or something.

<stikonas>well, I moved it to /tmp/bzip2

<stikonas>doras: though I think I'm hitting some other bug

<stikonas>bootstrap should have failed due to changed checksum, but it's continuing

<doras>That's... Odd.

<doras>stikonas @stikonas:libera.chat, does your bootstrap.cfg in tmp mention UPDATE_CHECKSUMS?

<doras>Also, have you ran a manual sha256sum to verify that the hash actually changed?

<stikonas>no, it has export UPDATE_CHECKSUMS=False

<stikonas>and hash changed...

<stikonas>well, it was empty package, not it's not empty

<stikonas>I'll do one more run with log

<doras>I'm suggesting that maybe it's still empty for some other reason.

<doras>So worth checking.

<oriansj>well this is certainly going to need some cleanup after the fact

<oriansj>and I am uncertain how much of a gain the extra flexibility is.

<oriansj>16TB should be enough space to bootstrap Linux

<Hagfish>i'm trying to think of what would have to go wrong for that to take more than 16 TB

<oriansj>Hagfish: clearly god forsaking us all

<Hagfish>maybe logging every processor instruction in ASCII?

<oriansj>block sizes of 512+4096bytes should be enough

<oriansj>block pointer sizes of 16+32bits would enable a low end of 32MB disk support (for 512byte block size and 16bit pointers) and a high end of 16TB disk support (for 4096byte block size and 32bit pointers)

<Hagfish>that's a nice balance

<stikonas>doras: so it is not just bzip2 package

<stikonas>all packages up to gawk fail

<stikonas>which I think means we have a bug in mescc-tools-extra/sha256sum

<doras>:(

<stikonas>oh, maybe that sha256sum does not accept piped input

<stikonas>yes, that's the problem

<stikonas>doras: I created a bug https://github.com/fosslinux/live-bootstrap/issues/166

<stikonas>I guess the fix is to create a temp file...

<stikonas>ok, I have a fix now for both issues. Just need to test it, get new checksums, etc...

IRC channel logs

2022-05-15.log