<stikonas>maybe because I had to inject busybox into /after/bin/sh <stikonas>but anyway, we still need to somehow properly reach bash... <stikonas>but my assembly is not strong enough to fix this :( <siraben>OriansJ: did my email properly send to the mailing list? I don't use mailing lists often so I may have just replied to you <fossy>stikonas: iirc it might have not worked under busybox but did under kaem <fossy>siraben: sorry my assembly is not very good either <stikonas>fossy: for now I replaced that jecxz with I think jm and then it compiles <stikonas>fossy: so I think it might be possible to get musl compiled if that helps us at all... <stikonas>(I don't have any proper scripts yet, most of my testing was manual) <fossy>stikonas: that would be useful.... <fossy>more complete than mes c library <stikonas>configured it with CC=tcc AR="tcc -ar" sh ./configure --target=i386 <stikonas>but I guess without shell we can still pre-create configuration include file <stikonas>then removed src/complex src/fenv, patched that sigsetjmp.s file <fossy>stikonas: is it autotools or no <stikonas>this needed some manual help due to missing coreutils <stikonas>which creates about 20 line config.mak file with variables <fossy>what if we used heirloom bash <stikonas>but that configure script is not essential <fossy>stikonas: yacc was segfaulting right <stikonas>I probably need to intall musl more properly... <stikonas>although, I think I was able to recompile libtcc.a and tcc <deesix>OriansJ, yt_, I'm switching more archs to M2libc in M2-Planet tests (just saying to avoid duplicated work). <bauen1>fossy: currently the only syscall supported is exit, i'm working on a proper process model and the open syscall, bash requires quite a few different syscalls (mostly related to process groups) so i'm not sure <bauen1>fossy: if you can give me a list of syscalls (normally) used i can tell you if it is feasible to implement without bloating the kernel <fossy>yeah it uses process groups quite siginificantly from strace <fossy>also the rt_* family (idk what that is) <fossy>stikonas[m]: i see two options for flex/bison <fossy>1. musl (full) or just src/multibyte make it into a library then nbs <fossy>2. use older versions of flex <bauen1>fossy: it kind of depends on how much you can patch out (while retaining critical functionality) tbh, implementing stub syscalls that just return -EPERM (or similiar) is also very easy <bauen1>fossy: but yes, it will be a bit before i start to think about process groups, job control or controlling terminals <stikonas[m]>We don't really need interactive terminal for bootstrap kernel <stikonas[m]>fossy: since older version of heirloom flex might not be able to bootstrap flex. Although, if full musl fails, that's an option <malina>stikonas[m], kinda funny, I am building/bootstrapping my musl lfs now, (4 years since I did the original glibc one) and I came across some of your comments or repo and so on. where you mentuoenc heirloom. I happened to compile it myself just some hours ago but for different reasons. I didn't have tr even so kernel headers wouln'd install :D <malina>I am trying to keep the toolchain set as minimal as possible so I grabbed the tr from heirloom and compiled that :) <malina>the flex however, failed iirc. but I didn't realyl poke around much in it [heirloom] right now. <malina>being totally new to musl so. but by the time I return to the heirloom , I am sure you troopers are already far ahead again <stikonas[m]>Can't be me, I've never heard about heirloom even 3 days ago <malina>OH, it seemed from above you are giomarco or something? <malina>suckless/heirloom/bb/toybox , all KISS. ***ChanServ sets mode: +o rekado_
***rekado_ is now known as rekado
<malina>ya, I ddn't check, was just saw 'giomasce' then came across it later today myself bychance, since im doing finally a musl strap. nvm. <fossy>stikonas[m]: i mean older versions of GNU flex <stikonas[m]>Anyway my testing yesterday showed that musl might be feasible <stikonas[m]>And I need to understand implications of jecxz better... <Hagfish>i know javascript tooling written in rust isn't an obvious topic for this channel, but i'm starting to be impressed with how Deno is replacing NodeJS <Hagfish>it's something that is often taken for granted, but somehow they had managed to get wrong in their v1.0 design <Hagfish>basically, all dependencies would be defined as just a package name, without a specific version, and even when you specified a version, there's no guarantee that the code associated with that version wouldn't change <Hagfish>but now they guarantee: "All files served from the registry are immutable. They can not be changed or removed by the package author." <Hagfish>(at the cost of centralisation, although in principle it should be possible to host your own registry) <Hagfish>and they plan to improve things further: "Give all modules a score based on ... are dependencies pinned to specific version" <Hagfish>it'll be interesting to see how well packages deal with security vulnerabilities in their (transitive) dependencies <Hagfish>i don't think they have an equivalent of "npm audit" yet, or a culture of packages updating their dependencies to non-vulnerable versions <Hagfish>there's also the question of whether packages backport security fixes to old versions, or if the solution for a vulnerability in version 3.x is to upgrade to version 8.y <Hagfish>maybe next year it will be clearer what the engineering culture is like in that ecosystem, and whether the packages available are enough to base any serious projects on <stikonas>ok, finally got some more progress on live-bootstrap, although, on slightly orthogonal thing. grep seems to build fine <stikonas>(and somewhat newer than guix in commencement.scm, got 2.4 working rather than 2.0) <stikonas>bauen1: so maybe then we build bash later, not in bootsrap kernel. but do you know if it would be possible to easily reboot into newly compiled kernel? <Profpatsch>Hagfish: are they just making all the golang mistakes all over again? <gforce_d11977>fossy: are you there? i have a proposal for CI-integration of live-bootstrap: question: i will introduce an ARG1 of 'CI', which does the things -> (./rootfs CI) - are you OK with this approach? <Hagfish>Profpatsch: yup, it seems like it, except they are also trying to fix them. maybe that's easier with a newer ecosystem, although it would have been easier not to make the mistakes in the first place <gforce_d11977>fossy: i have problems starting 'init' from your live-boot in a x86_64 qemu-instance. "file init" says it is "ELF 32-bit LSB executable, Intel 80386, version 1", maybe this is the underlying problem? <gforce_d11977>fossy: Run /init as init process -> Failed to execute /init (error -8) <gforce_d11977>stikonas: i "compiled" the AMD64 hex0 kaem and will test (fingers crossed!) <stikonas>well, I only tried AMD64 manually on mescc-tools-seed <stikonas>since bash is so heavy on syscalls, it might be easier later to do more stuff earlier on bootstrap kernel if we insert it before bash... <gforce_d11977>wow, now further progress on QEMU, there seems to be more wrong than just init. init now works, but i get: <gforce_d11977>Run /init as init process +> ./bootstrap-seeds/POSIX/x86/kaem-optional-seed mescc-tools-srandom: fast init done <gforce_d11977>Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000100 <stikonas>that printk from kernel that just happened to be printed at the same time <stikonas>gforce_d11977: I think you need to look at why your qemu fails to run 32 bit binaries... <gforce_d11977>stikonas: when i just run on my laptop: bootstrap-seeds/POSIX/x86/kaem-optional-seed mescc-tools-seed.kaem.run <gforce_d11977>i get: +> ../bootstrap-seeds/POSIX/x86/kaem-optional-seed mescc-tools-seed-kaem.kaem <gforce_d11977>the error is strange and i a chroot the live-bootstrap works for me, but the real test (a minimal x86_64 qemu) should also work <gforce_d11977>fossy: do you test in qemu or chroot? for the qemu-case: what is your kernel .config? <stikonas>I think both fossy and I use both qemu and chroot <stikonas>You should likely turn this on, unless you're 100% sure that you don't have any 32-bit programs left. <gforce_d11977>stikonas: thank you, i did not know this, i was always thinking, the CPU handles it automatically <stikonas>well, running executables involves both kernel and CPU <gforce_d11977>[ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name <gforce_d11977>oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),task=vm,pid=108,uid=0 <stikonas>gforce_d11977: how much RAM do you have? <stikonas>well, RAM is needed to store all sources... <stikonas>where did you get a box with 128 MB ram? <stikonas>and most of your ram is already used up before you even start bootstrapping <gforce_d11977>+> cd tcc-0.9.26 +> catm config.h +> /after/bin/mes --no-auto-compile -e main /after/bin/mescc.scm -- -S -o tcc.s -I /after/mes/lib -I /after/mes/include -D BOOTSTRAP=1 -I . <gforce_d11977>mes[559]: segfault at 0 ip 00000000010187d4 sp 00000000ffbc3454 error 6 in mes[1000000+1b000] <gforce_d11977>Code: 83 c4 04 85 c0 8b 45 08 89 45 fc b8 00 00 00 00 89 45 fc 85 c0 b8 00 00 00 00 bb 00 00 00 00 50 89 d8 8b 5d fc 01 d8 89 c3 58 <88> 03 80 <bauen1>stikonas: yes, you could implement something like kexec in myunix, or you could implement module loading, so you could have the core kernel very simple and then as you get tinycc available build + load more complex modules (like an journaled ext4 rw driver) <bauen1>stikonas: it would probably be easiest to make myunix (or a similiar bootstrap kernel) capable of (cross) compiling (and all dependencies necessary) the linux kernel, glibc, bash, gcc, and other toolchain tools you need <bauen1>stikonas: so you somehow get to myunix, then build the intermediate toolchain (in stages, starting from hex0 or something similiar) up until you receive a minimal linux system from which you can continue bootstrapping other things with potentially more complex feature requirements against the kernel (maybe utilising containers and other features if you want to) <bauen1>stikonas: this minimal linux image could then serve as the base for distributions to compile their own distribution-specific (patched) toolchain <bauen1>i personally would really like allowing modules to be loaded into the myunix kernel, care must be taken to ensure that there is no cross-contamination, but makes bootstrapping the bootstrap-kernel easier <bauen1>while you can write a rw ext4 driver in (pseudo) assembly, it isn't very fun and probably harder to verify that it does the job correctly <bauen1>in the end a modular to micro kernel might be the best thing for the bootstrap-kernel <stikonas>we can probably reach tcc and some basic utils with fairly few syscalls <bauen1>do keep in mind that myunix is an experiment of mine, so has some weird C code, e.g. using enums as error values, so while it can be compiled using tcc it might not be a good fit for bootstrapping <stikonas>at first we can try to use some slimmed down linux kernel... <gforce_d11977>stikonas: my qemu-instance runs with an 'allnokonfig' kernel + 6 options (e.g. ELF + 32bit support 8-)) <stikonas>ok, it might be worth documenting which options are needed <gforce_d11977>but it still has procfs,sysfs and printk support, so it can be around 300k smaller <gforce_d11977>(but when it comes to small kernels, we can also boot an early 2.6kernel which boils down to ~300k <bauen1>that's quite good, very close to ~280kb myunix currently uses if you disable debug informations <fossy>gforce_d11977: please do document your config <fossy>if you could pastebin it I would be very ahppy <fossy>But it sounds like it is not working? <fossy>gforce_d11977: can you elaborate on how the CI would work? <fossy>stikonas: any progress on musl? all good if not <stikonas>fossy: first of all I have grep (but that is a bit orthogonal) <stikonas>fossy: since musl build needs rm/mkdir, etc... <gforce_d11977>fossy: i will add documentation about kconfig/config tomorrow <stikonas>fossy: it's a huuuge Makefile for coreutils... <gforce_d11977>stikonas: you are right, these sizes are compressed, so 'myunix' is much smaller <fossy>gforce_d11977: myunix is not complete enough yet for use <stikonas>fossy: I've got most important utils working (last one is "ls") <stikonas>that's in my test environment, not in live-bootstrap, but usually it's close enough <fossy>gforce_d11977: its going to be because of the slimmed down kernel, some option will be missing <fossy>stikonas: cool, what tools do you think we need? <stikonas>basename cat chmod cksum csplit cut echo expand factor false fmt fold head id kill li\ <stikonas>nk ln logname mkfifo mkdir mv nl od paste pathchk printf ptx pwd readlink rmdir seq split sum tai\ <stikonas>fossy: some are a bit broken, but I think important ones are working <stikonas>I probably need another 2hours or so to finish this <stikonas>but I think we can live without ls for a bit longer <fossy>stikonas: I think we can too <stikonas>anyway, I'm now starting to test coreutils in live-bootstrap <stikonas>fossy: hmm, something is wrong with GH PR request... <stikonas>I've pushed those fixes into my branch, but somehow PR didn't spot new changes...