IRC channel logs
2022-12-26.log
back to list of logs
<stikonas>this should prevent environmental variables from propagating backwards from child to parent <stikonas>and I think environmental variables in M2libc/UEFI are now fully functional <Hagfish_>unsigned data_size = strlen(data) * 2 + 2; <Hagfish_>great find, though, and legendary work, as always <stikonas>Hagfish_: not exactly off by two error, it's just not necessary to count null terminator there <stikonas>though in this new change I actually need it to be 0 for empty string <stikonas>I should probably make it more obvious by using uint16_t* rather than char* <Hagfish_>err, that's not more obvious to me, but yeah, anything that makes it more obvious sounds like a good idea <Hagfish_>i like that you documented your assumption for _wipe_environment too <stikonas>anyway, next thing I'll have to implement is change directory functionality <stikonas>but that relies on having environmental variables working <stikonas>cause I need to pass current working directory to child processes <Hagfish_>it's unintuitive to think of subdirectories not being the same "environment", so that never would have occurred to me as being a problem, or something that needed implementing <stikonas>well, you need to somehow make relative paths absolute <stikonas>if you are at root level, you call e.g. program subdir/file <Hagfish_>yeah, i guess something has to be keeping track of what directory you are in / what a relative path means <stikonas>but once you do "cd subdir", you can call program file <Hagfish_>yeah, i guess the PATH is part of the environment, so i can see there is a link there <oriansj>thank you for a nice M2libc Christmas present ^_^ <stikonas>this was probably more complicated than some other methods of implementing uefi variables, but now we are compatible with UEFI shell <fossy>stikonas[m]: patch looks good, thanks <fossy>i'm going to push a few smalelr fixes in the next few days before i pr python, just small adjustments that are needed <fossy>btw, i'm planning to remove xbps from live-bootstrap. the benefits that appeared to exist didn't really work out in the end, and the multiversioned nature of live-bootstrap means xbps isn't really an appropriate fit. the tarball fakeroot system we are using is completely fine for everything we do (and now that the finish line is very clear) i don't think its required <stikonas[m]>fossy: yeah, do some smaller pushes first, that should make big python PR a bit smaller <fossy>stikonas[m]: even still, with tarball, unless a file has been overwritten, tar tf | xargs rm <stikonas>well, yeah, we could do that... anyway, I think I'm fine with removing xbps <stikonas>it adds more complexity that what we really need <fossy>yes, we use openssl for other things now, no reason not to keep it around <fossy>it becomes a prereq for python at some point too <oriansj>that would put atleast 2 distros in immediate bootstrap range (void and gentoo) correct? <stikonas[m]>Of course once you start using distro packages, they would be using pregenerated files... <stikonas[m]>But at least we are giving them all the tools to rebuild those if they want <stikonas>I suspect other distros should also be close one we have newer GCC and python <stikonas>hmm, perhaps storing env lines in M2libc was not the best idea... <stikonas>I have to do quite a bit of conversion back and forwards <stikonas>maybe linked list would have been better <stikonas>now I always have to parse strings like var=value <oriansj>stikonas: well bad ideas, allow us to understand the problem better <stikonas>well, in terms of code it's might be alright if I refactor it into function... <stikonas>though speed is not the first priority in bootstrap <oriansj>well it is down on the list but if a test takes 40minutes, then you probably are going to have brutally slow development process. <stikonas>oh no, nowhere that slow... environment list in stage0-uefi would be 20 or so variables <stikonas>so perhaps it's not worth optimizing too much <oriansj>if it takes under 10 seconds, then it is faster than the start time of a C# static binary <Christoph[m]>It sounds as if you fix the order in which programs get bootstrapped by hand. Could this be automated? Like similar to guix, where each package specifies all its dependencies and guix figures out a path through the emerging directed acyclic graph. <oriansj>Christoph[m]: well yes, using guix in the step after guile gets bootstrapped <oriansj>prior to that step however, we do have to manually craft the order of things getting built because it is a shell script running the show (kaem-optional-seed) <Christoph[m]>Ah, I see. You don't want to reimplement this feature of guile in the shell script. That makes sense, thank you! <stikonas>oriansj, Christoph[m] it is possible to use dependencies after coreutils is built, but I'm not sure if it's woth the effort <stikonas>if you want, you can implement a dependency resolver using tsort <stikonas>but it's also nice to have some flexibility that manual ordering let's us <stikonas>e.g. rickmasters said that make is hard to run under builder-hex0 but he wants to have tcc 0.9.27, so I've swapped the order <stikonas>and one is not really a dependency of the other in that sense <stikonas>well, I guess we could add a dependency of make on tcc 0.9.27... <stikonas>so we could still use dependency resolver <oriansj>stikonas: good point on soft vs hard dependencies. <oriansj>it is easier to build with make but we can just as easily create a build script <stikonas>oh yes, make can also do dependency resolution <Christoph[m]>That sounds like an anti-dependency: make cannot run under builder-hex0. <stikonas>well, at least not in the current form... <stikonas>well, if you can come up with something nice, I'm sure we can take a look <duplexsystem_>as anyone looked into xz-embedded? I'm trying to set it up in live-bootstrap but curious if anyone has tried it before <stikonas>though you would still have to keep a manual list in parts.rst <Christoph[m]>Are all the bootstrap steps also available as guix packages? Guix has tools to study the dependency graph, for example a visualization. <stikonas>duplexsystem_: it might work, looks like something that could be built with tcc <stikonas>Christoph[m]: guix only implements shorterned bootstrap where it does not care about pregenerated files <stikonas>Christoph[m]: oriansj and I once created a partial dependency graph of live-bootstrap but it got a bit out of date <stikonas>Christoph[m]: the thing is that it's hard to tell exactly what the dependencies are <stikonas>especially once you start using autotools <stikonas>./configure scripts run quite a few tools that are not immediately visible from package build script <stikonas>duplexsystem_: if you are willing to do a bit of extra work, you could try to port it to mescc-tools-extra <Christoph[m]>Yes, pregenerated files, I think, ghc ships those with its tar balls. When compiling from the repository, one has to generate them, and that brings in more dependencies. Do you mean stuff like that? <stikonas>Christoph[m]: yes, that's one of the things we rebuild <stikonas>so far it's quite simple and doesn't do any error checking <stikonas>but should be good enough to get something running <stikonas>(oh, so far I've only added support for cd'ing into absolute paths, let me see if I can quickly fix that) <Hagfish_>i like how you replace asm("lea_rdi,[rsp+DWORD] %8"... with a simple strcpy. does that make it more portable? <duplexsystem_>stikonas: I'll try that. What subset of C does mescc-tools-extra support <stikonas>duplexsystem_: it's basically M2 subset (which is somewhat evolving...). So we don't have e.g. switch, there is no "++" or "--". For more complicated statements parenthesis are recommended to set the order or operations, i.e. 2 + 2 * 2 should be 2 + (2 * 2) <stikonas>and poitner arithmetic is not working so it you need ptr + sizeof(type) rather than ptr + 1 <stikonas>duplexsystem_: all other programs are examples of what you can use <stikonas>M2-Planet itself is written in smaller subset <stikonas>Hagfish_: no, the previous code is just copy paste of linux code <stikonas>on linux you call kernel syscall to set directory <stikonas>uefi itself has no concept of current directory <stikonas>so as I've said earlier, we just store it in environmental variable <Hagfish_>yup. so it's really like you're implementing a kernel on top of uefi? <stikonas>though everything runs in the same privelege level <stikonas>maybe it's more like we implement POSIX compatibility layer on top of UEFI <stikonas>since every application carries the same code <stikonas>there isn't just one "program" that manages others <Hagfish_>that feels a bit alien, but presumably that's simpler than trying to make a user-space(?) kernel <stikonas>well, there is fiwix kernel that we could try to build... <stikonas>but yes, it is simpler in the sense that I don't need to deal with device drivers <Hagfish_>is that most of what a modern kernel does? by line count? <Hagfish_>oh, good point. i guess uefi gives you that for free too <stikonas>we are using a very limitted subset of UEFI API <Hagfish_>the smaller the subset needed, the easier it is for competing open source UEFI firmwares to support what we need :) <oriansj>stikonas: you can replace narrow_string[0] != '/' && narrow_string[0] != '\\' with !in_set("/\\", narrow_string[0]) and it'll run a little faster <stikonas>oriansj: ok, I'll do that in the next patch <stikonas>I need to fix chdir function to itself add cwd if it's relative <oriansj>well that'll get complicated when the local path points to a different device <stikonas>oriansj: hmm, but that adds a dependency on bootstrappable.c <oriansj>stikonas: which you don't need to use if you don't want <stikonas>oriansj: hmm, I was thinking of not letting to use any other decvice... <stikonas>we just use the same device that we booted from <stikonas>should be enough to build bootstrap kernel <stikonas>anyway, this can be changed later if somebody has a requirement <stikonas>but even now I was already stripping stuff before : in cwd <oriansj>(we can even make them responsible for that work if they have that requirement) <stikonas>which wouldn't work with open, as open is based relative to root dir <stikonas>basically everything is more complicated <stikonas>it might be a bit safer to use fixed size string functions... i.e. strncmp... <oriansj>stikonas: well yes, it would be a better choice long term as match is just as simple as possible true/false string compare needed in bootstrapping. <stikonas>this should be enough to add some simple relative paths to cd <stikonas>I haven't yet looked at .. but it probably partially works (except that paths keep growing as you do .., so they become dir1/../dir2/..) <stikonas>oriansj: oh, M2-Mesoplanet seems to crash here when run without any arguments... <oriansj>and even built with gcc, it has issues <oriansj>well when run without any arguments, it reads from /dev/stdin for input <stikonas>oh yes, though neither --version nor --help work either <stikonas>maybe it has no such arguments, let me see <stikonas>hmm, and they also work on M2-Planet/x86 <stikonas>but M2-Planet/AMD64 compiled version segfaults <stikonas>I was chaning quite a bit of stuff recently <stikonas>but let me check which other arches work/fail <stikonas>oriansj: ok, gdb shows crash in _malloc_free_list <stikonas>and it's just M2-Mesoplanet that crashes on --help... <oriansj>well just hitting --help shouldn't hit malloc/calloc and definitely not hit free <stikonas>oriansj: ok, it's setup_env() where crash happens <stikonas>hmm, and then the first calloc fails there... <stikonas>even though that's not he first calloc overall <oriansj>well turning free into a no-op would probably the quick and dirty fix <stikonas>no, just returning in free () does not help <fossy>from earlier, i'm not a particular fan of explicit dependencies in live-bootstrap (apart from simple documentation); they add a lot of complexity. the process is by-and-large simple and reasonably linear, and so by the time a proper dependency resolver can be used, it's of reasonably limited utility 