IRC channel logs
2023-01-12.log
back to list of logs
<oriansj>doras: how about a write a program in M2-Planet that generates yaml, toml, json or any other format you want but it just reads the existing url checksum formatted file <oriansj>so that live-bootstrap wouldn't actually need any changes and that information is available to be generated early in the bootstrap <oriansj>I also could write a parser for those formats in M2-Planet without too much complexity <oriansj>as long as I am not expected to be 100% feature complete <oriansj>basic yaml doesn't look too hairy to parse <muurkha>sure, subsets of yaml such as json can be quite easy to parse <oriansj>why would we expect ourselves to support an arbitrary and huge standard, when we only need a tiny subset to get real work done? <muurkha>that was why I was shocked that you mentioned it as a candidate ;) <stikonas>well, space separate columns or comma separated values should be sufficient for us <stikonas>fossy: so with MACHDEP=linux first python here has the following hash: f26f74c3c8cc424acf401b5b3d198954d7b30d3b047f6296b827 Python-2.0.1_0.tar.bz2 <oriansj>muurkha: fair enough. But by now, it shouldn't come as a surprise when I keep only implementing the actually useful subset of languages. <stikonas>most of these slightly more complicated formats are useful when you want to extend your markup language but keep some compatibility with older versions, so you have both keys and values <stikonas>we are probably fine by not keeping any compatibility, so I would think just values should be sufficient <oriansj>although one could always argue that all markup languages are just poorly done versions of SQL databases <stikonas>anyway, it looks like doras might be doing the work, so we'll follow our usual principle that you decide if you do the work <doras>oriansj: it could be a fun exercise, but it won't provide much value compared to writing the same logic in Python. This is mostly needed so source tarball tracking and fetching can be scripted to be done pre-bootstrap without relying on rootfs.py itself. <doras>Click "Browse" under "Job artifacts" to see sysc packaged up post-bootstrap in an archive. <doras>We use sysc to build cross-compilation toolchains for x86_64 and aarch64, and then we use them to build native toolchains for these architectures. <doras>We then continue build a few more packages that the next steps of our bootstrap require, currently using qemu-user where relevant (to run native aarch64 on an x86_64 build server). <doras>Finally, we create a Docker image containing the native toolchains and additional packages, to be used as the seed system for bootstrapping freedesktop-sdk. <doras>Actually bootstrapping freedesktop-sdk successfully from that seed is the next step, but this system is already very similar to the system we used up until now (mostly with an older gcc and glibc), so it hopefully shouldn't be too difficult. <doras>If that all of that works, then we would have a complete bootstrap chain from hex0 + kaem + kernel up to GNOME OS, all GNOME apps and actually almost all Flatpaks (at least those building from source). Minus Rust which is a pain at the moment and still requires its own binary seed. <doras>Note that we don't remove pre-generated files outside live-bootstrap, so take as as you wish :) <doras>Well, technically you could consider some tools we use in CI to build the additional packages and Docker image itself (such as Podman and BuildStream) to be binary seeds in the process. If all Podman and BuildStream binary packages are deleted from history, the bootstrap chain is broken. <doras>So next we would need live-bootstrap to bootstrap all of BuildStream's dependencies (Python, buildbox-casd and buildbox-fuse), Podman, git and some other tools we use in CI so we could remove the copies we have in the CI image, leaving the CI image basically with nothing but bubblewrap and Python :) <doras>One could then claim that live-boostrap itself could be bootstrapped on bare metal in case all Podman, Python, bubblewrap binary packages are deleted from history, providing us these tools, so our bootstrap chain remains intact. <doras>But the last part is a stretch goal that we likely won't hit ;) <stikonas>doras: yes, I don't expect that downstream consumers would bother with pregenerated files <stikonas>just tried running python from live-bootstrap on my normal system (with glibc) <stikonas>and it works if I specify correct linker <stikonas>Could not find platform dependent libraries <exec_prefix> <stikonas>Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]