IRC channel logs

2023-01-13.log

back to list of logs

<oriansj>doras: actually I believe you probably could hit that stretch goal in 5 years with some help
<stikonas>well, at least if you don't care about pregenerated files then bootstrapping buildstream dependencies is probably fairly simple
<stikonas>python is already wip
<stikonas>podman just needs go but that comes almost for free with either GCC or bootstrap via G o 1.4
<oriansj>well if one ignores the pregenerated bits in original Go release, the Go bootstrap should be easy with even gcc 4.7
<stikonas[m]>fossy: I've removed tar installation hack and fixed tar package, now it contains tar...
<stikonas[m]>Though I noticed some other issue, somehow my after.she hook was not launched (at least in presseed mode)
<doras>stikonas, fossy, do you expect us to ever need to download different sources depending on the bootstrap mode or other parameters passed to ./rootfs?
<doras>I'm trying to decide if my --source-manifest command argument should always come with the rest of the existing arguments, or sort of be completely separate from them.
<stikonas>well, there is already some difference in sysa based on mode
<stikonas>qemu mode builds kexec and linux kernel
<stikonas>might need similar things to other kernel transitions
<fossy>stikonas: i've been playing with MACHDEP but for later pythons there seems to be a bit of weird behaviour, im trying -DPLATFORM in CFLAGS (PLATFORM seems to be where most stuff is hardcoded)
<stikonas>oh I see
<fossy>im going to be a few days before i do any real work on python though, had wisdom teeth extraction yesterday so still not 100%
<fossy>doras: differences in what packages are built, eg kexec can be expected, but not nessecary to avoid downloading tarballs, imo
<doras>Yeah, it only complicates the concept.
<stikonas>yes, definitely take some rest and recover first
<doras>It's actually easier to require all the existing command arguments code-wise, but I prefer to do the refactoring to avoid this and keep the code simple.
<fossy>i don't have any problems with it being in a separate script either, if that makes more sense
<doras>fossy: I'm also contemplating this option too, but the refactoring would be needed regardless to avoid logic duplication around SysX/SysGeneral and the potential new script.
<fossy>yeah, that is quite possibly true
<doras>Basically, I want the function that generates the manifest to be a static class function. We don't need an class instance for it, and I'm not happy with the idea of creating a dummy instance.
<doras>i.e., I want this to be enough to prepare a manifest: SysA.get_source_manifest()
<doras>fossy: is this an acceptable source manifest format?
<doras> https://gitlab.gnome.org/-/snippets/5162/raw/main/sysa-source-manifest
<doras>Would you like to have the values ordered differently?
<doras>This is currently "<destination> <url> <sha256> [filename]"
<doras>Very similar to the source file format in packages, with the addition of <destination> of where files should be placed in the source tree.
<sam_>in gentoo, we do: filename size hash1 [hash2..]
<doras>Actually, I called it <directory> in the source code.
<doras>sam_: size is interesting. I guess an optimization. In our case the manifest must describe from where a particular source should be downloaded and where it should be placed in the source tree.
<sam_>ACTION nods
<doras>fossy: an alternative with <directory> moved before [filename]: https://gitlab.gnome.org/-/snippets/5163/raw/main/sysa-source-manifest-alternative
<doras>A bit less human-readable, but potentially more logical since the filename is next to the destination directory.
<doras>I'm fine with either approach personally.
<doras>... but would appreciate an opinion so I know what to do :)
<stikonas>doras: actually it might be more human readable if you put url last
<stikonas>though logically the order is a bit "wrong" in that case
<stikonas>but length of URI is the field whose length changes most
<doras>stikonas: also true
<doras>fossy, stikonas: another alternative with "<sha256> <directory> <url> [filename]": https://gitlab.gnome.org/-/snippets/5164/raw/main/sysa-source-manifest-alternative2
<stikonas>yeah, I would say this is more human readable...
<doras>I mostly sticked with what we had in the package source files in the first option. I don't mind doing something completely different for the manifest.
<doras>I'll go with the last option unless someone stops me :)
<oriansj>sam_: the multiple hash bit probably allows one to choose hashes that make a preimage attack much harder computationally.
<oriansj>doras: no one will ever stop you from doing awesome work for improving the bootstrapping process for others.
<stikonas>oriansj: and it also allows to migrate hashes in Gentoo
<stikonas>some years ago different set of hashes was used
<stikonas>possibly something like MD5, SHA1 but I don't remember exactly now
<oriansj>stikonas: that is a good operational point
<rickmasters>I have a quick update on kernel bootstrapping.
<rickmasters>tcc 0.9.27 inside live-bootstrap can build Fiwix and the resulting kernel works.
<stikonas>nice!
<rickmasters>This was done with Fiwix as the driving kernel - so it's self hosting.
<rickmasters>But using builder-hex0 does not work.
<stikonas>any idea why?
<rickmasters>I need to implement file descriptors better to make it work.
<rickmasters>This was something I was probably going to do anyway since it would eliminate the need to patch mes.
<rickmasters>Basically builder-hex0 only supports one file descriptor per file.
<rickmasters>Building Fiwix ends up opening the same header file twice which requires two separate file descriptors using different offsets into the file.
<rickmasters>It shouldn't be too hard to implement but it's going to take a day.
<stikonas>well, hex0 code is hard to edit...
<stikonas>all those jumps...
<rickmasters>I'm leaving on a trip so I'll probably start on it Tuesday.
<doras>fossy, stikonas: I opened a MR for the source manifest: https://github.com/fosslinux/live-bootstrap/pull/225
<rickmasters>stikonas: I've relaxed my principles on hex0. Previously I only hand edited the hex0...
<rickmasters>Now I edit in hex2 and run a python script to convert to hex0 and then I review the diffs.
<stikonas>well, hex2 is better for development...
<stikonas>perhaps we should keep it in hex2 for now and if somebody wants, we can do hand conversion once everything is working
<stikonas>doras: I'll take a look in the evening
<rickmasters>I'm currently checking in the hex2 and hex0 edits simultaneously. For the last 17 commits or so since end of August
<stikonas>oh ok, then it should be fine
<stikonas>unfortunately, switching kernels is somewhat harder than compilers
<stikonas>otherwise we would be able to use builder-hex0 to build e.g. builder-hex2
<oriansj>rickmasters: how about we make life even easier for you and we do a half step?
<rickmasters>oriansj: what did you have in mind?
<rickmasters>i have to go but will check back later
<oriansj>rickmasters: Wouldn't working in M1 be much easier?
<acrow>nckx: Do you know if it is possible to run a xapian search on the log.guix files? Well, at least without needing to individually download each day's log.
<nckx>acrow: That's exactly what the search box is supposed to do! However… it seems like the DB hasn't been updated since… oh dear.
<nckx>I'll SSH in later & look for obvious errors.
<doras>stikonas: I'm contemplating always having the expected filename in the source manifest. The Python logic that comes up with the file name when it's not listed in the package source file is to call `os.path.basename(url)`. I think it may be best to keep this an implementation detail of live-bootstrap as well and not expect users to have an equivalent implementation at their disposal.
<stikonas>doras: up to you but keep in mind that manifest is tied to a particular commit of live-bootstrap (or a short set)
<stikonas>so you wouldn't gain that much by hiding this implementation detail
<doras>stikonas: my intention is that the manifest would be simple and explicit enough so its format can remain the same despite any internal changes in live-bootstrap around how package source files are managed.
<stikonas>true, manifest itself might change, but if format stays the same, then other tools processing it don't need changes
<stikonas>sure, if you want to add a name, add it
<stikonas>so "hash dir url name" ?
<doras>I think so.
<doras>I uploaded a new revision with this change.
<stikonas>doras: can you fix pylint?
<stikonas[m]>In general looks good
<doras>stikonas: thanks for the review! It seems pylint caught a small leftover, phew.
<stikonas>ok, merged
<doras>Thanks again :)
<stikonas>I guess you are integrating it here https://gitlab.com/freedesktop-sdk/freedesktop-sdk-binary-seed
<stikonas>doras: and then freedesktop-sdk is used to build freedesktop-gnome and freedesktop-kde?
<doras>stikonas: yes. And almost all Flatpak apps. Also GNOME OS.