IRC channel logs

2021-05-03.log

back to list of logs

<OriansJ>kaem-optional-seed is even required if one is willing to do a reboot; the first boot would be the hex0-seed building kaem-optional-seed from its hex0 sources and then the second boot would then run kaem-optional-seed as the init and the rest can run just fine from there.
<OriansJ>melg8: we could if they exist and we will learn a great deal when we examine them in depth. However we will not know for absolute certain until we remove the kernel and hardware that we didn't make ourselves from the root of trust.
***lukedashjr is now known as luke-jr
***ChanServ sets mode: +o rekado
<stikonas>well, just after tcc we build untar.c which can decompress tar archives but is not command line compatible, then we build gzip and gnu tar (old version)
<stikonas>so we indeed have tar very early
<stikonas>and untar.c can be made to build with M2-Planet
<gforce_de1977>stikonas: i did some resarch with SLAB/SLUB/SLOB and better gathering data, so this seems to be the final plot: http://intercity-vpn.de/bootstrap/bootstrap-usedmem-experimentQEMU-1375M.png
<gforce_de1977>rawdata + log: http://intercity-vpn.de/bootstrap/bootstrap-usedmem-experimentQEMU-1375M.txt
<gforce_de1977>which leads to the question: are we cleaning up after building packageX ?
<gforce_de1977>and OriansJ: early bootstrap needs "only" 550mb, not 1gig
<stikonas>gforce_de1977: no, we definitely are no cleaning after building packages
<stikonas>that's very clearly visible in final fs
<gforce_de1977>stikonas: ok, I will do an experiment how mem consumption behaves when auto-cleaning-up, but RAM is not an real issue for now, but at least nice-to-know, what is possible. it would be cool to go down a little bit...
<bauen1>stikonas[m]: i would need a tar that can create archives, but i guess that works out
<melg8>Hi, can somebody check if touch working from inside of build image. I can't change time of last access for existing file by doing something like 'touch -t 200901310846.26 test_file'. So it's like - me doing something wrong, or is it really not working, thanks.
<gforce_de1977>melg8: does a simple :>>test_file works?
<gforce_de1977>fossy: stikonas: a added a simple "rm -fR /after/$pkg" to helpers.sh and the bootstrap now need ~600mb RAM: http://intercity-vpn.de/bootstrap/bootstrap-usedmem-experimentQEMU-1375M.png
<gforce_de1977>i can add an "--autoclean" to roofs.py, if you like
<melg8>@gforce_de1977 can you show example of full command? and - even if it works, i need to set some fixed date - not just current date - i'm trying to fix reproducibility
<gforce_de1977>melg8: ok: this *is* the full command
<gforce_de1977>i will try to boot into a bootstrap-shell, give me 1 hour 8-)))
<melg8>can you link to your version of source? i could rebuild it too - maybe your cleanups fix some of reproduce issues
<melg8>ls is workings - meaning - showing output
<gforce_de1977>melg8: i'am just using master and have also probs, e.g. http://intercity-vpn.de/bootstrap/diff-16-1619253444-and-13-1619350041.html
<stikonas[m]>melg8: touch might be patched out not to work
<melg8>what i've been thinking of - is like - at the end of build - just find all files and change their dates with touch - so there are no modification date conflicts between builds. And why it patched not to work?
<stikonas> https://github.com/fosslinux/live-bootstrap/blob/master/sysa/coreutils-5.0/patches/touch-getdate.patch
<stikonas>to remove dependency on bison
<stikonas>coreutils 5 is built before bison is available
<stikonas>and coreutils 6 (from where we build only a few commands) does not work with bison 3.4.0
<melg8>but i try to use -t not -d flag
<stikonas>oh, rest should work
<stikonas>just date parsing
<melg8>can you recheck it? because i use same command on my host and inside of chroot - and get different result outputs of ls
<melg8>"find /after -exec touch -t 200901310846.26 {} +" - that was full command
<gforce_de1977>melg8: i think the idea from bauen1: is better: patch libc the output a monotonic raising artificial timestamp on each call starting at e.g. unixtime 1
<melg8>that would be great, but... monotonic rising timestamps would affect scenarious if there more than 1 thread involved
<gforce_de1977>melg8: more than 1 thread should be avoided IMHO. our upcoming mini OS will not support that anyway 8-))))
<melg8>btw, did i missed where bauen1 suggested it? or it was not today?
<gforce_de1977>bauen1 suggested that some days ago
<melg8>need to find it)
<gforce_de1977>Q: does somebody know how to freeze a QEMU-session (e.g. like suspend to RAM) and restore that later?
<gforce_de1977>that way we can provide a simple "ready build, you are in the shell now..." image
<stikonas>not sure if qemu provides that...
<stikonas>disks can be snapshotted, but we don't have any disk n live-bootstrap
<gforce_de1977>stikonas: disks are not interesting, i want RAM 8-)))
<gforce_de1977>i spotted something really strange: when running bootstrap into an interactive shell, there are a lot of tasks hanging aroung: /bin/busybox pidof make sed configure | wc -w -> 409
<gforce_de1977>maybe this has something todo with the way is changed the build. will investigate.
<gforce_de1977>melg8: your touch command does indeed also not working here, will tinker for a workaround
<gforce_de1977>melg8: touch -d also does not work: touch -d "1999-12-31 00:00:00" /after/run2.sh
<melg8>thanks for check!
<gforce_de1977>but a /bin/busybox touch -d "1999-12-31 00:00:00" /after/run2.sh succeeds, so touch is really hmmm b0rken
<melg8>i dont have busybox :(
<gforce_de1977>touch -r also does not work
<gforce_de1977>nobody has busybox, i have just injected it, so it is not part of bootstrap
<gforce_de1977>setting date also does not work
<melg8>about the solution with patched libc - but when there just starting mes/tcc/untar - and some of it saves files to disk - they have correct dates, where do these dates come from ?
<gforce_de1977>melg8: when creating files, e.g. :>myfile && ls -l myfile -> this works
<gforce_de1977>also: echo "fooo" >>myfile works, the date.now is also applied
<gforce_de1977>just touch does not work 8-))))
<gforce_de1977>have to leave, cu later
<melg8>gl
<bauen1>gforce_de1977: don't patch libc, patch the kernel but yeah
<melg8>thats... will be harder for me, than to patch libc xD
<melg8>But, than - chroot option of project - will not have same dates? is this times are from kernel?
<stikonas>well, touch -d is definitely not expected to work
<stikonas>as that is exactly what is patched out
<bauen1>melg8: reason being that you'd want the "time counter" to persist across programs, even once that don't use libc
<melg8>when some new files are first created as part of build process - from what source filesystem takes dates for created/modified attributes of this files?
<stikonas>can't we do what guix does
<stikonas>it uses normal timestamps, but later removes them
<stikonas>or someting like that
<stikonas>so guix store has timestamp 0
<melg8>thats why i've tried touching stuff) because on my host system that fixed like 40 mb diffs to few kb diffs
<bauen1>yeah that would go a long way, but could still leave some timing issues with make
<gef>> melg8: i think the idea from bauen1: is better: patch libc the output a monotonic raising artificial > timestamp on each call starting at e.g. unixtime 1
<gef>> ...
<gef>> btw, did i missed where bauen1 suggested it? or it was not today?
<gef>> bauen1 suggested that some days ago
<gef>well, was it bauen1 or, was it me? :-) (we might have proposed that independently oh each other though). There are merits to both keeping it 0 or increasing it monotonically, either is good according to what you want to achieve.