***Server sets mode: +cnt
<fossy>yeah, bauen1, src directory is gone <doras>I'm getting the following error when building bash-2.05b: <doras> make: *** No rule to make target `mkbuiltins'. Stop. <doras>But I can't figure out what's wrong. Everything looks correct as far as I can tell. <doras>This only happens when running the bootstrap using BuildStream, whose sandbox is internally based on bubblewrap as far as I know. <doras>Unfortunately there's no "env" to check if there is anything suspicious in the environment. <doras>And running "make V=1 mkbuiltins" doesn't show any additional information <doras>How do you debug such issues? <doras>I guess I could build static versions of those and copy them to the chroot <doras>I managed to attach to the bootstrap process with strace externally. I'll see if it helps me figure out what goes wrong. <doras>[pid 3241191] open("Makefile", O_RDONLY) = -1 EACCES (Permission denied) <doras>I think these "cp" commands somehow result in the file permission changing from "-rw-r--r--" to "--w-r--r--": <doras>Since the owner can't read the file (only write), and the owner is the one running "make", we get EACCES. <doras>I'm guessing umask may be involved here. <doras>kaem is missing the umask built-in command, so it's hard to check what it is... <stikonas>hmm, at that stage cp should be coreutils cp, not cp from stage0-posix <doras>Then this is what changed. Up until then there wasn't an issue with "cp" and permissions. <stikonas>well, stage0-posix cp does not support copying permissions <stikonas>so at least for executables we set 755 after that <doras>A difference in the BuildStream bootstrap compared to my bwrap mode bootstrap is that the user isn't a fake (namespace-mapped) or real root. It's the normal user. <doras>Maybe this makes "cp" behave differently... <stikonas>but why would it have some strange umask, hmm... <stikonas>so the stage0-posix cp might not respect umask at all <stikonas>oh, actually M2libc is probably unrelated <stikonas>let me see if meslibc has support for umask <doras>Does this mean that it always preserves permissions? <doras>[pid 3241120] read(8, "\n\0/dev/tty0\0\0\0\0\0umask stub\n\0\0\0\0\0"..., 4096) = 516 <doras>[pid 3241120] write(9, "\n\0/dev/tty0\0\0\0\0\0umask stub\n\0\0\0\0\0"..., 516) = 516 <stikonas>so it just prints it and doesn't do anything <doras>Is meslibc the libc library being statically linked with when "cp" is built? <stikonas>yes, at that time we don't have anything else yet <doras>Is "umask" something that the kernel manages? Or the standard library? <stikonas>it must be some information that process carries <doras>I'll see if I can reproduce the permissions change by calling cp myself. <stikonas>it might be something that buildstream sets <stikonas>if it for some reason sets umask, then it would be passed through all child processes <stikonas>and everything in live-bootstrap is a child process of kaem-optional-seed <doras>Strange. I see "-rwSr--r--" on the original files inside the chroot. <doras>What does it even mean that they have setuid? <doras>Actually, all files seems to have it. Hmmm... <doras>Also, I don't see why BuildStream would set such a strange umask. <doras>Removing read permissions for the owner? It's too strange to make sense. <doras>It seems that I can reproduce it in a non-BuildStream chroot. <doras>env -i PATH=/usr/bin sudo chroot /path/to/bootstrap/root cp /after/bash-2.05b/mk/main.mk /after/bash-2.05b/build/bash-2.05b/Makefile.test <doras>I end up with the following permissions: <doras>--w-r--r-- 1 root root 2572 Jan 0 00:00 Makefile.test <doras>And since I'm root, the read permission doesn't matter for some reason. "cat" on the file works. This must be why it doesn't reproduce when root is used for the build. <doras>Alright, so there issue is there anyway, but it's exposed only when non-root users are used for the bootstrap. <doras>I still suspect this stub. Maybe it results in "cp" getting the wrong impression about the umask. <doras>I'll try with "cp -p" to see what happens. <doras>It seems that "cp -p" works around the issue. <doras>It keeps the permissions of the original file: <doras>-rwSr--r-- 1 root root 2572 Jan 0 00:00 Makefile.test <doras>Haven't checked through BuildStream since it requires running the entire bootstrap again and I'm still exploring the sysroot, but I'm pretty sure it would work. <stikonas>so should be fine to just add it to live-bootstrap in general <doras>But it feels like a deeper issue, probably not specific to "cp". <stikonas>well, maybe something is wrong with umask <doras>And would we do if "cp" is used as part of the build of the following packages? <stikonas>but you can't easily check umask right now <doras>If our umask is stub, maybe it confuses it. <stikonas>anyway, it should be fine to use cp -p unconditionally <stikonas>since generally we weant to preserve permissions when copying <doras>stikonas: I was saying that our coreutils "cp" may be broken for non-root users, regardless of the actual umask. <doras>At least as long as we use meslibc. <doras>Or rather that it's built against it. <doras>So if we build the next step in the bootstrap, which internally uses "cp" too, or even the build of bash itself uses "cp" internally, we may end up with similar failures in those places too. <stikonas>and coreutils is not rebuilt until much later <stikonas>although it might be possible to move it to step 23 (after musl) <doras>I created a file with rw permissions for owner, group and other ("-rwSrw-rw-"). Then copied it using "cp" without "-p". I ended up with "--w-r--r--" <stikonas>so maybe you can test it and then do PR to fix it where it's necessary <doras>This is how "cp" creates the file: <doras>[pid 3247028] open("/after/bash-2.05b/build/bash-2.05b/Makefile-new.test", O_WRONLY|O_CREAT, 0100266) = 4 <doras>Notice the O_WRONLY, which means owner should have write-only permissions. <doras>And one line after the command above it uses stat() so we can see the actual permissions being 0244 (AKA "--w-r--r--"): <doras>[pid 3247028] fstat(4, {st_mode=S_IFREG|0244, st_size=0, ...}) = 0 <doras>Then it just keeps these permissions as-is and exits with success (0). <doras>You can see the entire execution as seen my strace here: <doras>stikonas: I'd rather fix "cp". <stikonas>doras: it might involve implementing the stub in mes libc <stikonas>mes libc is really minimal and not everything works when you build complicated executables with i <doras>Is the final sysc "cp" built with a different libc implementation? <doras>I'll check if I see the same behavior there too. <stikonas>doras: even step 30 cp is built with different libc <stikonas>but yes, we observed other peculiar behaviour with meslibc <stikonas>but since we don't use it for long, we didn't focus too much on it <stikonas>well, there is flex before musl but it's for some technical reasons <stikonas>(flex 2.5.11 links against heirloom tools, so have to be built against the same libc) <doras>This this looks rather different: <doras>open("/usr/share/info/make.info-1.copy", O_WRONLY|O_CREAT|O_EXCL|O_LARGEFILE, 0644) = 4 <doras>The last parameter is "0644", which looks like a valid permission mask, rather than "0100266" in the previous case. <doras>Aha, reproduced it with sysc's "cp" too. <doras>Hmm... well, not exactly. Never mind. <stikonas>hmm, so 0100266 must be coming from mes libc <doras>I previously looked at a newer version. <doras>I don't see any issue coreutils or in meslibc that could explain this. <doras>I'll try to add a bunch of prints in "cp" to see if it helps figure out what's going on here. <doras>"mode" is ~0 in that function, which is essentially -1. <doras>Sorry, I mean, "option->umask_kill" is. <doras>Yet this logic works fine on a different compiler. <doras>So either the compiler logic is broken in some form, or the defines are wrong. <doras>I added more prints to I can break down that statement into smaller parts and figure out which one is handled incorrectly. <stikonas>hmm, compiler is at that point tcc 0.9.26 <stikonas>although if necessary, we can build tcc 0.9.27 <doras>Or at least it is already built at that point. <doras>I highly suspect the compiler. I'll know in a minute. <doras>Surprisingly, it's not the compiler <doras>S_ISUID is 0400 instead of 04000 <doras>Now where is that one coming from? <doras>I basically need the package that creates our sys/stat.h <stikonas>no, we don't use linux headers at that point <stikonas>although, we'll also have to add that patch to mes-m2 <doras>Maybe this is also the reason all of our files in bootstrap have the setuid bit flipped. <doras>Or at least they appear to have it, I doubt they actually do. <doras>I'll start with a patch for live-bootstrap. Or is this before we build "patch"? <stikonas>although at that stage we have not yet rebuilt meslibc with patches <doras>"patch" comes after "mes" as far as I can tell. <doras>Unless stage0 creates "patch" too. <doras>"mes" is basically the first thing we build in after.kaem as far as I can tell. <doras>It's "mes" -> "tcc" -> "gzip" -> "tar" -> "sed" -> "patch" <stikonas>doras: yes, but there is another build of mes later <doras>"coreutils" is impacted by the first "mes". <doras>Which is why "cp" is broken. <stikonas>but ideally we also send patch to upstream mes <doras>I'll create a fork and test the fix. <stikonas>doras: it won't work if you just patch mes-m2 <stikonas>so patching mes-m2 will have absolutely no effect <stikonas>meslibc itself uses its own headers from build tree <stikonas>doras: some of the bugs from mes libc propatage a bit, so we had to build tcc and musl twice... <stikonas>but it's probably not this bug you found <bauen1>nice, i think i have a very simple and quite short uart read/write driver, thinks are actually easy if you're calculating the divisors with the correct clock frequency (and 24mhz != 20mhz) <doras>stikonas: it changes a few checksums. <stikonas>I think fossy is working on something better <doras>The build fails when it finds a different checksum. <doras>Do you run it all over again each time? <stikonas[m]>Well, you can try to inject busybox sh and manually run next steps <stikonas[m]>But it might just be simpler to let it run a few times <doras>I'll try to make the checksum failures a warning instead, and maybe write them to a file. <doras>I'm actually not building sysb at all <doras>So I'm not sure if it would also differ. <oriansj>nice find on S_ISUID; turns out M2libc is wrong for it as well <oriansj>also if umask was at the kernel level, the mescc-tools-extra cp would be impacted as well <oriansj>and guessing by the kernel ABI, it would probably be the sys_umask command <doras>Would you like me to submit a patch somewhere?