IRC channel logs

<matrix_bridge><cosinusoidally> fair enough, thanks. Makes sense given the intended use case. I ran into the calling convention differences when calling cc_x86 generated code from tcc generated code (which you probably didn't predict someone would try to do).

<oriansj>cosinusoidally: well no, honestly if one has a better C compiler; I generally assume one would use it whenever possible. It also reduces the chances of running into bugs caused by defects in cc_* or M2-Planet. (Which there are a good many if one is careless in their deployment)

<Guest991>what is error with this? what is wrong with this?

<Guest991>what is wrong with this? rm: cannot lstat `*/Makefile.in': No such file or directory

<Guest991>rm: cannot lstat `*/*/Makefile.in': No such file or directory

<Guest991>Subprocess error 1

<ekaitz>guys

<ekaitz> https://www.openwall.com/lists/oss-security/2024/03/29/4

<ekaitz>my friend picnoir just sent me this

<ekaitz>a supply-chain attack on the wild

<ekaitz>this is a pretty hardcore thing, it's a problem in xz

<ekaitz>oriansj: can we add this to bootstrappable.org ?

<ekaitz>i think this is huge milestone

<ekaitz>janneke: we should use this for future funding

<stikonas>yeah, I jus tsaw that too

<stikonas>ekaitz: but this is just normal backdoor

<stikonas>not one of those binary only...

<ekaitz>but it's only affecting the binary release

<ekaitz>they are using this as a binary in several distros, in the bootstrapping

<ekaitz>in Nix, for example

<stikonas>oh, I should read it in more details

<stikonas>though I did see that gentoo masked it

<ekaitz>we shouldn't have any problem with this because the source is ok

<stikonas>but these days gentoo also offers binary builds for those who want

<ekaitz>they changed the binary release only

<ekaitz>stikonas: but this is like they replaced the "releases" page in github where you can upload your own builds

<ekaitz>so the tar.gzs with source code, as they come also from the sources in github, are not affected either

<stikonas>my reading is that it is present

<stikonas>in normal releases

<stikonas>yeah, that releases page

<stikonas>is like any other ftp http download page with separate release tarballs

<ekaitz>yes

<ekaitz>if this whole thing was bootstrapped, no issue

<stikonas>yeah, but distros dont' consume those

<stikonas>normally they want release tarballs and not git snapshots

<ekaitz>some have to do, because this affected debian

<ekaitz>but the only ones affected are those that contain the compiled files

<civodul>woow, it’s a very elaborate scenario

<oriansj>once again another reason to build from source locally and not trust any binary providers.

<stikonas>I'm still reading though

<stikonas>but aren't upstream source release tarballs also backdoored?

<stikonas>trying to find where this says otherwise and failing

<stikonas>if anything it seems that just building yourself won't help

<janneke>ekaitz: wow

<ekaitz>stikonas: it says 'except for the "source code" links'

<stikonas>yeah, but like I said

<stikonas>most people use source release tarballs

<stikonas>those are affected

<stikonas>looks like version 5.6.0 already had a few complaints from Gentoo people, sam_ in particular and that guy was trying to adjust backdoor in 5.6.1 to avoid valgrind issues

<ekaitz>aaah i see

<ekaitz>we should go for git then instead

<ekaitz>in guix we use git in many places

<stikonas>yeah, just git is unaffected

<ekaitz>stikonas: but also using the source-code part that is generated by github would be great

<stikonas>well, that is just git archive I guess

<ekaitz>yes

<ekaitz>it's a git archive

<ekaitz>oriansj: the point of this one is we can't even trust the source-release providers

<stikonas>that's not new though :)

<stikonas>it's just that this time backdoor is in quite core project

<ekaitz>yeah

<ekaitz>it's a huge one

<ekaitz>xz, mate!

<ekaitz>in any case, having this written in bootstrappable.org would be interesting

<oriansj>ekaitz: we never could trust the source-release providers. Ultimately they could go crazy at any time (or bribed or blackmailed); hence the need for trusted source code, also includes trusted build scripts.

<ekaitz>yes

<ekaitz>build scripts, and configure scripts are also code

<ekaitz>if the `autoreconf -vif` generates a 20kLoC configure script, I also consider that almost a binary

<ekaitz>that's impossible to read and review

<oriansj>and have to be reviewed, audited and ideally so simple that it can't do anything bad.

<stikonas>the thing is that even autoreconf -vif wouldn't help here

<stikonas>the backdoor was an extra injected m4 file

<stikonas>but yes, those huge scripts are why live-bootstrap doesn't trust them until autootools is bootstrapped

<stikonas>at least then you only need to review m4 macros

<stikonas>(though for this backdoor, it was in m4)

<oriansj>a simple list of ${cc} -c ${source} -o ${source/.ext/.o} should be enough for compiled languages (with a little bit of conditional for different host platforms)

<oriansj>but yeah, autotools and configure scripts are ripe for many potential backdoors

<ekaitz>yes

<oriansj>now makefiles support that out of the box: https://www.gnu.org/software/make/manual/html_node/Conditional-Syntax.html

<oriansj>so configure scripts and autotools in theory could be abandoned but I doubt that is likely to occur

<stikonas>well, I guess these days people tend to go for cmake or meson instead of autotools

<ekaitz>which btw idk if it's good or bad

<oriansj>well, it depends. On one hand a clean cmake or clean meson build is fine. They too can be abused to inject backdoors

<oriansj>(Even makefiles can be abused in that fashion)

<oriansj>so, the desire for clean and simple builds is perhaps the key

<stikonas>well, anything can be abused with backdoors

<stikonas>I think somebody even wrote backdoored hex0

<stikonas>or something like that

<oriansj>yes they did

<stikonas>though backdoor couldn't propagate too far

<oriansj> https://gitlab.com/bauen1/stage0-backdoor.git

<stikonas>oh yes, bauen1...

<ekaitz>haha that's great

<ekaitz>the goal is to reduce the surface

<sam_>[17:31:32] <stikonas> looks like version 5.6.0 already had a few complaints from Gentoo people, sam_ in particular and that guy was trying to adjust backdoor in 5.6.1 to avoid valgrind issues

<sam_>this is actually nuanced, btw

<sam_>because it turns out, this led to a *real* gcc bug being found, in my cse

<sam_>NOT just working around the backdoor or anything

<sam_>but I DO regret not looking more into why there was an issue on Fedora but not Gentoo

<sam_>(because we never hit the valgrind problems, which it sounds like is because it was only setup to fire if rpm was found in the build env)

<sam_> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=114115

<stikonas>I wonder if this commit is trolling; https://github.com/tukaani-project/xz/commit/af071ef7702debef4f1d324616a0137a5001c14c

<oriansj>hard to say. sam_ also another reason why these sorts of attacks will only get easier and easier to do as we go forward; the deltas are too large and unless something is behaving badly no one is looking too deeply.

<stikonas>we also have more sanity tools though

<stikonas>so attackers have to work-around more stuff

<sam_>absolutely

<sam_>I have some reflections on what we could do better but it is not the time for that yet

<sam_>need to do other stuff first (like more auditing)

<stikonas>yeah, this person seems to have contributed to other projects

<ekaitz>it's pretty wild what just happened

<oriansj>well I was thinking guix with more explicit configuration, isn't that far off from a gentoo install. It just would be a huge breaking change and take 2-3 years before it could take off

<oriansj>but I would love to read your ideas for improving the situation sam_

<stikonas>in any case the bug kind of comes from the blob

<stikonas>it's just not the program blob but test blob...

<luke-jr>re stikonas's ref'd commit... I wonder if the goal was to get any snooping to self-dox to him faster than if they had to investigate more details first

<bauen1>holy moly didn't actually expect such an elaborate backdoor by xz

<bauen1>oh yes, I remember I had fun trying to backdoor stage0 ; Not sure how far it actually propagates, iirc it really doesn't get far

<bauen1> https://github.com/tukaani-project/xz/commit/af071ef7702debef4f1d324616a0137a5001c14c

<bauen1>oh someone already posted thta, I'm not even sure what to make of that ...

<bauen1>oriansj: arguably it would be very interesting to take a whole bunch of projects, regenerate and verify the autoconf scripts from source (even if it's the source in the tarball itself) but iirc it's very hard to filter false positivies of "doesn't reproduce"

<bauen1> https://news.ycombinator.com/item?id=39868483 <- A perfect summary really :(

<ekaitz>bauen1: exactly

<ekaitz>autotools

<bauen1>Maybe this will get debian (and other linux distros) to insist on reproducible source tarballs and start to consider autohell scripts as straight up binaries

<bauen1>it just keeps on giving: https://news.ycombinator.com/item?id=39868158 " The best part is everyone disabling security tests that started failing "

<bauen1>oh and the amount of people running ldd to check if there's potentially something evil without realising, that ldd literally executes the binary to find out what libraries are used

<clemens3>best is, it is not clear how long back the guy had been the main maintainer in the background already

<clemens3>at lest from 2022, likely earlier

<bauen1>now this reminds me that coreutils or util-linux iirc doesn't have a git repository for debian, just tarballs

<bauen1>ah coreutils it is

<bauen1>at the time I was just annoyed, now I'm concerned

<clemens3>you shouldn't be concerned not just now

<clemens3>with hundred millions of lines of code in a normal distro, and being in this channel:)

<bauen1>clemens3: I mean, I literally wrote a backdoor for hex0 as a fun exercise, I was concerned back then too :(

<clemens3>yeah

<clemens3>how many thousand contributors

<clemens3>last time i checked systemd had 600'000 lines of code

<clemens3>i heard it is way way more now

<clemens3>a modern web browser will never ever be audited completely

<bauen1>tbh. I'm not too concerned about systemd compared to autotools generated scripts, it's just way too easy to inject some evil code in there and you most likely wouldn't need to discuise it either

<clemens3>i am concerned about the complete jungle

<clemens3>autotools, systemd, rustc cargo dynamic downloads

<clemens3>blah blah blah

<clemens3>linux kernel regressions every second update

<ekaitz>i'd say the most dangerous projects are those small projects that nobody really maintains

<clemens3>it's all together

<ekaitz>the maintainers are bored, they don't want to keep them and they give them away to anyone that comes and shows interest

<ekaitz>they are an easy target

<ekaitz>systemd, as long as there's a lot of people reading everything, that's better imho

<ekaitz>still very bad

<ekaitz>complexity is horrible

<clemens3>there no where lot's of people reading it

<clemens3>sometimes like this xz guy someone drills down

<clemens3>someone brievly looks over some contributions

<clemens3>but normally no one studies and audits all the stuff

<clemens3>is impossible

<ekaitz>yes

<clemens3>someone writes the code, and most of it, no second eyes ever over it

<ekaitz>depends on the project

<clemens3>how many times you read your own code over again

<ekaitz>more than I'd like to

<clemens3>just when you debug or continue writing more features

<ekaitz>haha

<clemens3>and then half the people really write horrible complex code just to toy around with something, learning and making it more complex

<clemens3>after all, that's why we are here, to play with stuff and learn something by doing

<clemens3>just that the alternative, closed commercial source, is so much worse:)

<clemens3>seems the guy contributed to some other fundamental projects too

<clemens3>zstd, llvm

<ekaitz>i'm not afraid of the commits

<ekaitz>that might be corrupt but we can fix

<ekaitz>but changing releases and stuff like that... that's too much

<ekaitz>it's way harder to follow

<stikonas>I wonder if this is just some random actor or state level actor

<ekaitz>stikonas: a very long investment of 3 years in xz just to make this...

<stikonas>exactly...

<xd1le>honestly I wouldn't be surprised if other common tools have been backdoored like this

<ekaitz>yeah...

<xd1le>is guix affected?

<xd1le>I hope this places more importance on reproducible builds

<ekaitz>not yet

<ekaitz>we use older xz

<xd1le>ok, thanks

<ekaitz>but reproducible or not, if we all use their source... and the source release is broken...

<ekaitz>we need to start using a git reference

<xd1le>yeah..

<clemens3>i would be surprised if there are no backdoors, i would say it is impossible that there are no (intentional) backdoors

<clemens3>not counting the zeroday bugs

IRC channel logs

2024-03-29.log