IRC channel logs

2014-09-25.log

back to list of logs

<civodul>Hello Guix!
<civodul>Bash issue not fully addressed: http://seclists.org/oss-sec/2014/q3/685
<civodul>other patch: http://seclists.org/oss-sec/2014/q3/690
<philed>You mentioned that bash is an input to most guix packages. Can it be swapped for other shells easily enough?
<civodul>that's not clear
<civodul>many package build systems implicitly rely on Bash, i think
<civodul>so there'd probably be a lot of breakage
<civodul>it wouldn't save us time
<philed>I wasn't so much thinking about time-saving, just whether it would be worth thinking about it long-term, looking for a smaller shell?
<civodul>we plan to use GNU packages primarily, obviously
<civodul>now, for user accounts, one can choose another shell
<philed>Ah sorry, I'm pretty new to this stuff, so things aren't that obvious to me. :) Is BASH the only shell in GNU?
<civodul>np! :-)
<civodul>there's another one called Rush
<civodul>but it's a different beast
<civodul> http://gnu.org/s/rush
<philed>Ah cheers.
<civodul>oh one should hook Guix into Rush, for the chroot support
<jxself> https://lkml.org/lkml/2014/9/21/138
<jxself>Oh, maybe 3.17 will be out this weekend.
<civodul>oh
*civodul wonders what to do with the 2nd CVE: http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2014-7169
<civodul>nobody replied to the patch Chet posted
<civodul>so i'm tempted to wait for confirmation that it does fix something
<jxself>Have other distros applied it? That could be a good indicator.
<civodul>NixOS has just applied it
<civodul>dunno about the others
<civodul>i'd like to have a more formal confirmation
<gryAway>i assume we already know about bash vuln. here?
<gryAway>nvm yes, i see it was mentioned above
<civodul>yes
<civodul>i just sent an update to the list
<davexunit>gah this bash thing is bad :(
<davexunit>and trisquel's maintainer is away...
<davexunit>STILL no patch.
<civodul>there's one, but we don't really know what to think of it
<davexunit>the first patch has made it to debian, ubuntu, etc., but not trisquel.
<civodul>i wanted to merge our branch this morning, but then i found about the second CVE
<civodul>so now i'm leaning towards waiting for the real fix
<davexunit>civodul: just realized that I pushed a commit with the wrong message. It should say "libsodium" but I wrote "libtoxcore" because I was working on an experimental package for it at the time.
<davexunit>I could ammend the commit easily, but I of course don't want to git push --force to master.
<davexunit>that was quite the 'thinko' on my part. sorry.
<civodul>davexunit: i noticed, but it's too late now
<civodul>and no big deal anyway
<davexunit>damn.
<davexunit>I wasn't thinking.
<davexunit>libtoxcore isn't even in the repo because upstream hasn't released it. I've mislead whoever reads the log. :(
<civodul>yeah, but don't worry too much either ;-)
<davexunit>I'll try to be more careful in the future.
<davexunit>I was messing with the new GUIX_PACKAGE_PATH variable. I like it a lot.
<civodul>yeah i think it was really needed
<civodul>i have been keeping local changes in a git stash
<civodul>but that's really not convenient ;-)
<davexunit>I was writing some personal packages for things that are still too experimental to go upstream. libtoxcore depends on libsodium, and then I saw on HN that libsodium just released 1.0.0... and the rest is git history.
<civodul>he
<civodul>and i really like that Emacs UI
<davexunit>I haven't used it much yet.
<civodul>i very much depend on it already :-)
<davexunit>haha
<civodul>the cool thing with GUIX_PACKAGE_PATH is that, for instance, the guile-toxcore repo could come with its own package module
<davexunit>yeah, that would be great for bleeding-edge development stuff.
<davexunit>and then combine that with 'guix environment' when it exists
<davexunit>and you have a pretty awesome development environment
<civodul>heh, yes
<davexunit>now I can host my own custom package recipes on gitorious.
<mark_weaver>civodul: I think we need a way to deploy security updates to core packages without having to rebuild everything, at least in the short term. I have a couple of ideas that I think are workable.
<mark_weaver>the short-term idea is that we provide a way to specify that package C (a core package to which we will later need a security update) to be specified as an original package object to build, and a later package object that is later "swapped in" without recompiling the reverse-dependencies, as follows:
<mark_weaver>so when another package P depends on package C, the build process is as follows: first we create a derivation that builds P1 against C1 (the original version of C), and then another derivation is used to build P from P1 by simply replacing all occurrences of the C1 hash with the C2 hash.
<mark_weaver>so, this is still purely-functional. if you build P against C (where C is defined a C1 later patched to C2), then it will *always* build P first against C1 and later patch to work with C2.
<mark_weaver>the updated P would have an updated hash to reflect the fact that it has been updated, but it would be much much faster.
<mark_weaver>sorry, that could have been explained more clearly, but maybe you get the idea.
<civodul>mark_weaver: i think this is similar to the hash-rewriting thing mentioned in TODO
<civodul>probably a good idea to speed up deployment
<civodul>(but note that the problem today is the lack of a fix, rather than slowness to deploy it)
<civodul> https://github.com/NixOS/nixpkgs/commit/d1662d715514e6ef9d3dc29f132f1b3d8e608a18
<civodul>so yes, that's definitely doable
<mark_weaver>civodul: well, lack of a fix is part of the problem, but the other problem is that to change one line of code, we have to do the equivalent amount of work as all the mental work that has been done by humans in the last 10000 years.
<mark_weaver>(well, that's a total guess, but I think it's probably comparable :)
<civodul>right :-)
<civodul>i wouldn't consider the CPU time too much of a problem, because processing is cheap ;-), and because it should be rare enough
<civodul>still, i agree it's better if we also have a fast way
<Calinou>better save CPU than RAM or bandwidth
<Calinou>from an ecological point of view
<mark_weaver>it doesn't seem to be too rare
<mark_weaver>and I don't think this amount of processing is cheap
<civodul>yeah, of course it's not really cheap
<civodul>but upgrading one's machine takes much fewer resources than rebuilding the complete distro
<civodul>but regardless, what you suggest would be a clear improvement
<mark_weaver>ideally, I'd like a system closer to the spirit of Emacs and Guile: after the pure build outputs have been produced, I'd like to be able (if I really want) to modify *any* line of code in the system and have that change take effect system-wide, efficiently on a modest system without using a build farm.
<civodul>yes, i see what you mean
<civodul>well, that's pretty much what we get with "imperative" distros
<civodul>so it's a matter of allowing that style, while keeping closer control on side effects
<mark_weaver>right, I want to have my cake and eat it too. I want to be able to create _pure_ builds, but I also want to be able to efficiently make mutated variants of those pure builds when needed, for the short term (either for experimentation or security patches).
<mark_weaver>my YeeLoong is still running with the glibc CVE, because I've not yet finished a rebuild on it before the next CVE prompted me to abort and restart the build.
<civodul>yes, some sort of C-M-x
<civodul>yeah, bummer
<DusXMT>Where can one buy a yeelong? I can't seem to find any offers on the internet
<mark_weaver>I really think that, going forward, it's going to be consideredd increasingly important to be able to apply security updates as quickly as possible. failure to do so effectively amounts to giving criminals such as the NSA access to our machines.
<mark_weaver>DusXMT: http://tekmote.nl/ still has some in stock, I think, the 8101B 10" being the best one that doesn't require blobs.
<DusXMT>mark_weaver: thanks
<mark_weaver>Danny Clark (former FSF sysadmin and founder of the now inactive "Freedom Included") also has several working Yeeloongs that he'd probably be happy to sell, and you'd be more likely to get one that actually works that way.
<Calinou>what's the point in Yeelong when there's X60
<mark_weaver>well, that's a good question. maybe none.
<Calinou>is the supply on X60s short like Yeelongs?
<Calinou>neither are manufactured today
<DusXMT>I like the fact that it's not x86, will give me an opportunity to learn a new assembly language ;)
<civodul>mark_weaver: agreed
*civodul has to go
<civodul>ttyl!
<mark_weaver>well, yes, it's important to support diversity in machine architectures.
<Calinou>what architecture is Yeelong?
<Calinou>I thought it was x86
<mark_weaver>it's based on 64-bit MIPS.
<Calinou>how is it called in Debian, mipsel?
<mark_weaver>the processor is actually a novel design, the Loongson processor architecture, which is compatible with MIPS.
<mark_weaver>preferably mips64el. last I checked, upstream Debian only supported 32-bit MIPS (mipsel), which is not taking very good advantage of the machine.
<mark_weaver>however, there is a good mips64el port of Debian now which will hopefully be pushed upstream to Debian proper soon.
<sneek>I'll keep that in mind.
<mark_weaver>sneek: forget however, there
<sneek>Okay.
<mark_weaver>sneek: botsnack
<sneek>:)
<mark_weaver>also, from a security perspective, I think there is a non-trivial advantage to running a minority architecture, since I guess there will be less effort put into developing exploits for it.
<mark_weaver>also, I take some comfort in the fact that the Loongson 2F processor in the YeeLoong does not have flashable microcode, so that's one less place where a back door could be planted. in fact, the early 2Fs had a bug that we've had to work around in our assemblers to avoid certain instructions.
<Calinou>if it's not updatable, why is the risk of backdoor lower?
<Calinou>not being flashable is to me a bad thing purely
<Calinou>bugs happen
<Calinou>Intel microcode updates aren't done only to put backdoor stuff :p
<Calinou>but very few people install them anyway
<mark_weaver>I think flashable is good when I have access to the source code of the flashable code and the means to build it from source.
<mark_weaver>otherwise, it's just a black box, and all I can do is accept some blob from the company and install it, without having the slightest clue whether it's inserting a new back door, maybe because someone from the government demanded that they do it.
<mark_weaver>or because the key used to sign the updates has been compromised.
<mark_weaver>or for that matter, if someone breaks into my machine just once, they can flash this blob that I have no way to audit to achieve persistence.
<mark_weaver>well, I guess the persistence depends on it really being "flash", which is not always the case.
<mark_weaver>well, that's my current opinion anyway. feel free to disagree :)
<jxself>Hmm. Wasn't there some way to re-use a package definition in another? This seems a good idea for linux-libre-3.14 but I don't seem to see mention of it in the documentation, unless I am missing it.
<davexunit>jxself: (package (inherit linux-libre) ...) ?
<jxself>Ah yes. Thanks. :)
<davexunit>when you use inherit you still need provide some of the fields like 'source', so using something else might make more sense.
<jxself>Ah, well I'm open to ideas if inherit isn't the best choice. It just seemed better than duplicating the whole package defintion.
<jxself>The only differences will be the version numbers, hashes, and the kernel configuration which will need to exist in a different file.
<davexunit>inherit should work fine, then.
<oiuoiu>civodul: where's the build log located, as produced by the -K option?
<oiuoiu>s/option/flag/
<oiuoiu>I've recently git pulled and now trying to build hello, which fails while building guile
<oiuoiu>In the log, I see memoize.c:515:***Mismatching FUNC_NAME. Should be: `#define FUNC_NAME s_"@prompt"'
<oiuoiu>and .libs/libguile_2.0_la-posix.o: In function `scm_tmpnam': /tmp/nix-build-guile-2.0.11.drv-2/guile-2.0.11/libguile/posix.c:1488: warning: the use of `tmpnam' is dangerous, better use `mkstemp'
<civodul>oiuoiu: "guix build --log-file guile" should give you the log's location
<civodul>but the above things are warnings, not errors
<oiuoiu>civodul: the build got killed after 3600 secs of silence
<oiuoiu>how do I investigate?
<civodul>oiuoiu: i would check the tail of the log first, to get an idea of where than happened
<civodul>then, if you used -K, you could try to source the 'environment-variables' file, and run 'make check' from there
<civodul>it's under /tmp/nix-build-guile-2.0.11*
<oiuoiu>civodul: tail -n500 doesn't seem to list anything useful
<oiuoiu>the only test which fails currently is guix-register (due to the sqlite3 error that has been already discussed several times)
<mark_weaver>oiuoiu: btw, the log is saved even if you don't pass -K. the -K just means to keep the build directory if it fails.
<mark_weaver>oiuoiu: what was it doing in the build when it went silent for 3600 seconds?
<civodul>oiuoiu: oh so is it Guix or Guile that fails to build?
<mark_weaver>civodul: he wrote above that the guile build failed (timeout after 3600 seconds)
<civodul>yes, but then they wrote something about guix-register
<mark_weaver>sounds like he's on a YeeLoong (based on his mention of the sqlite3 problem)
<mark_weaver>well, I'm not sure of that. now that I think of it, the sqlite3 problem I'm thinking of only happens when using 64K pages, (which I'm somewhat unusual in using)
<civodul>there was also a problem fixed in 689142cd
<civodul>that should have been a separate commit, though
<civodul>we're not done yet with the Bash thing: http://seclists.org/oss-sec/2014/q3/719
<mark_weaver>well, enough guessing, we should just wait for oiuoiu to speak :)
<mark_weaver>I read some of the discussion, and frankly it makes me uneasy :-(
<mark_weaver>it's really too bad that environment variables are exposed to such a complex parsing mechanism every time bash runs.
<civodul> http://www.openwall.com/lists/oss-security/2014/09/25/32
<civodul>urrgh
<mark_weaver>ouch
<civodul> http://lcamtuf.blogspot.com/2014/09/quick-notes-about-bash-bug-its-impact.html provides a good discussion
*mark_weaver reads
<civodul>mark_weaver: i couldn't resist to try out your suggestion: http://paste.lisp.org/+32ZO :-)
<civodul>and it actually works, but of course here it works because both libunistrings have the same SONAME
<mark_weaver>oooh!
<civodul>now we need a way to make it practical
<mark_weaver>my idea is that we'd replace the binding for 'bash' with a special kind of 'hash-patched' package object that refers to both the original package and the updated package. and then every package that depends on 'bash' would automatically become 'hash-patched' as well, and this would propagate upward.
<civodul>oh, ok
<mark_weaver>and then, in another branch, we'd replace 'bash' with the newer version (not 'hash-patched') and do a pure rebuild.
<civodul>yes
<mark_weaver>but that users would be able to use the hash-patched version in the meantime.
<civodul>yes, that's how it should be used
<civodul>i was thinking of having "grafts", like git used to call them
<civodul>so we would declare grafts somewhere in the distro
<civodul>and they would be magically used
<civodul>(a graft would be a mapping from bash -> bash-fixed)
<mark_weaver>yeah, "graft" is a good word for this
<civodul>then we need to figure out the details of the magic :-)
<mark_weaver>in the general case, a package's transitive dependencies might include multiple grafts.
<civodul>right
<mark_weaver>in which case, I guess the package should be turned into a derivation that uses as an input the same package but with only the non-grafted versions of all transitive inputs.
<mark_weaver>and then the derivation would simply find/replace all of the hashes of the grafts.
<civodul>yes
<mark_weaver>so, we'd need a procedure to compute the derivation of a package but ignoring all grafts (always using the originals).
<mark_weaver>and then another one to compute the set of grafts in the transitive inputs.
<mark_weaver>if that set is empty, then simply use the ungrafted derivation.
<mark_weaver>if the set if non-empty, then make a graft-substituting derivation based on the ungrafted derivation.
<mark_weaver>now that I think it through, it doesn't sound too hard.
<civodul>right
<mark_weaver>but then, I don't know the internals well enough to know
<civodul>heh
<civodul>actually, it may not be enough to look just at the inputs
<civodul>because some inputs may not be referred to in the build result
<civodul>or rather, looking at inputs may be too conservative
<mark_weaver>I may be using the wrong terminology, but by 'inputs' I meant the set of things that is made available in the chroot during the build process, and by 'transitive inputs' I mean the transitive closure of that.
<mark_weaver>so I didn't mean to exclude inputs whose hashes are not present in the outputs.
<civodul>ok
<mark_weaver>anyway, I'm happy to let you figure out any further details, if you're willing :)
<civodul>:-)
<civodul>i appreciate having your input on this
<mark_weaver>no problem, thanks for being receptive to my input :)
<mark_weaver>it would be interesting to see how often the grafted builds differ from the pure rebuilds.
<mark_weaver>my longer-term idea is to somehow track dependencies much more precisely. e.g. track which files are actually accessed in the inputs during a build, and eventually maybe even rig the compiler or linker to track which definitions are actually used, etc. but that's a much bigger job, and would also require changing some fundamental aspects of nix.
<mark_weaver>(probably it will never happen :)
<mark_weaver>one problem with the compiler/linker rigging idea is that it would be difficult to get right, and maybe brittle.
<civodul>yeah it would be nice to have a fine-grain view of what's actually used during the build process
<civodul>yeah
***inquiryq1eue is now known as inquiryqueue
<davexunit>how do we deal with test suites that attempt to test networking features?
<mark_weaver>we need to disable those tests
<davexunit>okay
<mark_weaver>unless the tests are purely local (e.g. between a server and client both running within the build container)
<davexunit>I think this test is using localhost only
<mark_weaver>that should work in principle, but the bare configuration in the build environment might be missing some things that the tests expect to find.
<mark_weaver>the build environment, /etc/hosts has just one entry, for localhost
<mark_weaver>(search for /etc/hosts in nix/libstore/build.cc)
<davexunit>thanks
<davexunit>ah, this test also expects there to be an ip6-localhost entry
<mark_weaver>hmm, we could add that, I suppose. maybe post to the list about it?
<civodul>yes, ::1 would be nice
<davexunit>I see that it's in Debian's default /etc/hosts
<davexunit>not sure about other distros
<davexunit>this would be a change to nix though, right?