IRC channel logs

2014-09-24.log

back to list of logs

<nully>mark_weaver: Hey, do you know of any dirs on hydra that do *not* need to be backed up?
<nully>Or regexes we could ignore?
<nully>Dont get me wrong, i wanna do full backups of hydra, but I'm not sure if we are backing up junk data :)
<nully>I say this because growing the volume on hydra has made space on the backup array a bit more crampped (although, i added more space so we're fine, this is non-urgent)
<nully>stuff like the .cache directory.... :D
<mark_weaver>nully: well, technically everything in /gnu/store could be rebuilt.
<mark_weaver>so maybe it's okay to exclude that from the backups, but best to ask civodul.
<nully>mark_weaver: will do :
<nully>:)
<nully>ya, i just dont wanna back up / restore stuff that you'll just rm/regen as soon as i turn it over to you
<nully>(assuming total dataloss)
<nully>i'm moving us to rdiffbackup, so that will help too
<nully>thx :)
<nully>how goes everything in the world of GUIX?
<mark_weaver>exciting times for Guix! we have managed to attract quite a few contributors
<mark_weaver>so the situation with /gnu/store is that it would take a long time to rebuild all of it, but it could be done.
<mark_weaver>it's certainly not "junk", and we'd never rebuild it if we didn't have to.
<mark_weaver>but if we lost it, it wouldn't be disastrous.
<mark_weaver>the primary purpose of hydra is to orchestrate our build farm to build the contents of /gnu/store and make it available to users so they won't have to build all of that stuff themselves.
<mark_weaver>nully: is hydra's /gnu/store on RAID?
<nully>raid1
<nully>single mirror
<nully>I would like to eventually move all our VMs to raid10
<nully>via fiberbackbone SAN
<nully>probably iscsi for transport
<nully>raid1 is slow, even more slow when the same spindle is part of other volumegroups (partitions)
<nully>I'm sure you're not seeing the best disk speed.
<nully>that will be a fun transition when its time.
<mark_weaver>heh :)
<nully>(I'm already testing fibercards/corebooting targets)
<nully>Hopefully before the end of the year we start the move.
<mark_weaver>well, as long as we're on RAID, the probability of losing anything in /gnu/store is low enough that we could probably suffer the inconvenience/delay of rebuilding /gnu/store.
<mark_weaver>if we lost /gnu/store because of a security breach, we'd probably want to rebuild everything from scratch anyway.
<nully>how often does the stuf fin /gnu/store change?
<nully>like how many GB of diff a day you think?
<mark_weaver>and /gnu/store is the overwhelming user of disk space on hydra.
<nully>or a ratio
<mark_weaver>it doesn't change at all except to add new stuff to it.
<nully>oh, well then. Maybe i'll just migrate that to rdiff-backup.
<mark_weaver>i.e. new files and directories are added to /gnu/store, but nothing that's already in /gnu/store should ever change.
<mark_weaver>things are deleted from /gnu/store as well, I should have mentioned that too.
<mark_weaver>/gnu/store is not only the biggest user of space on hydra, it's also where most of the change is happening, in the form of new files/dirs being added and others deleted from it.
<mark_weaver>as for the rate of change, it's *highly* variable.
<nully>mark_weaver: what is /nix/store?
<mark_weaver>it's the old name for /gnu/store, and I guess maybe hydra has some things there too, but /nix/store probably doesn't change anymore, or at least not often.
<mark_weaver>(Guix is derived from Nix, and Nix kept everything in /nix/store. a while back we changed Guix to use /gnu/store instead)
<nully>hahaahah
<nully>its in the excludes directory for your box
<nully>haha, thats funny.
<jxself>If it's excluded perhaps /gnu/store should be too?
<nully>mark_weaver: just curiously, how long ago do you think that changed?
<nully>jxself: Depends, I think i'ma give it a shot on an incremential system.
<nully>i just bump'd our backup array from 10TB -> 14TB today so :D
<mark_weaver>nully: we did the switch in March of this year.
<nully>mark_weaver: that matches my graphs perfectly :D
<mark_weaver>heh :)
<nully>our old backup system bases its stats on du, and its just a pain to read.
<nully>Well anyways, that solves that mystery. I've learned a lot.
<civodul>Hello Guix!
<civodul>ph4nt0mas: seems like you have networking issues :-)
<ph4nt0mas>hey civodul , yeah It's doesn't want to work
<ph4nt0mas>and it's getting on my nerves
<civodul>heheh :-)
<civodul>did you have a chance to look at my comments for wip-hurd ?
<ph4nt0mas>yep I have a patch ready
<ph4nt0mas>did find the chance to sent it
<civodul>oh cool
<ph4nt0mas>I wanted to ask you something though
<ph4nt0mas>give me a sec to remember what it was
<ph4nt0mas>:P
<phant0mas>civodul: about the libpthread-glibc-preparation.patch
<phant0mas>about the #define __PTHREAD_SPIN_LOCK_INITIALIZER __SPIN_LOCK_INITIALIZER
<phant0mas>I didn't found any mail on the list that told me to initialize with that
<phant0mas>I just saw it done that way in the debian package I think
<phant0mas>it gets initialized to 0
<phant0mas>and it seems to work
<civodul>heheh, ok :-)
<civodul>then could you just add the URL of that patch?
<civodul>presumably it was blessed by youpi
<phant0mas>I am looking for that :P
<phant0mas>I am stranded on Paros, with no stable connection
<phant0mas>I am using a long range wifi antenna
<phant0mas>to borrow someone's else connection :P
<civodul>heh
<phant0mas>btw I am glad about getting some newer releases from the hurd guys
<phant0mas>hope they start using the latest edition of glibc
<phant0mas>civodul: http://osdir.com/ml/commit-hurd-gnu/2009-04/msg00006.html check this commit log from Samuel
<phant0mas>he actually __SPIN_LOCK_INITIALIZER to __PTHREAD_SPIN_LOCK_INITIALIZER in some files and started using it, so they are actually the same thing
<phant0mas>he actually changed*
<phant0mas>changed/renamed
<phant0mas>it seems along with my connection, my language skills are refusing to work as well
<civodul>:-)
<civodul>oh ok, so please refer to that commit in the patch
<civodul>(use a git.savannah.gnu.org URL, not osdir.com)
<phant0mas>okay
<phant0mas>sending the patch in 10
<civodul>no rush, take your time ;-)
<phant0mas>let's hope my connection will not crash before that :p
<phant0mas>civodul: what commit message should I use about the fixmes ?
<civodul>phant0mas: it doesn't matter because we'll merge the commits
<phant0mas>okay!
<phant0mas>sent the patch using a connection 10 miles away
<davexunit>"remote code execution through bash" http://seclists.org/oss-sec/2014/q3/649
<mark_weaver>ouch
<mark_weaver>I confess I have no idea what he means by "trailing code in function definitions was executed, independent of the variable name", but based on his conclusion, I'll assume it's bad :-(
<philed>env x='() { :;}; echo BOO!' bash -c "hello"
<philed>env x='() { :;}; echo BOO' bash -c "hello"
<philed>or something like that
<mark_weaver>*nod* readingah, I didn't even know that bash supported exporting functions
<mark_weaver>bah
<mark_weaver>*nod* I'm reading the followup message which talks details. I didn't even know that bash supported exporting shell functions.
<mark_weaver>ouch, this is really bad :-(
<jxself>Time for an update.
<mark_weaver>and another full rebuild :-(
<jxself>Just for bash?
<davexunit>I haven't paid close attention to security updates in guix. patching software as fundamental as bash requires rebuilding basically everything, which means our response time to users is a lot slower than other distros.
<jxself>er BASH
<mark_weaver>bash is an input to every package, and our libc includes an absolute path to its own bash for use by 'system'
<jxself>A bigger build farm is a solution to that?
<davexunit>jxself: bash is an implicit input to the gnu build system. since nearly everything uses that system...
<davexunit>I guess not 'nearly', 'every'.
<mark_weaver>in theory, we could just patch the one that users see directly, without patching the one that's used to build everything else. but the one that's hardcoded by libc is still a problem.
<davexunit>I believe nix has some hack to deal with it, but it didn't sound great, but I don't really know.
<mark_weaver>and it's probably also a problem that the hardcoded path to bash is incorporated by a lot of the stuff we build (put in the shebangs at the top of scripts, etc)
<mark_weaver>well, the patch is already available, so I guess we should start moving on this.
<mark_weaver> http://ftp.gnu.org/pub/gnu/bash/bash-4.3-patches/bash43-025
<mark_weaver>wow, context diffs. I almost forgot about those :)
<mark_weaver>civodul: urgent issue
<mark_weaver>"remote code execution through bash" http://seclists.org/oss-sec/2014/q3/649
<mark_weaver> http://ftp.gnu.org/pub/gnu/bash/bash-4.3-patches/bash43-025
<civodul>ouch
<civodul>ok, let's do it again
<civodul>mark_weaver: do you want to handle it or should it?
<civodul>*should i?
<mark_weaver>if you could handle this one, I'd be grateful.
<civodul>ok, let's go :-)
<mark_weaver>most of the actual details are here: http://seclists.org/oss-sec/2014/q3/650
<civodul>ok
<civodul>thanks for the heads-up!
<mark_weaver>np!
<mark_weaver>thanks for agreeing to take care of it :)
<civodul>mark_weaver: should we apply the whole series?
<civodul>i'm asking because the patch doesn't apply as is
<civodul>it modifies patchlevel.h, which depends on earlier patches
<davexunit>FSF just promoted our hackathon on pump.io, gnu social, and twitter. yay!
<davexunit>now back to the critical security vuln.
<civodul>cool :-)
<mark_weaver>civodul: yeah, I was also wondering about that.
<civodul>i think i'll just apply them all
<mark_weaver>civodul: it would probably be ideal to include the entire series.
<civodul>yeah
<civodul>ok
<mark_weaver>I spoke to nully yesterday, and hydra has much more disk now, so that's good.
<mark_weaver>(thanks nully!)
<civodul>great
<davexunit>I think that for the hackathon I'm going to focus on generalizing 'guix import' to support multiple backends and then add the pypi backend.
<davexunit>civodul: got someone that is interested in translations for gnu guix. is there anything that they could work on during the hackathon? I don't know anything about our translation system.
<civodul>davexunit: definitely: figuring out how to use the www.gnu.org translations and handle them with the TP
<civodul>:-)
<civodul>i can provide more details later
<mark_weaver>I hand-patched 'bash' on hydra, since the update is not yet available for trisquel.
<davexunit>mark_weaver: yeah, we're still waiting for the patch in trisquel here.
<jxself> https://trisquel.info/en/issues/12244
<jxself>Given that, I suspect it won't be until after quidam returns this weekend. He went on a 68 mile hike with some friends.
<davexunit>thanks jxself
<jxself>Assuming he survives the hike. :)
<mark_weaver>ouch.
<mark_weaver>well, it's not hard to patch the debian package. a few little gotchas, but not bad.
<davexunit>something something bus factor
<mark_weaver>probably you could just use the ubuntu package. I suspect that trisquel hasn't actually modified the bash package at all.
<mark_weaver>(and it if has, the trisquel patches are probably not as important as this)
<jxself>OK, two people have said they like the longterm release kernel idea. I guess that means "go ahead." I was thinking of 3.14 because it's the newest and supported until September 2016, unless people would prefer an older one/multiple ones.
<jxself>er August 2016. Sorry.
<jxself>I guess I can't read https://www.kernel.org/category/releases.html :/
<civodul>i've pushed a branch with the patch series
<civodul>can mark_weaver or somebody else create the jobset on Hydra?
<civodul>i need to take care of the kids, will be back in 1.5 hours
<jxself>You have kids? News.
<civodul>yup
<mark_weaver>civodul: okay
<mark_weaver>thanks for patching it!
<jxself>I have a question about the longterm kernel stuff although I'm not sure how to ask it. So I'll just ramble and hopefully it'll make sense to someone.
<jxself>I was thinking of making a package named linux-libre-lts to hold 3.14, since it seemed to go well wit[D[D[Dh the current name.
<jxself>Since kernel.org has multiple longterm release kernels at the same time, what to do when the next one becomes available? Ignore it? People might want it? Support it too? AFAIK a given package can't be for multiple versions at the same time so multiple kernel packages would be needed if multiple longterm kernels are to be supported at the same time?
<jxself>In that case a different package name is probably needed, like linux-libre-3.14? Then what happens once 3.14 is end-of-life? They'll stop getting updates unless they manually select a newer kernel package? I'm not sure if it's ideal or not.
<mark_weaver>I think linux-libre-3.14 is a good name, and that we should leave open the possibility of having multiple lts kernels.
<jxself>And then when 3.14 reaches end-of-life? There'd be no future updates for that package then, so they'd need to go manually pick a newer kernel package as opposed to an upgrade being offered by the package manager?
<jxself>Or maybe I'm just overthinking?
<mark_weaver>I think when a new lts kernel becomes available, we should add that new one without removing 3.14, and when 3.14 reaches EOL then we should remove 3.14.
<mark_weaver>does that make sense?
<jxself>It does. I guess I'm just concerned over leaving people potentially with an unsupported kernel version unless they somehow know to take action to get a newer one.[C
<jxself>But it's probably the best option I suppose.
<jxself>Just thinking is all.
<mark_weaver>well, when we remove the 3.14 package, the next time they run guix system reconfigure it will report an error that the package doesn't exist, and then they'll know to choose a newer version.
<jxself>OK.
<mark_weaver>okay, hydra is busy building the branch with the security fix.
<mark_weaver>and I cancelled all the pending MIPS builds on master
<mark_weaver>I wonder why hydra sees fit to only have 3 concurrent builds total.
<mark_weaver>nvm, the number of increasing.
<mark_weaver> http://hydra.gnu.org/eval/100731?compare=master
<mark_weaver>^^ to track the build progress
<mark_weaver>civodul: nice job with the bash patches! It would have taken me a *lot* longer to do the job as nicely as you did.
<mark_weaver>gnupg-verify* should probably accept a keyid, and make sure it was signed with the expected key.
<mark_weaver>well, better yet, a fingerprint
<mark_weaver>I've actually been thinking that our package descriptions (or maybe origins) should include fingerprints of keys that are expected to sign the source code, for use by guix refresh.
<civodul>mark_weaver: yes, could be a good idea
<civodul>for GNU packages that info must be available somewhere already
<Calinou>hi, I will look into French translation later.
<Calinou>bye for now
<civodul>hi Calinou
<civodul>ok
***tadni` is now known as tadni