IRC channel logs

2015-03-17.log

back to list of logs

<phant0mas>good morning guix
***tschwing_ is now known as tschwinge
<civodul>Hello Guix!
<civodul>sneek: seen mark_weaver
<sneek>mark_weaver was here Mar 15 at 04:02 pm UTC, saying: rekado: looks good to me, please push!.
<phant0mas>hey civodul there is something wrong with the hurd dynamic linker, that's why coreutils executables fail to run on hurd
<phant0mas>it fails with EXC_BAD_ACCESS, investigating with gdb
<civodul>phant0mas: you could investigate first with the LD_DEBUG environment variable (try LD_DEBUG=help)
<civodul>and then, it it turns out that it fails earlier than that, you could try rpctrace
<civodul>phant0mas: BTW, make sure to submit your proposal on melange
<phant0mas>I am finishing my proposal today and will upload it probably within the day
<civodul>ok
<Sleep_Walker>can guix-daemon use ccache?
<civodul>no, and that would defeat the reproducibility goal
<Sleep_Walker>in which way?
*davexunit hacks on guix publish
<davexunit>it's strange, only certain store items get corrupted.
<davexunit>I just don't know how...
<davexunit>ruby fails, but I just tried srt2vtt, pflask, and httpd and they all work.
<davexunit>nvm, it only worked on cached narinfos from hydra
<davexunit>my narinfo looks right. the NarSize field is exactly the same
<civodul>davexunit: could it be that you have erroneous cached narinfos?
<civodul>you can "rm -rf /var/guix/substitute-binary/cache" to be sure
<davexunit>I clear the cache each time
<civodul>okay
<davexunit>the narinfos look right
<civodul>so you manage to substitute some items, but not all?
<davexunit>I thought so, but that's wrong.
<davexunit>I forgot to clear the cache once
<davexunit>and had successful substitutes
<civodul>also see http://bugs.gnu.org/19621 notably
<civodul>it's the kind of thing that can hurt badly
<davexunit>civodul: current backtrace http://paste.lisp.org/display/146336
<davexunit>I removed 'with-error-handling' from guix substitute-binary to get this
<civodul>19621 actually this one leads to an assertion failure, so that's ok
<civodul>hmm
<davexunit>I have gotten this before, on the server: ERROR: In procedure fport_fill_input: Connection reset by peer
<davexunit>
<davexunit>but that's due to the the above crash
<davexunit>which closes the connection to the server
<civodul>and what do you have in /tmp/foobar/bin ?
<civodul>when the error is raised
<davexunit>pflask, an executable
<civodul>and is this file complete or truncated?
<davexunit>is what I'm left with
<civodul>or corrupt?
<davexunit>I think truncated... let me see
<davexunit>Job 1, “/tmp/foobar/bin/pflask ” terminated by signal SIGSEGV (Address boundary error)
<civodul>can you compare its size and contents with the real file?
<davexunit>sure
<davexunit>the real plask is 43K
<davexunit>the corrupt one is 16KB
<civodul>ok
<civodul>to try to isolate the problem, could you download the nar with wget
<civodul>and then feed it to the 'import-path' (singular) procedure?
<davexunit>okay
<davexunit>I can see that the contents of the corrupted pflask start to vary from the real one
<davexunit>I'll try import-path now
<civodul>hmm
<civodul>actually there's no singular version, wait
<civodul>well you can feed it to restore-file
<civodul>the nar is uncompressed, right?
<davexunit>yes
<civodul>ok
<civodul>at least if you try with wget we can see if it's a server-side problem or not
<civodul>like, you could download it several times and make sure you always get the same result
<civodul>no truncation, etc.
<davexunit>I get the same result with restore-file
<davexunit>seems to be the same result from the server
<civodul>ok
<davexunit>I just use write-file
<civodul>now, if you just use 'write-file', do you obtain a bit-for-bit identical stream than what you get with wget?
<davexunit>let's see...
<davexunit>hmm... no.
<davexunit>different hash
<civodul>ok, that's a good hint
<davexunit>encoding problem?
<civodul>dunno
<rekado_>I have a couple of small patches for icedtea6 to make it easier to reuse build phases in a future icedtea7 package (which still fails with a linker error at this point).
<civodul>davexunit: does the server always produce the exact same byte stream, if you run wget multiple times?
<davexunit>yes
<rekado_>Should I submit these non-functional patches already or rather with the addition of icedtea7?
<civodul>rekado_: if they don't break icedtea yes, sounds like you can submit it already
<civodul>davexunit: can you use "wget --save-headers" and inspect that?
<rekado_>civodul: they don't break the build, but they don't add any features or anything.
<davexunit>civodul: sure
<civodul>that is: "wget -O foo --save-headers http://localhost/..."
<davexunit>I'm looking at the nar in hexl-mode
<civodul>and then edit 'foo' and check the top
<davexunit>the first difference I see is that at some point it writes 3f01 instead of c001
<civodul>we should add a procedure to do like 'tar tvf'
<davexunit>HTTP/1.1 200 OK
<davexunit>Content-Length: 54848
<davexunit>Content-Type: text/plain;charset=utf-8
<davexunit>
<davexunit>that's the correct content-length
<civodul>"application/octet-stream" would be better
<davexunit>I'll change content-type
<davexunit>yeah
<civodul>but i don't know if that would make a difference
<davexunit>I was under the impression that it didn't, but maybe...
<davexunit>no difference
<davexunit>Content-Type: application/octet-stream;charset=utf-8
<civodul>remove 'charset'
<davexunit>I was about to mention that
<davexunit>I don't specify that
<civodul>'response-body-port' in (web response) takes it into account
<davexunit>it just does that
<davexunit>I'll have to read the docs some more
<davexunit>'((content-type . (application/octet-stream)))
<davexunit>that's all I specify for headers
<civodul>i think you can make it ((content-type . (application/octet-stream)) (charset . #f))
<davexunit>okay
<davexunit>#f is no good, one sec...
<civodul>yeah actually 'sanitize-response' forces a charset
<civodul>so you have to make it "ISO-8859-1"
<davexunit>THAT WORKED!!!!
<davexunit>I knew it was going to be something silly like that.
<davexunit>thanks for working through this with me
<civodul>bah, this is terrible, but good news anyway
<civodul>thanks for persevering! :-)
<civodul>i think we should change the http server to allow us to not provide a 'charset'
<davexunit>I'm surprised I haven't been bitten by this earlier with guix web
<civodul>but that 3f instead of c0, do you know roughly what the corresponds to?
<civodul>i can't see where 'write-file' would use textual output procedures
<civodul>it shouldn't be sensitive to the port's encoding
<davexunit>it's within the contents of the pflask executable
<civodul>oh but maybe wget itself would do some interpretation of all those bytes too
<civodul>yes, that's what's happening
<davexunit>yeah, but the guile http client does the same
<davexunit>no?
<civodul>yeah probably
<davexunit>okay
<civodul>well
<civodul>normally 'restore-file' doesn't use textual input procedures eithe
<civodul>*either
<civodul>so it shouldn't be sensitive to the port's encoding
<civodul>hmm
<davexunit>civodul: as a first step, could I post a patch for 'guix publish' that doesn't do compression, and then make a later patch that adds it?
<davexunit>since there's work to be done on the compressed port procedures
<civodul>yes, definitely
<davexunit>okay, cool.
<davexunit>civodul: if you visit <http://hydra.gnu.org/nix-cache-info>, are "WantMassQuery" and "Priority" configurable things in Hydra?
<davexunit>I have them hardcoded at the moment
<civodul>they're also hardcoded in Hydra :-)
<davexunit>heh
<davexunit>alright then!
<civodul>WantMassQuery is there because of multithreading in the substituter
<civodul>(which i still hope to get rid of)
<davexunit>ah, I see.
<davexunit>alright, so I'm going to add just a couple of options to the script then: --port and --repl
<davexunit>should be enough for now
<civodul>yes, sounds good
<rjcode`>Howdy - anyone installed Guix on VMWare?
<davexunit>I don't know if anyone here has tried that yet.
<davexunit>I'd recommend using a free virtualization platform, though.
<rjcode`>VBox, sure can do that.
<davexunit>someone here has used virtualbox, but I think the complication at the moment is that we don't have ISO images.
<davexunit>so there's an extra step involved that I can't remember. anyone else know what it was?
<rjcode`>Using NixOS, but prefer Emacs/Lisp and the rest. So hoping to get going with Guix.
<davexunit>awesome. :)
<rjcode`>Nix language just not working out :)
<davexunit>you'll find that we have less stuff than Nix, but a more unified UI and API
<davexunit>since everything is Scheme.
<rjcode`>Yup, Guile looking good.
<rjcode`>So would imagine Emacs/Elisp on NixOS -> GuileEmacs/GuileScheme on Guix was the concept
<rjcode`>But just read a bit on Guile etc so far.
<davexunit>our emacs is the upstream emacs, thought guile emacs is a project we're certainly interested in :)
<davexunit>s/thought/though/
<rjcode`>But seems nix way of storing things is fundamentally better way
<davexunit>what do you mean?
<rjcode`>hash-program-version
<rjcode`>so can mix & match, atomic etc
<davexunit>oh, yeah.
<davexunit>we do the same.
<davexunit>we use all the low-level Nix stuff.
<rjcode`>just don't get why Nix language...lisp or haskell would have seemed more natural...maybe I'm thinking about it all wrong though
<davexunit>yeah, we feel that using an existing general-purpose language is a better decision.
<rjcode`>yup
<davexunit>Guile already has a bunch of useful libraries, Emacs integration, etc.
<rjcode`>In case I'd be brave enough to work on Guix, which parts of NixOS etc do you guys build on? Is there a document that lays that out?
<rjcode`>cool
<DusXMT>rjcode`: The nix daemon is in place. Guix, in essence, generates derivations and hands those to the nix daemon. Of course, it does more as well
<davexunit>rjcode`: we use the Nix daemon and the same file formats for representing derivations and such.
<davexunit>everything else is our own.
<davexunit>so we don't use the NixOS stuff at all.
<davexunit>just some Nix stuff.
<rjcode`>Ah, great!
<rjcode`>Looked at gnu git, saw two guix repos
*DusXMT heard a rumor that even the nix daemon may be rewritten in the future, since it's in c++, to lighten up dependencies
<davexunit>yeah, it would be ideal for the daemon to be written in Guile.
<davexunit>more hackable, easier for us to maintain.
<civodul>+1
<civodul>probably post-1.0 though :-)
<davexunit>yeah
<davexunit>civodul: do you have a preferred way for validating command line options?
<davexunit>so the user doesn't get a backtrace when they do "guix publish --port=foo"
<civodul>davexunit: if you use with-error-handling + parse-command-line, that should DTRT
<davexunit>okay
<civodul>in the example above you also need (unless (string->number port) (leave (_ "invalid port number")))
<davexunit>okay, that's what I was looking for
<davexunit>thanks
<civodul>there's actually 'string->number*' which does exactly that, in (guix ui)
<davexunit>perfect!
<davexunit>totally missed that, thanks.
<Sleep_Walker>how can ccache defeat the reproducibility goal?
<Sleep_Walker>for the same input it provides same output...
<Sleep_Walker>at least, it is intended to do so...
<Sleep_Walker>or is also different timing an issue?
<Sleep_Walker>damn
<Sleep_Walker>sorry about that commit
<Sleep_Walker>I'm trying to clean the gnash package and I prematurely pushed agg
<civodul>Sleep_Walker: ccache relies on state which by definition would be outside of Guix's control
<civodul>and typically, ccache can sometimes fail at providing reproducible builds
<rekado_>civodul: I see you added python2-rsvg. I have this in my queue.
<rekado_>I created a function that produces packages for the gnome-python components.
<bavier>civodul: ccache can be configured to be fairly conservative. I wonder if we might be able to find some settings that could work for us.
<civodul>rekado_: oh, excellent, sorry for the message then
<civodul>err, *mess
<Sleep_Walker>I'm afraid this is difference between 'use it' and 'rely on it' - it can be still helpful even when not absolutely reliable
<civodul>rekado_: feel free to override the one i added
<Sleep_Walker>but it can speed up development
<civodul>do you think so?
<rekado_>civodul: I noticed that we are still missing a couple packages to build all of the components, so I haven't yet proposed my patches to the ML.
<rekado_>but I shall do so soon.
<civodul>sounds good
<civodul>oh the Manifesto is 30 this month
<civodul> http://www.newyorker.com/business/currency/the-gnu-manifesto-turns-thirty?intcid=mod-latest got everything right
<civodul>(which is quite unusual for large-audience papers, i think)
<davexunit>civodul: I know, right?
<davexunit>I just read it and was very impressed.
<Sleep_Walker>during my gnash journey I rebuild it ~7 times till the end - the build took at least 25 minutes (sorry, I can just estimate here)
<Sleep_Walker>some of the rebuilds could have been speeded up
<Sleep_Walker>last one - I missed command for publishing browser plugin - I had to wait whole the time just to get the plugin :)
<civodul>yeah
<rekado_>Sleep_Walker: the icedtea builds always took a little more than an hour on my machine. And the tests took much longer than that :) It's annoying when you get something wrong somewhere at the end.
<davexunit>yeah
<davexunit>it would be nice to be able to jump into the build container like Sree mentioned on guix-devel
<Sleep_Walker>Well, my question is - is the possibility of non-reproducible builds complication that big we'd rather not use it even though we don't need to rely on it?
<davexunit>one thing that I think Nix might have an advantage on is that build phases are formalized and exposed to user code, so it's easy to run individual build phases over again.
<davexunit>I've seen nix-env users do it.
<davexunit>Sleep_Walker: well, non-reproducible builds are bad.
<Sleep_Walker>If I want reliable build, I won't use ccache, if I want prepare package, I'd enable that...
<Sleep_Walker>or are you trying to protect your users from themselves?
<davexunit>the goal is to approach 100% build reproducibility, for correctness and security reasons.
<davexunit>the same input should yield the same output, and we must work to fix any sources of non-determinism.
<bavier>I know it might be difficult, but I'd like to see an example of ccache not providing reproducible output.
<Sleep_Walker>do you think that the reproducibility problems I may meet will influence reproducible builds of all others?
<iyzsong>the Paludis, package manager for Exherbo, can save and resume build phases. one of features I miss from it ;-)
<davexunit>perhaps we should formalize build phases for all build systems.
<davexunit>need a better understanding of the negative effects of that, though.
<civodul>is anyone handling the xcape patch?
<civodul>would be nice :-)
<civodul>davexunit: yeah in Nix the build phases are Bash functions, so they can be called from a nix-shell environment
<civodul>i guess we could have a Bash built-in or something to do that
<davexunit>yeah, maye, I dunno what the implementation of that would look like.
<davexunit>taylanub: just saw your mesa patches. I'm hoping they fix a regression that sirgazil brought to my attention.
<davexunit>SDL can no longer open X windows with a GL context
<davexunit>thus my game engine is broken on Guix :(
<taylanub>davexunit: any error message or so? I don't know of a regression; just dlopen() calls that were never patched (so they never worked, probably .. unless if you're using guix on another system maybe)
<davexunit>I can't find anything yet. all I know is that an SDL call fails.
<iyzsong>I don't think the patch for mesa will get SDL/GL work, SDL can't find GL now >.<, I will fix it next day, if no one beat me.
<taylanub>by the way I thought ccache would not affect reproducability, or anything; I thought it intends to be 100% transparent, not changing compiler output in any way. of course, there is bug potential.
<davexunit>iyzsong: I can't find anything that shows that SDL can't find GL
<davexunit>the build looks good when glu is added as an input, and strace doesn't show anything alarming to me.
<taylanub>davexunit: any dlopen calls in the trace? any dangling .so references in ldd output for libsdl or w/e?
<davexunit>taylanub: how do I tell if a .so reference is dangling?
<taylanub>if the ldd output says "=> (not found)" or so for an .so, or points to a path that isn't /gnu/store/...
<taylanub>maximally generally, I wrote this dirty script to find all dangling .so references in one's /gnu/store, but it takes very long: http://sprunge.us/JEjD
<davexunit>nope, everything looks good
<davexunit>there's no references to libGL though
<taylanub>indeed, I wonder how it loads libGL
<iyzsong>oops, SDL use SDL_LoadObject to dynamic load libGL, which was disabled by me
<taylanub>there's a configure flag --disable-sdl-dlopen
<taylanub>maybe it has no non-dlopen fallback for loading GL, so it disabled GL support entirely?
<davexunit>iyzsong: ohhhh
<taylanub>fixing dlopen() uses manually seems very difficult to me by the way. I wonder if we can solve it in a smarter way.
<davexunit>hmmm
<davexunit>iyzsong: hmm, it would suck to have to revert that...
<iyzsong>davexunit: does add '-lGL' to sdl not work?
<davexunit>that didn't help
<iyzsong>davexunit: My last try, please change 'video->gl_config.driver_loaded = 0' to '1' in src/video/SDL_video.c
<iyzsong>with the '-lGL'
<taylanub>if you have the build with -lGL lying around, I'd first check if it references /gnu/store/...-mesa.../ at all, via 'guix gc -R' or ldd
<davexunit>will try
*iyzsong still wonder wheather disable dlopen for sdl is a good idea or not
<davexunit>might not be :)
<iyzsong>ok, go sleep now, bye #guix
<davexunit>see ya
<davexunit>taylanub: well, it references libgl now
<civodul>bavier: impressive work with all these Perl modules!
<civodul>are the latest ones also for Hydra?
<bavier>yes
<bavier>thanks
<bavier>primarily the perl-dbix-class-schema-loader
<davexunit>bavier: I can't believe how many perl libs you've packaged
<davexunit>very impressive
<bavier>davexunit: thanks. I've lost track of how many I've committed so far
<davexunit>I wish I could get the basic ruby libraries packaged
<davexunit>but I've just run into problem after problem
<remi`bd>hi, maybe I should precise I’m Rémi (started the last thread about GSoC)
<davexunit>I don't know how to avoid propagating every ruby library that a package depends on. :/
<bavier>davexunit: unfortunately that's what needs to happen with the perl modules also
<bavier>if there's a way to get around that with ruby, that'd be great
<Sleep_Walker>hi remi`bd :)
<davexunit>bavier: I've been working on it. I have a hack that could work, but only for executables.
<remi`bd>I want to apologize: I didn't see Rohan Prinja’s mails before sending mine, otherwise I would not have sent my proposal on the same project.
<remi`bd>(well, my draft proposal)
<civodul>hi remi`bd
<civodul>remi`bd: no need to apologize
<civodul>you could discuss with Rohan and see what they think
<civodul>if one of you wants to pick another project, the better! ;-)
<civodul>ph4nt0mas: thanks for the proposal on melange, looks good to me!
<remi`bd>well, I’m already looking at another idea :)
<civodul>good :-)
<civodul>ph4nt0mas: the "13 June – 23 June" milestone is about porting GuixSD (not just Guix), if you can correct it