IRC channel logs
2015-06-14.log
back to list of logs
<civodul>mark_weaver: i'm testing a fix for the glibc/RUNPATH issue <davexunit>hmm, how to name 2 packages that differ only by the ruby interpreter used <davexunit>for python, we have python-foo and python2-foo <davexunit>there's ruby 2.2, ruby 2.1, and I might even add in ruby-1.9.3 <davexunit>ruby-2.1-foo doesn't work well, because shell commands interpret that as a package called "ruby" with version "2.1-foo" <davexunit>but I need some way to distinguish between minor versions <lf94>davexunit where is the best place to learn how to create a guix package <lf94>I'm going to take a crack at zim <ewemoa>i'd eventually like to be able to determine which package(s) lead to the existence of files w/ particular names ending up in the store (e.g. which package(s) contain files with names containing the string "texi2pdf") -- has anyone worked on this sort of thing? perhaps a running hydra might be in a good position to collect relevant info? <rekado>The GuixSD website behaves a little odd when zooming around. <rekado>The text zooms, but the decoration does not. <rekado>e.g. the hover highlights in the menu bar seem to have a fixed distance to the text, but the dark gray bar does not scale along with the text, so the highlights end up in the lighter grid-patterned area. <rekado>The text in the grid-patterened area will also overflow when zoomed in. <rekado>It would be nice if the background heights also zoomed together with the text. <alirio>ewemoa: do you know "find /gnu/store -name '*texi2pdf*'"? <rekado>alirio: this works, and I use it all the time, but it is obviously limited to things that have been built. <rekado>you cannot discover the package that provides a particular file. <rekado>I can only see this work if the package definitions also included a list of files. <rekado>anything else would require talking to a build farm that builds everything. <rekado>we have come to a similar conclusion earlier when we talked about enhancing the linter to ... erm, I forgot. Something with a list of executables. <amz3>maybe it was about "package-missing" bash feature <amz3>in would be nice :) not sure it's useful for a lot of people <amz3>(ebuild.find was my very first web app) <rekado>I'd much prefer a solution that doesn't *require* the use of a build farm. <ewemoa>well if the p2p building works out, perhaps participants can share results :) <alirio>ewemoa: this is a distributed build farm <rekado>as people have to build recipes locally anyway before submitting them upstream I wonder if we could generate a file list at this point and deliver it together with the recipe. <alirio>rekado: it would not be source, so you won't use git for that <rekado>alirio: I don't understand what you mean. <ewemoa>not sure all files would be determined in a specific build (e.g. could some results differ by architecture?) -- but may be results can be collected <alirio>rekado: you submit a patch made with git, containing source. a file list would be a byproduct of the build and it should be build by the farm to ensure trust, reproducibility, ... <rekado>alirio: well, with two passes over the build the file list could be made an input. <mark_weaver>I've tried using zoom to make every very large and very small, and everything including the bars seem to scale appropriately. <rekado>ah, "View > Zoom > Zoom Text Only" was enabled. <rekado>alirio: hydra can do this (like anyone else) because it has built the package. <rekado>I think it would not be good to add a feature that depends on a build farm. <alirio>rekado: lets make people upload the binaries so we don't need build farm at all </irony> binaries and file lists are both byproducts of the build and I can't see any reason to handle specially the latter <davexunit>I need to determine if the gem archives on rubygems.org count as complete, corresponding source code. <davexunit>if they are, it might make the ruby build system simpler, and make it much easier to write an import script. <amz3>davexunit where is the code of the guix webui? <davexunit>amz3: I mean, is a Ruby gem equivalent to a binary in the legal sense? <davexunit>is it actually the complete corresponding source code (to use the GPL's language)? <amz3>it's probably better to install from source code and not use binary upstream distribution <davexunit>ruby is an interpreted language so the source code *is* in the gem <davexunit>but it might be missing files used to build the gem <davexunit>or perhaps there's programmatically generated ruby code in it. <davexunit>there's some hubub going on on a bioinformatics mailing list that I'm subscribed to <mark_weaver>davexunit: how would it make it simpler to use the gems directly? <davexunit>mark_weaver: ruby has a utility to easily install them, right now we have to build the gem archive and then install it. <davexunit>a .gem file is equivalent to something like a release tarball you'd get for a piece of GNU software. <mark_weaver>does using the gem mean that we are using some non-source code that was built elsewhere? <davexunit>mark_weaver: that's the open question. it seems that often the answer is "no" <davexunit>also, the 'gitify' phase could go away, which is sort of like not having to use GNU autotools to bootstrap the configure script and Makefile. <mark_weaver>for security reasons, I'd actually prefer to move toward a model where we avoid using any non-source code at all, including things like configure scripts built on developer's machines elsewhere. <mark_weaver>but I suspect that civodul disagrees, so I'll have to talk to him about it and see what he thinks. <davexunit>mark_weaver: some gems contain native extensions, but I know that those are *not* included in the gem. they are built on the user's machine. <davexunit>but perhaps someone could create a gem with some pre-generated non-source Ruby code. <davexunit>and I'm looking for gems that leave out important files that the gem unreproducible <mark_weaver>to be more precise, I'd prefer to use the copy of the source code that most developers are actually looking at on a regular basis, rather than some copy that no one looks at. <mark_weaver>because it would be harder to introduce malware into the code that the most eyes are on. <davexunit>thus the reason I use release tarballs, luckily github auto-generates them from git tags. <davexunit>I just found one example of a missing file: the ruby-i18n gem doesn't include the Rakefile (equivalent to Makefile) <mark_weaver>taking my preference a step further would mean going directly to the source repos and not using release tarballs at all, but again, this is at odds with civodul's preference, so I'll have to discuss with him :) <davexunit>yeah, I've been thinking about why it's considered OK for the configure script, Makefile, docs, etc. to be pre-built. <mark_weaver>it could be okay to use prebuilt stuff like that *IF* we have a system in place for independently verifying that those pre-built things are valid. <mark_weaver>right now, it seems to me that everyone is just taking it on faith. <mark_weaver>but anyway, these comments go quite a bit beyond the question you asked. <mark_weaver>just food for thought; I leave it to your best judgment :) <davexunit>I'm trying to use good judgement, but it's a tricky question. <davexunit>I'm feeling pressure to just use the .gem files, but they *feel* like the equivalent of binaries to me. <mark_weaver>davexunit: I don't see why the 'gitify' phase is a problem, because it's in one place, right? it doesn't add to the work needed for each ruby package added, does it? <davexunit>also, the gem would have to be extracted and re-packed because of source shebang patching and stuff. <davexunit>mark_weaver: it's not a problem, but it's a step that we wouldn't need to take using the pre-built gems. <mark_weaver>yeah, but that work is done by hydra, and it's not a lot of work. so why do they care? <davexunit>hmmm, about the patching, the gnu build system runs a patching phase on the instaled files? <davexunit>mark_weaver: no one mentioned that in particular, that was just a thought of mine. <mark_weaver>what's an example of a specific problem they cited with your current approach? <davexunit>the argument is that the gems include the source code and are the things that the developers are testing to make sure they work. <davexunit>so we shouldn't do things the "hard way" by building our own gems <mark_weaver>we'll have to build our own gems anyway, because of the shebang patching etc. <mark_weaver>it's just a question of where we get the source code from. <mark_weaver>and the source code in the gem *should* be the same as the source code from the tarball, right? if not, that sounds fishy to me. <mark_weaver>well, I could be wrong about some of this, I'm lazily making some assumptions based on what you've said. <mark_weaver>remember also that one of our goals is bit-for-bit reproducable builds, even if we're not quite there yet. are the gems built elsewhere done in a reproducable way? <mark_weaver>and even if they are, if we rely on a gem built elsewhere, then we don't have a mechanism to easily perform that independent verification. <mark_weaver>we are basically taking on faith that the third-party resource used to build the gem isn't compromised. <davexunit>I think the shebang thing is a non-issue, because there's a patch-shebangs phase that operates on the outputs. <mark_weaver>but the source tarball is one step closer to the source code in the repo. <mark_weaver>what about things like dlopen'ing shared libraries, where we'll want to patch those with absolute pathnames. <mark_weaver>well, I've said my piece. do as you think best. I don't really care about ruby anyway, personally. <davexunit>ruby isn't my favorite language, but I find myself spending a lot of time working with it, and I want Guix to facilitate sane Ruby environments. <davexunit>because the current state-of-the-art tools are terrible for managing multiple versions of ruby <mark_weaver>I can see the value in it, even though I don't personally use this stuff. <davexunit>but ruby's been tough to get into shape. it, like most "modern" programming languages, have thought they've improved upon packaging and distribution, but have really made it more difficult. <davexunit>if you aren't using their blessed system, you're sort of screwed. :) <davexunit>because their system masks all sorts of scary problems, like circular dependencies. <davexunit>on another topic, I'd like to avoid the use of 'system' in favor of 'system*' in guix environment. <davexunit>but not sure how to write the CLI to make that work <davexunit>I added something to my bashrc that uses $SHLVL to count how many shells deep I am, and I notice that each 'guix environment' invokation increments it by 2 <alirio>davexunit, mark_weaver: './configure;make;make install' become the standard to install from source (in a time the configure, Makefile, ... are not generated) , not just on gnu but elsewhere. tarballs are made with that in mind. removing the configure, Makefile, ... would be like buying a black car to remove the paint and paint it red because most people don't buy unpainted cars <davexunit>alirio: yes but the tarballs also come with the configure.ac/Makefile.am that those scripts were generated from. <alirio>but the cleaner way is ask for unpainted cars <davexunit>then the git repo or whatever would have that <efraim>how important is the kernel option CONFIG_DEVPTS_MULTIPLE_INSTANCES=y? I can't find it in the menuconfig for my armhf board <alirio>davexunit: in the future I hope everyone would make this the common practice, so we would have demand to 'really source tarballs' <alirio>efraim: try "grep DEVPTS_MULTIPLE_INSTANCES config" <efraim>oooh / brings up the search options <davexunit>mark_weaver: another complication that makes pre-built gems attractive: automatically generated change logs from the git/hg history. <davexunit>my current build system can't recreate those. *alirio wonders about isl code in cloog: "Makefile:796: recipe for target 'libcloog_isl_la-constraints.lo' failed", "./include/cloog/isl/constraintset.h:18:2: error: unknown type name ‘isl_constraint’" <davexunit>civodul: complications in ruby packaging land. not sure whether to continue building gems from tarballs or if we should download them rubygems.org. <alirio>from commit 372cf03981fea2e8f95be88c8a986f996c812d14 and now 7871724df7218428fac53133496c474bac8c5ea8 <davexunit>I've been at this crossroads before, chose tarballs, now things are less clear when I discover things like unreproducible change logs. maybe that's not so important. <civodul>davexunit: yeah, dunno, if it's just change logs that are missing, perhaps that acceptable <davexunit>by downloading the pre-built gems, we may miss things that the devs forget to include like the Rakefile (sort of like a Makefile but for Ruby) <civodul>alirio: interesting, maybe we should add this patch to our gettext in core-updates <davexunit>so we wouldn't be able to unpack the gem and run the test suite, for example. <davexunit>but the big issue to me is that we have no guarantee that the gem contains the complete corresponding source code. <civodul>because the gem contains object code? <davexunit>it doesn't, but it *could* contain non-source Ruby code generated by some program, or it could be missing things that aren't essential for runtime like the Rakefile, test suite, readme, etc. <alirio>civodul: actually I'm updating to amd64, so I will not work on that ATM, but I can consistently reproduce it in my machine. I wonder if it wouldn't be easier to get the git version instead of a patch <civodul>davexunit: the situation sounds messy ;-) but i'm afraid i don't sufficiently understand the details <civodul>alirio: you can reproduce the gettext issue, or the cloog issue? <davexunit>I've got some pressure on me to use the gems from rubygems.org <civodul>alirio: so what's the cloog issue actually? :-) <davexunit>bleh, I just need to walk away from this for awhile. <alirio>civodul: I guess cloog's isl and cloog itself doesn't match, but it's weird hydra don't catch that <civodul>alirio: so "guix build cloog" reproduces the issue? x86_64 or i686? <rekado>davexunit: no pressure. Not using the gems doesn't sound so bad to me. <alirio>x86_64, will try "./pre-inst-env guix build cloog" <rekado>Gems are usually generated from gemspec files and we should be able to build the very same gems from the repository. <davexunit>if you notice the current ruby gem packages + the 11 patches I have waiting on the mailing list ;) *davexunit goes off to the movies <civodul>alirio: could it be a parallel-make issue? try with "guix build cloog -c1" <alirio>civodul: fails one file earlier, same error *alirio wishes to "./pre-inst-env guix build cloog isl --keep-all" <civodul>alirio: i need to test but i'm currently on another branch <civodul>could you email bug-guix with all the info? <alirio>sure, all the info means output of "./pre-inst-env guix build cloog -c1" and a description or something more? <civodul>the architecture, commit id, and the last ~20 lines of log would do <civodul>and anything that sounds important to you :-)