IRC channel logs


back to list of logs

<rekado>zimoun: hello!
<rekado>I’ve been thinking about R and propagated inputs
<rekado>I’ve been stumbling upon a problem repeatedly: guix install r-something; time passes; guix pull; guix install r-whatever —> conflict
<rekado>this cannot be worked around with “guix pull && guix upgrade r-”
<rekado>because for some reason “guix upgrade” won’t upgrade package *variants*
<rekado>so colleagues who follow my advice of upgrading all R packages before installing new ones are confused
<rekado>and so am I
<rekado>the latest workaround is to use a manifest
<rekado>guix package --export-manifest > /tmp/m; guix package -m /tmp/m; guix install r-new
<rekado>this is clumsy, though
<civodul>rekado: i and others came to the conclusion that we should recommend 'guix shell' as the main tool for researchers
<rekado>yeah, guix shell is the best
<civodul>this is what Konrad suggested in
<rekado>it’s predictably, declarative, fast, etc
<civodul>(last slide)
<rekado>I don’t know why it’s not used more often here, though
<civodul>not that it helps you with your immediate problem though :-)
<rekado>it might be a mental block
<rekado>the name “guix install” suggests permanence and stability
<civodul>until recently, i/we would advertise "guix install" as the way to get started
<civodul>because it looks familiar
<rekado>for things like guix.install() (in R) I don’t think I can use “guix shell”
<rekado>I’ll probably switch to building and installing manifests under the hood, though
<rekado>I wonder if we can ever do without propagation in R
<zimoun>rekado: here my advice is to only use manifest.scm (and channels.scm).
<zimoun>I just ban “guix install”, well it is just for a quick test but the only way to control the environment is manifest.scm + channels.scm.
<zimoun>Moreover, for newcomers I simply mention “guix shell”.
<rekado>still, I feel icky about propagated inputs
<zimoun>Then, depending on the skill of the users, I explain profiles.
<rekado>I would like us to avoid the problem in the first place
<rekado>the cause of the conflict error is that we refuse to install variants of the same package into the profile directory
<rekado>that’s fair
<rekado>but we only attempt to install variants of the same package because of propagation
<rekado>and we use propagation because simply adding /gnu/store directories onto the load path wouldn’t help as the order of discovery is linear and not scoped per package
<rekado>the load path is a global variable, and the namespace registry mapping files to names is global, too
<rekado>I wonder if we can scope it
<zimoun>Yeah, I agree about propagated inputs. I remmber reading something on guix-devel or help-guix about a similar issue.
<rekado>(and I wonder the same for Guile)
<civodul>in pretty much all languages but Node.js, there can be only one module with a given name
<civodul>i'm not sure scoping would help if the language is not prepared for it
<civodul>what would help is a mechanism à la RUNPATH
<rekado>in R you can load namespaces but not attach them
<zimoun>IMHO, we have a kind of dilema between “imperative” which requires a solver to avoid conflict and “declarative” which implies the manifest dance (no guix upgrade)
<rekado>and some packages then use the equivalent of Guile’s (@ (other namespace) foo)
<rekado>so there’s the assumption that namespaces have unique (and predictable) names
<zimoun>Well, I do not find back the message.
<zimoun>Hum, I am missing how propagated inputs is avoidable for R. But it is possible, for sure it will be better.
<zimoun>However, for the general case, the only way to avoid some conflicts of propagated inputs when using “guix upgrade” would be to add a version resolver… which would defeat the main message about reproducible computational environment.
<rekado>I don’t think it’s possible to avoid with R.
<rekado>I just read the code in src/library/base/R/library.R
<rekado>there’s a clear assumption that there’s a global mapping from plain names to objects; when a namespace has been attached it will be loaded
<rekado>when a namespace has been loaded it won’t be loaded again
<rekado>we would have to patch R significantly to work around this, and I’d rather not attempt it
<rekado>R packages would have to have their own namespace mapping (falling back to a parent environment); while the mechanism for nested environments exists it is not used for namespace discovery
<rekado>the machinery is also distributed between different procedures, each of which would have to be aware of the modified discovery mechanism
<zimoun>well, heavily patching R would lead to more work for maintenance
<rekado>taking a step back: could we just create a really long load path with absolute /gnu/store directory names instead of propagating things?
<rekado>let R deal with conflicts
<rekado>from R’s perspective it is totally fine to have multiple conflicting entries on the load path
<rekado>as long as nothing from conflicting locations is attached and loaded R won’t mind
<rekado>instead of having a package propagate its inputs it would merely record the locations of its inputs
<rekado>a profile hook would then assemble the R_LIBS_SITE value by concatenating and uniquifying the recorded locations for all leaf packages
<rekado>we would shift the problems (due to bad user habits) from Guix to R
<rekado>in other words: let R be the bad guy
<zimoun>is there no conflict when uniquifying?
<civodul>ACTION just had a tech discussion on the maintenance of Guix on
<zimoun>civodul: oh cool! And so, conclusion? Guix is not enough staffed? ;-)
<civodul>heh :-)
<civodul>it was mostly a recap about what needs to be done when updating the daemon, whether/where to run 'guix publish', etc.
<civodul>currently there are ~8 sites, each of which runs 'guix publish'
<civodul>and so there's like 10 URLs behind '--substitute-urls'
<civodul>which in hindsight is a bit overkill :-)
<civodul>that's an interesting setup because that's a "cluster of clusters"
<zimoun>wait, they are serving substitutes. Is it not public?
<civodul>they're serving them on the "local" network, within Grid5000
<civodul>i don't think it's very useful though
<zimoun>well, it is useful for communicating, at least. :-)
<zimoun>I mean, somehow a kind of call. ;-)
<civodul>sure it can serve as an example
<rekado>zimoun: when each package installs its own search path, and the search path is made up of absolute file names then there should not be a conflict in generating the effective search path
<rekado>zimoun: my only worry is that the search path could get very long, and I don’t know if R is okay with that
<zimoun>rekado: hum, I do not know. Well, do you think it is worth to have a workaround for bad user habits?
<rekado>it’s not Guix’s job to police users
<rekado>there are two benefits: we get rid of propagation and we don’t fail until the user actually does something bad
<rekado>propagation clutters manifests, so getting rid of it is a benefit in and of itself.
<rekado>bah, this is disgusting:
<rekado>r-doubletcollection from Guix ends up trying to install some python thing via conda
<rekado>I find code like this to be incredibly disrespectful
<rekado>this screwed up a user’s conda installation, and since conda doesn’t have roll-backs it’s messy to restore the environment
<zimoun>It is not about police users. It is to set “something bad” with more stringency. ;-)
<zimoun>Yeah… « reticulate::py_install("scrublet",pip = T) » is… ouch!
<zimoun>ACTION has to go, see you.
<rekado>sorry, “police” is too strong
<rekado>to me it’s an unnecessary restriction brought about by the fact that we use propagation