IRC channel logs


back to list of logs

<efraim>that's amazing!
<PurpleSym>Phew, clinfo works with a Guix-built now. Do we have anything in place to tell the ICD loader where to find it? It seems it is looking at my system’s /etc/OpenCL directory.
<civodul>PurpleSym: i think it's needed supposed to check /etc/OpenCL, no?
<PurpleSym>Yeah, but thankfully there’s a search path if you install ocl-icd :)
<civodul>ah, good
<civodul>BTW, i got my first successful pytorch (CPU) build yesterday
<civodul>now trying tests...
<PurpleSym>Any chance of getting ROCm into Guix proper? It’s entirely open source, but – well – it needs firmware blobs in the kernel. So probably nonguix territory?
<civodul>i suppose the userland bit (the part that's free software) could go in Guix proper
<civodul>proprietary firmware is needed for AMD GPUs?
<PurpleSym>Yes, their GPU’s don’t do anything useful without firmware nowadays.
<civodul>bah, ok
<PurpleSym>OpenCL being one of those things.
<rekado>PurpleSym: are you aware of Katherine’s blog post on OpenCL with Guix?
<PurpleSym>No. I’ll have a look. Thanks!
<civodul>oh i had forgotten about this one
<civodul>so "pip install torch" downloads an 830MB wheel that contains things like and (!)
<civodul>i think we should take a step back and raise awareness of these issues
<civodul>cuz i/we keep preaching to the choir but the vast majority of users probably doesn't understand why we'd bother with Guix packaging when "pip install" works "so well", even for non-Python packages like this one
<rekado>pushed tensorflow-lite
<rekado>civodul: this is terrible
<rekado>it’s a real challenge to communicate. What could be wrong about this when it evidently works?
<rekado>it’s the same barrier that needs to be broken when communicating what functional package management brings to the table
<rekado>the first brick falls when people admit that conda/docker/whatever may not be as “elegant” as Guix, but it “does get the job done”
<rekado>but I feel that to many users this reframing as “elegance” makes Guix seem “academical” in the worst sense.
<civodul>rekado: yeah, it's not really helping
<civodul>it's like "you folks are doing nice work, thumbs up; in the meantime, i'll get the job done with that other tool"
<rekado>and this is the exact response I got from some users at the MDC
<civodul>perhaps it's a mistake of ours to communicate on improvements to what we already have, when the basic motivation is not well understood outside of our circles
<efraim>perhaps a more in-depth example of using transformations and manifests? I was thinking of writing something up.
<efraim>currently I'm replacing sdl2 with a later version and grafting openssl compiled with -march=native for the marginal boost
<rekado>“you are doing nice work, thumbs up; in the meantime, I’ll get the job done with LD_LIBRARY_PATH.”
<civodul>efraim: that'd make for a great blog post! i feel we need more hands-on posts, like "how i did X"
<civodul>rekado: heh :-)
<civodul>is "pip install" actually supposed to work in Guix?
<civodul>it downloads stuff and eventually fails with "Read-only file system", apparently while trying to write to /gnu/store/...-python
<civodul>i'm told one can disable pre-built binaries for pip:
<efraim>probably need --user for pip
<efraim>so... `guix environment pytorch -- pip install --user pytorch --no-binary all` or something?
<PurpleSym>Or in a virtual environment. I do that quite often with editable projects.
<civodul>ah yes, --user does the trick
<civodul>tested in: guix environment -CNP --ad-hoc python-wrapper nss-certs openssl coreutils grep sed findutils
<civodul>so, isn't pip stepping on CONDA's toes?
<wehlutyk[m]>hello all
<wehlutyk[m]>I was following the conversation from afar and thought I can add something to the communication challenge mentioned earlier
<wehlutyk[m]>you're probably aware of this, but the incentive system in academia is also working against people being able to see this problem. A PhD is 3 years, with evaluations every year. Most postdocs are 1 or 2 years, sometimes down to 6 or 9 months.
<wehlutyk[m]>The pressure for stable positions after postdocs is huge and growing. So it's very difficult for people to spend time making sure their code will reproduce 10 years down the line.
<wehlutyk[m]>This is probably even worse in deep learning where the shelf life of a new network architecture is less than 2 years
<civodul>wehlutyk[m]: agreed!
<civodul>i think pushing scientific reproducibility, and thus reproducible deployment, also means inevitably bringing such issues to the table
<wehlutyk[m]>On the other hand, my experience is that Guix is an easy sell for HPC users outside computer science and physics, as larger teams need to install a lot field-specific software, and coordinate people's environments (at least this is what I'm seeing now in neuroscience -- the only challenge still being that I haven't found the time to get a shared daemon to run as non root)
<wehlutyk[m]>civodul: yes
<civodul>first time i hear it's an "easy sell" :-), but yes, it should be possible to make a point that Guix helps with these things
<wehlutyk[m]>maybe some publishers or funders could change that pressure, although I don't see anybody forcing people to use a specific package manager :)
<civodul>oh sure
<civodul>i think we need to raise awareness of the issues before we can discuss specific tools
<civodul>pytorch's in the house! \o/
<civodul>rekado: we have xnnpack now, which i think tensorflow-lite could depend on
<civodul>(not sure what it brings exactly, but surely something)
<drakonis>the pitch could use some mild touching up, convincing people to try it after getting too comfortable with nix has been an uphill battle
<drakonis>explaining to them why they should use it over nix
<drakonis>most of the time the appeal to them has been nix's package availability
<drakonis>it has everything
<rekado>civodul: woo, excellent job packaging pytorch!
<rekado>civodul: tensorflow-lite can use xnnpack, but it insists on building it from source. I wanted to patch the build system later to include xnnpack.
<rekado>I don’t think it’s that important to convert nix users to Guix.
<rekado>drakonis: I’m pretty sure that the R packages in Nix are not actually all usable.
<rekado>their quantity over quality approach is frustrating to me, because it means that I need to do all the hard work instead of being able to just copy :)
<rekado>also: their tensorflow-lite package only builds the .a lib, which is not great.
<drakonis>oh they arent
<drakonis>its mostly functional
<drakonis>anything that is pure R works fine
<rekado>pure R is okay
<rekado>as soon as you need a system library it gets bad
<rekado>and you know… in that case you might as well use install.packages, devtools, ratpack, or the many other pure R package managers.
*rekado –> dinner
<drakonis>that's where everything falls apart
<drakonis>what frustrates me about nix is that besides the obvious quantity over quality problem, is that it is really inconsistent
<drakonis>it has a dozen ways to do the same thing, each one with specific edge cases and focus
<drakonis>there istn consistent interfaces
<drakonis>also i was pulled into a conversion with a haskeller interested in guix because he got burnt by nixpkgs maintainers being really irresponsible with changes
<drakonis>one of the things he pointed out was that there's a combinatorial explosion in nixpkgs
<drakonis>there is a set of packages duplicated for each different ghc version
<drakonis>nixpkgs isnt curated enough
<civodul>drakonis: i guess some would argue that having one package set per ghc is a feature
<civodul>though i must say i'm not convinced
<drakonis>they sure would
<civodul>back when i contributed there wasn't much of an effort to keep things consistent (there was no peer review actually)
<civodul>but things have improved since then, it seems
<drakonis>mostly improved
<drakonis>nixpkgs is still being blocked for days though
<drakonis>reviews aren't done equally
<civodul>ah well, i can sympathize :-)
<civodul>i mean, scaling up peer review is hard
<drakonis>it's understandable, in a way
<drakonis>they have bots for checking if a patch builds
<drakonis>necessary when they get hundreds of patches a week
<drakonis>it's impressive how they haven't dropped below 4 digits of issues and PRs
<drakonis>it's been years since it got there
<dstolfa>civodul: does guix have a way to say "please build all of the packages that depend on this package"? if so, it might be possible to scale up review in a way that a buildbot can just build everything in CI and give automated feedback, at least for some things.
<drakonis>it would be interesting to have a branch that has automatically bumped packages
<drakonis>there's a lot of trivial bumps
<dstolfa>drakonis: these are also many bumps that look trivial, but aren't
<dstolfa>e.g. recent fallout with sbcl
<dstolfa>2 line change, it builds, great! except like thousands of packages fell over as a result
<drakonis>a substantial amount of PRs in nixpkgs are that
<drakonis>i see
<drakonis>fair enough
<civodul>dstolfa: yes, there's "guix refresh -l", and cbaines has been working for some time on integrating it:
<civodul>here you can see patches from guix-patches and red/green buttons telling whether they apply, build, etc.
<drakonis>regarding sets of ghc packages, it should be trivial to write a function to switch the ghc version for the packages, right?
<dstolfa>civodul: cool, so this is basically building all dependents of the package too?
<dstolfa>drakonis: guix already has that for a number of languages
<civodul>dstolfa: ah that i'm not sure actually, but it could
<civodul>could end up being costly though
<dstolfa>yep, but the build farm is very big... :)
<dstolfa>and would prevent fallouts of untested things
<drakonis>dstolfa: i mean invoking map to apply the package to a set
<drakonis>i see a certain appeal in mentioning that
<drakonis>so it uses a different compiler version
<dstolfa>drakonis: you mean like this?
<dstolfa>(this is for rust)
<drakonis>one sec, network is janky
<drakonis>will be home soonish
<drakonis>sheesh this is taking a while to load
<drakonis>afaik, all build systems let you pick which package you want to use for the build, but there isn't any written example on how to get a group of packages to build using a different version
<civodul> shows how to rewrite the dependency graph for example
<drakonis>that works for me, but they might not understand how that correlates to this
<drakonis>by the way, why isn't isnt the manual pointing to the latest commit by default?
<drakonis>people tend to read the manual for 1.3.0 and not the current one
<drakonis>i've had an occurrence of someone checking the manual and looking for a service that's already included in the latest revision
<civodul>drakonis: gives the links to both the "stable" and latest versions
<drakonis>i'm aware, but it's not about me here
<drakonis>i always check the latest
<civodul>it'd be nice to have version links in a corner of the manual web pages
<civodul>we could try that
<drakonis>it's more in the interest of not having to tell people that they're looking at the manual for the stable version and that it is out of sync with the latest one
<drakonis>since search engines like google point to the stable manual by default
<drakonis>civodul: thanks for your work with kicking off guix btw
<drakonis>regarding retiring as a co-maintainer
<civodul>yw :-)
<drakonis>dstolfa: that is indeed what i want, but applying that to a group with packages, map would probably do the trick?
<dstolfa>drakonis: is there a reason it can't be explicit in the packages themselves? it would be good for it to be explicit, if only for people looking at it later to be able to see that it depends on a particular compiler version
<drakonis>i guess
<drakonis>it'd create package duplication though
<dstolfa>huh. why do you need to duplicate the packages?
<dstolfa>i'm not sure i fully understand your problem
<drakonis>okay so, nixpkgs has a series of sets of packages with different compiler versions
<dstolfa>okay, why is that needed?
<drakonis>its not really needed, but there's people that don't really understand how to set the compiler version for each package for their own local consumption
<dstolfa>drakonis: i'm not sure i understand why this is necessary in the first place, what's wrong with using the default?
<dstolfa>(a tested default, mind you)
<drakonis>its because some people are uhhh
<drakonis>how do i put this, rather concerned with how they don't get to pick which compiler version they want to use for the software
<drakonis>it doesnt really have an impact
<dstolfa>and is there a reason they are rather concerned, or is this just complaining for the sake of complaining? i'm asking because there might be a way to achieve what they want without any of this
<drakonis>unless they want to test behavior against different compiler versions
<drakonis>afaik its this ^
<drakonis>there is always a way to achieve what they want without making it overly complex
<dstolfa>drakonis: uh, the simplest way to do this i guess is to just change the default compiler version
<dstolfa>see gnu/packages/rust.scm, the very last line
<drakonis>okay i almost forgot to expand it
<drakonis>i mean doing things like getting a group of packages and defining them with a different compiler i guess
<drakonis>programatically so they dont need to write a shitload of code i guess?
<drakonis>i'm guessing a lot here arent i
<dstolfa>if they want to do it for everything, just set the default compiler to something else build everything locally i guess
<dstolfa>use a custom guix channel and that's it
<dstolfa>though i really don't see the point
<dstolfa>it's doable, it's just not really sensible IMO
<drakonis>sensible's the name of the game
<dstolfa>i understand some people need to change to compiler for one reason or another, but if the reason is "i wanna use a different compiler", then it's probably best to just change the default and build everything locally *shrug*
<dstolfa>of course you can map over it since it's just guile records but i feel like it's not the prettiest code
<drakonis>hardly, but its there i suppose
<drakonis>the option exists
<drakonis>technically this is the same kind of hackery that happens in nix afaik
<dstolfa>they'd probably need to check that the builds they're doing are reproducible and compose well with the rest of guix... there's no saying what changing a default compiler might do (or building some with 1 and others with another without testing)
<drakonis>i don't think that'd be much of a consideration right now
<drakonis>for this person
<drakonis>god, nixpkgs makes no sense
<drakonis>its a never ending cocktail of hacks
<drakonis>i was checking how it defines the packagesets and woah this stuff is just ugly