IRC channel logs

<civodul>any more comments on https://gitlab.inria.fr/guix-hpc/website/-/blob/master/drafts/continuous-integration.md ?

<civodul>i'll publish it this afternoon

<rekado>I read it, but I had a hard time empathizing with the problems proposed in the introduction.

<rekado>things like “will MPI use the high-speed interconnect” – is this something that’s actually addressed by continuous integration / delivery?

<rekado>we can know in advance what software will run on the cluster, but there seems to be a lot of runtime variability that’s platform dependent.

<rekado>building software and running its test suite (if available) on a separate system with different specs does not seem to increase confidence in how the software would behave at runtime, given the modular nature of MPI and scheduling systems

<rekado>I guess I’m confused about how the introduction fits with the rest of the article

<rekado>(I could very well be missing something, other than sleep)

<civodul>rekado: ah, interesting!

<civodul>so, i wanted to suggest that these problems are used to justify lack of CI/CD

<civodul>hence people build software locally and validate things manually

<rekado>isn’t the validation step still necessary, though? Because the target system may elicit have unusual runtime behavior?

<civodul>it is, but it doesn't have to be as thorough as when you build it all by hand

<rekado>true

<rekado>I remember fiddling with one of the set of packages that are known under the name “dune”, and while building the software worked fine it didn’t seem to use the desired interconnect

<rekado>we went through a couple of rebuild+test cycles

<civodul>this part is hopefully debunked by https://hpc.guix.info/blog/2019/12/optimized-and-portable-open-mpi-packaging/

<rekado>I will say, though, that this was a lot more comfortable than doing this by hand

<rekado>another colleague — in a misplaced bout of competitive behavior — attempted to rebuild with different options in the traditional haphazard way and got lost in a forest of state and environment variables.

<civodul>:-)

<rekado>I guess I’m just lacking experience with HPC culture

<civodul>maybe https://paste.debian.net/1273111/

<rekado>sounds like an improvement, yes

<civodul>cool

<civodul>the Spack talk i mentioned says a lot about this

<rekado>there’s real value in building software in predictable environments, and since performance is no longer a good argument for not building binaries elsewhere I guess HPC people need to be told that their systems are not special enough to warrant pouring out the reproducible deployment baby with the compilation bath water.

<civodul>exactly, that's the message i'd like to have

<rekado>good good

<civodul>maybe that's not clear/explicit enough though?

<rekado>I don’t know why, but that’s not what I got out of the introduction initially

<rekado>as I read on my thoughts were: “huh, cuirass? How does that help with MPI…?”

<rekado>like the introduction primed me to focus on something that the blog post didn’t deliver on

<civodul>ok, maybe the MPI question in the 1st paragraph is off-putting

<rekado>could be

<civodul>maybe "Will it perform well?" would work?

<rekado>my confusion grew when it became clear that it was not actually about cuirass

<civodul>as a reference to performance portability

<civodul>ok

<rekado>I think the second paragraph of the introduction might benefit from some expansion

<civodul>right

<rekado>it’s all there, really, but I still — embarrassingly — missed the point

<rekado>“HPC doesn’t always benefit from … CI/CD” — I think that was the first point of confusion

<rekado>there are two ways to understand this sentence: 1) there is little to no benefit to CI/CD in the context of HPC; or 2) in practice HPC people do not use CI/CD with their HPC systems

<rekado>to me there’s also a potential confusion about who is addressed. HPC admins, users of scientific software, or research engineers targeting HPC users+systems

<civodul>ACTION nods

<rekado>at the MDC we only really have the first two classes

<rekado>HPC admins care about little more than the scheduler and the hardware itself; users on the other hand are removed enough from software development to have any opinions one way or another about continuous integration (beyond the question “will it *install*?”)

<civodul>well, it depends on the kind of users i guess

<civodul>folks doing linear algebra care a lot about that

<civodul>an update: https://paste.debian.net/1273115/

<rekado>yes! “take advantage of CI” is much clearer!

<rekado>“as much as touted in other fields” probably needs to change too

<civodul>or be removed?

<rekado>yes, perhaps

<rekado>the updated intro is good

<civodul>coolio, thanks for pinpointing its failures

<civodul>i guess i'll go ahead with this version

<rekado>sounds good!

<rekado>thanks for humoring me :)