IRC channel logs
2021-01-29.log
back to list of logs
<civodul>today my goal is to do a number of boring things, and that includes the Guix-HPC annual report <rekado_>I can’t find motivation to do anything. <civodul>alright, i was lacking motivation but i managed to write several things! <rekado_>I’m working on reusing scripts in the GWL by delaying the look-up of process variables <rekado_>to make use of this, local definitions in the “procedure” field really should be moved to a field of their own, so that they can be separated from the payload and their lookup be delayed <rekado_>I’m struggling to find an appropriate name for this new field <rekado_>Snakemake uses “params”, but I don’t want to use abbreviations <rekado_>another question is whether to make the recommended case easier to type <rekado_>currently, a {{placeholder}} in the payload will be replaced with the symbol “placeholder”, a variable name, no matter where it is defined. <rekado_>I wonder if I should instead only look up placeholders in the new local definitions field and only look up global variables when they are prefixed <rekado_>so {{foo}} will be a look up of the “foo” value in the “parameters” field, while {{global:foo}} would be the global variable “foo” <rekado_>it would also clash with the existing keyword “indexing” we have for the inputs and outputs fields. <civodul>looks similar to packages in Guix: you'd want some sort of a inspectable lambda <civodul>for packages, 'inputs' sorta plays the role of the formal parameter list of a lambda <rekado_>I named the field “values” for now and started rewriting pigx-rnaseq.w; “values” accepts a list of keyword-tagged values (for consistency with “inputs” and “outputs”), but … <rekado_>for one, “values” must be a list with named items. This means that any procedures that are used to compute any of the values must be defined before it. <rekado_>this means that procedures cannot use previously defined “values” <rekado_>I’d love to have a “real” environment with definition context, but I can’t think of a way to serialize it <rekado_>let-wrapping the final keyword-tagged list is possible, but it’s still awkward because values would need to be repeated (once for the definition and then later again in the tagged list) <civodul>rekado_: d'you have a pointer to an example of what it currently looks like? <rekado_>there are local definitions under “procedure” <rekado_>what’s bad about this is that the value for “tmp” gets baked into the script <rekado_>“tmp1” and “tmp2” depend on the process argument “sample”, so there will be one generated script per sample instead of one script that gets arguments once per value of “sample” <rekado_>this makes me think: maybe it’s enough to delay evaluation of the “procedure” field and include it wholesale in the generated script; pass it the process argument as the script argument. <rekado_>(oh, I just realize that I pasted the whole file…) <civodul>so "procedure" is code that is staged, right? <rekado_>it’s not actually staged, but it seems obvious to me now that it should be! <civodul>but isn't #{ ... } like staging? like quote? <rekado_>the # { … } is read syntax that returns a (code-snippet …) record that contains a list of string tokens and variables <civodul>ah and that's where you do string interpolation, right? <civodul>well that's a form of staging anyway <rekado_> # /bin/bash { echo "hello {{who}}" } –> (code-snippet '/bin/bash '() (list "hello " who "")) <civodul>but "procedure" also contains code executed right away, right? <rekado_>because otherwise the returned code-snippet will always differ, even when produced from the same process template (which really is just a procedure returning a <process>) <rekado_>and that means: different job scripts instead of shared job script <civodul>we have the exact same problem with derivation build scripts! <civodul>the context/tradeoffs are probably different though <rekado_>for derivation build scripts some parameterization is done via environment variables, like the output location, for example. <rekado_>my problem in delaying execution is that I can’t serialize a closure <civodul>only the output, and that's because it's the only way to do it <civodul>you can't serialize a closure, but you can serialize an sexp <civodul>so you'd have to make the whole thing quoted maybe <rekado_>when it’s all quoted I can’t capture global variables, though <civodul>i'm saying this as a passerby though <civodul>"free advice: you get what you pay for"