IRC channel logs


back to list of logs

<civodul>hey there!
<civodul>today my goal is to do a number of boring things, and that includes the Guix-HPC annual report
<rekado_>I can’t find motivation to do anything.
<civodul>alright, i was lacking motivation but i managed to write several things!
<rekado_>I’m working on reusing scripts in the GWL by delaying the look-up of process variables
<rekado_>to make use of this, local definitions in the “procedure” field really should be moved to a field of their own, so that they can be separated from the payload and their lookup be delayed
<rekado_>I’m struggling to find an appropriate name for this new field
<rekado_>Snakemake uses “params”, but I don’t want to use abbreviations
<rekado_>another question is whether to make the recommended case easier to type
<rekado_>currently, a {{placeholder}} in the payload will be replaced with the symbol “placeholder”, a variable name, no matter where it is defined.
<rekado_>I wonder if I should instead only look up placeholders in the new local definitions field and only look up global variables when they are prefixed
<rekado_>so {{foo}} will be a look up of the “foo” value in the “parameters” field, while {{global:foo}} would be the global variable “foo”
<rekado_>perhaps this is too messy
<rekado_>it would also clash with the existing keyword “indexing” we have for the inputs and outputs fields.
<civodul>looks similar to packages in Guix: you'd want some sort of a inspectable lambda
<civodul>for packages, 'inputs' sorta plays the role of the formal parameter list of a lambda
<rekado_>I named the field “values” for now and started rewriting pigx-rnaseq.w; “values” accepts a list of keyword-tagged values (for consistency with “inputs” and “outputs”), but …
<rekado_>this is a little awkward
<rekado_>for one, “values” must be a list with named items. This means that any procedures that are used to compute any of the values must be defined before it.
<rekado_>this means that procedures cannot use previously defined “values”
<rekado_>I’d love to have a “real” environment with definition context, but I can’t think of a way to serialize it
<rekado_>let-wrapping the final keyword-tagged list is possible, but it’s still awkward because values would need to be repeated (once for the definition and then later again in the tagged list)
<civodul>rekado_: d'you have a pointer to an example of what it currently looks like?
<rekado_>here’s an example of what it looks like before the introduction of the “values” field in the “process” record:
<rekado_>there are local definitions under “procedure”
<rekado_>what’s bad about this is that the value for “tmp” gets baked into the script
<rekado_>a better example with procedure definition:
<rekado_>“tmp1” and “tmp2” depend on the process argument “sample”, so there will be one generated script per sample instead of one script that gets arguments once per value of “sample”
<rekado_>this makes me think: maybe it’s enough to delay evaluation of the “procedure” field and include it wholesale in the generated script; pass it the process argument as the script argument.
<rekado_>(oh, I just realize that I pasted the whole file…)
<civodul>so "procedure" is code that is staged, right?
<civodul>it can be any language IIUC
<rekado_>it’s not actually staged, but it seems obvious to me now that it should be!
<civodul>but isn't #{ ... } like staging? like quote?
<civodul>it's funny to read wisp code :-)
<rekado_>the # { … } is read syntax that returns a (code-snippet …) record that contains a list of string tokens and variables
<civodul>ah and that's where you do string interpolation, right?
<civodul>with the {{braces}}
<civodul>well that's a form of staging anyway
<rekado_> # /bin/bash { echo "hello {{who}}" } –> (code-snippet '/bin/bash '() (list "hello " who ""))
<civodul>but "procedure" also contains code executed right away, right?
<civodul>got it
<rekado_>and I think I need to delay that
<civodul>yes, maybe
<rekado_>because otherwise the returned code-snippet will always differ, even when produced from the same process template (which really is just a procedure returning a <process>)
<rekado_>and that means: different job scripts instead of shared job script
<civodul>we have the exact same problem with derivation build scripts!
<civodul>the context/tradeoffs are probably different though
<rekado_>for derivation build scripts some parameterization is done via environment variables, like the output location, for example.
<rekado_>my problem in delaying execution is that I can’t serialize a closure
<civodul>only the output, and that's because it's the only way to do it
<civodul>you can't serialize a closure, but you can serialize an sexp
<civodul>so you'd have to make the whole thing quoted maybe
<rekado_>when it’s all quoted I can’t capture global variables, though
<rekado_>well, I could unquote, I guess
<civodul>i'm saying this as a passerby though
<rekado_>it’s helpful :)
<civodul>"free advice: you get what you pay for"