<rekado_>zimoun: I’m currently attempting to translate pigx-rnaseq from Snakemake to GWL <rekado_>it’s a little difficult to turn a workflow with file-based targets into processes <rekado_>Snakemake (like GNU Make) encourages this kind of backward connection of the graph <rekado_>when we wrote PiGx we realized that we would actually like to have named tasks that don’t correspond to file names. <rekado_>the result was a hacky association list from ‘target’ names to files that we expect to be produced <rekado_>much of the work in translating this to the GWL is to turn the workflow inside out <zimoun>yeah, I had hard time to explain to my colleagues this :-) That file-based is limitation… And because Snakemake is Python, it is possible to turn into function but it becomes a mess <rekado_>one thing that has become very clear is that Snakemake falls apart as soon as you feel the need for procedures <zimoun>let me know if it is possible to help by sharing the workload or anything <rekado_>Python restricts lambdas to one line, so even simple inline things quickly turn ugly <rekado_>zimoun: I may need to pick your brain later <rekado_>I don’t know if any of this looks easy to people who are not me. <zimoun>exactly! Which appears naturally for complex workflow, for example doing different alignment at different step: which naturally fits as a function with parameters. <zimoun>well, the recent updates you did appear easy to me, at least. ;-) <rekado_>Snakemake has a ‘log’ field for its rules, but I don’t know if they are treated specially <rekado_>I moved that as a named output to ‘outputs’ <rekado_>but I wonder if there should instead be generalized handling of I/O streams <rekado_>another thing that bothers me a little is local definitions <rekado_>the syntax for ‘let’ in Wisp is awkward, so I made the ‘procedure’ field introduce a definition context <rekado_>it’s possible to define values at the top of the ‘procedure’ field <rekado_>but this also means that people have to write ‘procedure’ all the time just to be able to define local values. <rekado_>this makes the definition of the process ‘payload’ a little uglier <rekado_>I added a definition of ‘get’, which is useful for safely accessing nested alists <rekado_>(e.g. for parsed configuration files) <rekado_>so ‘get config "counting" "counting_mode"’ is like config['counting']['counting_mode'] but with error checking <zimoun>about ’procedure’, reading your file, it does not appear to me akward <zimoun>I find clear that ’log’ is in ’outputs’ <rekado_>near the bottom we have report1, report2, and report3 — all of these can be merged with just one extra argument to the process <rekado_>I wanted to translate first and simplify later <rekado_>another thing that’s up for debate is the handlebars style of variable interpolation in code snippets <rekado_>it’s ironic how people can put up with this while also complaining about parentheses in S-expr syntax :) ***rekado_ is now known as rekado
<zimoun>the file looks clear to me. Except the usual WISP akwardness. For example, genemeCoverage, the dots and string-appends or the outputs where the dots and indentation: subtile :-) <rekado>yeah, I’m not entirely happy with wisp. <zimoun>instead of {{foo}}, what do you suggest? <rekado>but I was wondering if it would be a good idea to implement some sort of unquoting <rekado>the {{placeholders}} can only take on a previously defined value <rekado>with : and :: they can take on one or muliple named values from lists <rekado>e.g. {{inputs:gtf}} or {{inputs::reads}} <rekado>BTW, this idea also came about because I wanted to avoid the awkward wisp let <rekado>so maybe *that* is something that should be addressed instead <rekado>maybe by offering a clojure-like let <civodul>(not quite what you had in mind, but now every time i read "Clojure" i shout "Lokke") <zimoun>personally, reading {{foo:bar}} is fine with fine. The dot and indentation wisp is a bit less clear. I mean, I am mentally translating to Scheme with parenthensis to understand. Maybe not enough habits, I do not know. <rekado>in wisp land you need to start a line with a dot when the first thing on the line is not a procedure (or syntax) <zimoun>civodul: I miss the Clojure / Lokke reference )-: <rekado>zimoun: lokke is Clojure for Guile <zimoun>not always, string-append NEWLINE “foo” “bar” should work in WISP <zimoun>civodul: I have not be selected for the training on next Monday. I will be in Group 2. I hope you are prepared because I will have questions about Guix :-p <rekado>zimoun: can you point me to a line where this is not the case? <rekado>in wisp it must be: string-append\n . "foo" "bar" <rekado>because otherwise "foo" would be applied to "bar" and the result is fed to “string-append” <zimoun>in you file, process translate_sample_sheet_for_report with cwd, trim_galore_pe drop-right\n string-split\n 2, just below without-exension read2\n “_val…” <zimoun>process salmon_quant in procedure define arguments, string-append\n “-r” <rekado>you can think of the dot as “identity” <rekado>looks like I’m missing a few dots … <zimoun>well, that’s the problem with WISP, IMHO. <rekado>isn’t it known ahead of time that we cannot possibly apply a string literal to anything? <rekado>maybe there’s something we can do about this… <rekado>either by making strings, numbers, and keywords applicable, or by being smart somewhere earlier. <rekado>it doesn’t collide with srfi 105 <rekado>I’ll talk to Arne about the leading dot. I’m sure he’s thought about this for a while. <rekado>zimoun: there may be a way around this similarly to how commit 7e099dd2428dbe21bcb1530e9bc49e3c4514cb99 made code snippets self-applicable <zimoun>civodul: is guix-hpc still an alive alias? Is this guix-hpc@gnu.org the right email to coordinate the annual report? <civodul>hmm i think efraim, bonz060, and PurpleSym are not on that alias, but they may have something to share <rekado>(call-with-input-string "string-append\n \"hello\" \"world\"\n \"yes\"\n\n" (@@ (language gwl spec) gwl-scheme-read-chunk)) ==> ((string-append "hello" "world" "yes")) <rekado>this is only for strings that follow line indentation, so I think this is not so bad <rekado>now the question is whether Arne can be convinced to accept this change <rekado>unfortunately, it doesn’t help with keywords. I can’t recognize srfi-88 keywords. Strings and numbers are easy, though. <zimoun>civodul: thanks, I will drop an email later this evening to guix-hpc + CC the 4 other people. <zimoun>rekado: awesome the extension. If the extension has the same name as a script, then the script is first. Maybe this precedence could be reverse? WDYT? <zimoun>rekado: about string and WISP, in my mind . means “no new (“, so I am confused by your example. <rekado>zimoun: I thought that extensions might be broken, so you’d always want the core Guix features untouched. But then it’s just a matter of unsetting GUIX_EXTENSIONS_PATH, so maybe it’s better to let extensions override core commands. <rekado>that would allow for more flexibility <rekado>for example, I could offer a new version of ‘guix deploy’ that adds AWS support, falling back to the existing ‘guix deploy’ script for non-AWS stuff. <rekado>re wisp: there’s no dot in my example. With this change you can start a new line with a string without having that string be applied to anything else. <rekado>so it does the right thing even without a leading dot. <zimoun>about extension, I agree. What I have in mind is override ‘guix search’ for instance. <rekado>yeah, I think it makes sense to invert the order. <zimoun>re wisp, I see. Well, I need to read myself :-) <rekado>I think I’ll do without srfi-88 and use (read-set! keywords 'prefix) instead <rekado>I’ll lose keywords like ‘this:’, but I can still use ‘:this’ <rekado>I just didn’t want to have the noise of ‘#:this’ <rekado>with keywords that start with a colon I can tell the wisp reader to avoid application of keywords