IRC channel logs


back to list of logs

<ArneBab_>New article: Using Guile Scheme Wisp for low ceremony embedded languages:
<ArneBab_>ACTION is happy ☺
<mark_weaver>ArneBab_: looks nice!
<mark_weaver>I like the minimized command to download, compile, and get to a wisp REPL :)
<mark_weaver>ArneBab_: I've forgotten the details of wisp syntax: why is the '.' needed before the "\\n" in the definition of First_Witch ?
<mark_weaver>ah, because otherwise it would become ("\\n")
<mark_weaver>sorry, I should RTFM instead of asking here :)
<davexunit>ArneBab_: that is really neat!
<davexunit>the use of Shakespeare is great.
<davexunit>one could picture this used for a visual novel engine
<davexunit>RenPy is the popular free software visual novel engine, but you write in a language that is like Python, but not actually Python, so you are restricted in numerous ways
<taylanub>what does everyone think about the argument order (hashtable-for-each ht proc) and (hashtable-fold ht knil kons)? so the procedure arguments are at the end. comparisons:
<Tsutsukakushi>is there documentation for ice-9 somewhere ?
<taylanub>Tsutsukakushi: some of the ice-9 modules are documented in the manual, here and there, especially under (info "(guile) Guile Modules")
<taylanub>hmm, Racket, Gauche, and MIT/GNU Scheme use the 'hashtable, proc' order for for-each. SRFI-69 calls it "walk" instead of for-each to do the same and avoid controversy. I guess Guile is more an outlier here. question would be whether the order for hashtable-fold is acceptable that I propose
<mark_weaver>taylanub: I don't know, I can see arguments on both sides.
<mark_weaver>on the one hand, it seems a glaring inconsistency for 'hash-table-for-each' to have a different order than all the other '<type>-for-each' procedures in R7RS
<mark_weaver>on the other, I suppose the rationale for putting <proc> first in the other cases is to allow multiple lists/vectors/strings/whatever. but that would not make sense for hash tables since the order is non-deterministic.
<mark_weaver>well, unspecified.
<mark_weaver>taylanub: I think you should put a dash between 'hash' and 'table'
<taylanub>I'm basing this API on the R6RS hashtables API
<mark_weaver>ah, okay, fair enough
<mark_weaver>what a rarity for a R7RS API to base itself on R6RS, lol :)
<taylanub>yeah, I (now) also dislike the anti-R6RS attitude many people seem to have...
<taylanub>by the way I just looked around a bit and proc-last order in hash(-)table-for-each is used by: SRFI-69 (called "walk" instead of "for-each" though), Racket, Gauche, and Bigloo. proc-first is in Guile, Gambit, and Larceny. Chicken and Kawa don't have native hash table APIs. don't know if I missed any significant Scheme implementations...
<taylanub>(R6RS's API has no such operation)
<taylanub>I guess that's a pretty even distribution, and there's some very respectable implementations on the proc-last side, and it makes code much nicer in some cases, so I'll go with it for now
<mark_weaver>the fact that hash tables have no specified order means that the behavior and results of 'hashtable-for-each' and 'hashtable-fold' are sensitive to that unspecified order, which, to my mind, calls into question whether the same names should be used, and even whether these operations are a good idea at all.
<taylanub>I still worry about the 'knil, kons' order in -fold though. normally it's 'kons, knil', and e.g. Gauche goes with that (but puts them after the hash table argument)
<taylanub>hmm, Riastradh also suggested dropping them entirely, but I think they're pretty invaluable...
<mark_weaver>I'm sympathetic to SRFI-69's decision to use the word 'walk' instead of 'for-each'. I haven't looked at the discussion archive, but it seems to me that the lack of specified ordering argues for the use of a different name. it's not just about argument order.
<mark_weaver>and if you're going to be inconsistent with the argument order, then that's another good reason to avoid the term 'for-each'.
<mark_weaver>but 'hashtable-fold' may just be a bad idea entirely, to be honest.
<mark_weaver>because it's results are *definitely* unspecified.
<mark_weaver>*-for-each operations in scheme are normally done in a well-specified order.
<mark_weaver>I think this, combined with the change in argument order you propose (which seems reasonable) strongly argues for a different name, so I'd go with (hashtable-walk table proc)
<mark_weaver>however, if I were you, I would drop 'hashtable-fold' entirely
<taylanub>I guess I'll change it to hashtable-walk then. hm, I could drop hashtable-fold but there's also hashtable-map->list (Guile's hash-map->list) and I think at least that specialization of hashtable-fold is pretty invaluable
<mark_weaver>What's Riastradh's rationale for dropping these, btw?
<taylanub>the unspecified order leading to bugs, in his experience. so the same.
<taylanub>it would also mean obsoleting R6RS's hashtable-keys and hashtable-entries though. would be a pretty radical way to go I think.
<taylanub>by the way there's also hashtable-map! (modifies the values of entries as per the given procedure's return value on each key/value pair). maybe that should be renamed too.
<mark_weaver>taylanub: I don't know. there should probably at least be some way to get the set of keys and entries, but probably such operations should be discouraged. the existence of 'hashtable-fold' will rather encourage the use of this kind of operation, which may be a bad idea.
<mark_weaver>taylanub: yes, I think that should be renamed. Maybe just use R6RS's 'hashtable-update!'?
<taylanub>that updates single entries.
<taylanub>hm, actually, map's order is normally unspecified anyway, so maybe it's fine. the only problem is it doesn't do a real "map" but maybe that's fine?
<taylanub>it maps the values of the entries to new values (destructively)
<taylanub>(I mean, it's guaranteed to mutate them. don't know if "destructive" is the right term.)
<mark_weaver>taylanub: the order in which the procedure invocations is run is unspecified, but the results of 'map' are fully specified.
<taylanub>hm, right, the order of the results matches the order in the input list after all
<mark_weaver>taylanub: I think the word 'map!' is confusing here.
<taylanub>maybe R6RS's route of only providing -keys and -entries procedures wasn't so bad, but there's the problem that it forces allocation. for a hash table with a million entries (don't know if that's an overblown example), you will be allocating a vector of length 1M. that's roughly 8 MB on a 64b platform if I'm not mistaken.
<taylanub>worse with lists instead of vectors
<mark_weaver>taylanub: well, there's 'hashtable-walk'
<taylanub>mark_weaver: I guess I could call it -update-all!
<taylanub>mark_weaver: one isn't allowed to mutate during a walk unfortunately. (not sure what implementations could deal fine with that)
<mark_weaver>taylanub: sure, that seems a reasonable name.
<mark_weaver>taylanub: yeah, I was wondering about that.
<mark_weaver>I don't know, I have mixed feelings about hash tables in general. admittedly, they are invaluable for their efficiency, but they definitely tend to coerce code that uses them into an imperative style.
<mark_weaver>which I find very problematic
<mark_weaver>I'm sympathetic to both R6RS and Riastradh's inclination toward minimalism in a hash table API
<mark_weaver>(well, I don't want to put words in his mouth; to be fair, I'm making assumptions about his position :)
<taylanub>quoting his suggestion to me: "Don't expose any ordering. MIT Scheme has it, the order is not deterministic, and it has been a source of many heisenbugs."
<mark_weaver>IMO, that's very good advice.
<mark_weaver>or at least don't encourage use of APIs that expose ordering. creating convenient tools that are tempting to use but expose ordering, like hashtable-fold, seems questionable to me.
<mark_weaver>IMO, anyway. I can certainly understand your position as well.
<mark_weaver>(honestly, I think that we should be discouraging the use of hash tables altogether)
<taylanub>yeah, sooner or later I'll look into Clojure's maps and other functional data types. (or had ijp implemented them all already?)
<mark_weaver>taylanub: I've heard that both ijp's pdfs and wingo have implemented clojure-style HAMTs for guile
<mark_weaver>(well, ijp's is probably in portable R6RS)
<mark_weaver>rlb is also interested in bringing clojure idioms and data structures into guile
<taylanub>BTW Larceny implemented many R6RS APIs in the form of R7RS libraries. that's pretty neat for porting from R6RS to R7RS.
<taylanub>hm, I might just add warnings to -fold and -map->list saying that -fold should only be used with functions that are "commutative" in a sense (or would that be abuse of terminology?), and that the result of -map->list should be considered a set.
<wleslie>and commutative
<taylanub>mark_weaver: might 'hashtable-sum' be a better name for -fold?
<wleslie>reduce feels less directional than fold
<taylanub>that has different semantics though. less general if I'm not mistaken.
<taylanub>it wouldn't allow implementing hashtable-map->list (easily) from what I can tell
<mark_weaver>I guess my position is that the use of hash tables should be discouraged altogether, so it's better to keep the API fairly minimal.
<mark_weaver>and at the very least, I would avoid convenience APIs with questionable semantics that tempt users to introduce non-determinism into their code. hashtable-fold is an example of such an API, no matter its name.
<wleslie>btw did you hear that pypy's hashtables are now ordered
<taylanub>indeed, ArneBab brought that up on the SRFI-125 discussion
<taylanub>I wonder if they really use a tree underneath, or how?
<wleslie>the (key, value) pairs are stored separately from the hash -> index array
<wleslie>so you use the hash to look up the index in an array, and that array happens to be in insertion order
<mark_weaver>well, that seems like a good way to restore determinism with only a modest cost: an extra memory access per lookup
<taylanub>it would probably be too risky to suggest that as the standard hash table API for Scheme/R7RS though? or maybe I can try to persuade people.
<mark_weaver>taylanub: it's probably not something that can be mandated in R7RS
<taylanub>yeah :\\ I guess for now I'll just move forward with hashtable-sum, and hashtable-map->lset (new name, or is that too strange...)
<mark_weaver>implementors will want something that can be a thin wrapper over their existing hash table implementation
<mark_weaver>what is hashtable-sum ?
<taylanub>replacement for hashtable-fold
<mark_weaver>what API would it have?
<taylanub>just a different name, that is. to make it clearer that the procedure should be commutative and associative
<taylanub>(well, not exactly the procedure itself I guess. maybe I need different terms to describe the concept.)
<wleslie>could be hashtable-foldu, by analogy with foldr/foldl for directional folds
<wleslie>hmm, still ugly
<taylanub>e.g. when considering the result of (hashtable-fold ht '() (lambda (key value acc) (cons (cons key value) acc))) to be a set ("lset") rather than a regular list, then the order of application is clearly unimportant, and *some* clear semantics are retained... (one could use an actual set data type, then it wouldn't be just by convention either.)
<taylanub>or as a simpler example, it could indeed take the numeric sum of the values (which are assumed to be numbers)
<mark_weaver>wleslie: in pypy hashtables, what happens when you remove an entry from the table? can its index be reused?
<mark_weaver>are the indices compacted during GC ?
<wleslie>that's a good question. it's been a while since I've looked at the implementation. let me pull it up.
<wleslie>if 50% or more allocated entries are dead, the table is compacted
<wleslie>the head keeps stats on things like this.
<mark_weaver>wleslie: thanks!
<amz3>héllo :)
<davexunit>civodul: more on topic here in #guile, I'm trying to come up with a naming style for describing "render combinators", a purely functional interface to describing 2D/3D graphics rendering procedures with OpenGL
<civodul>the idea of having a purely functional interface to OpenGL sounds neat
<civodul>reminds me of "functional PostScript"
<davexunit>I have something working already, but I'm just unhappy with the names
<davexunit>civodul: yes, this API is Sly-specific, but maybe could be generalized later.
<davexunit>Sly has special data types that abstract over OpenGL constructs like textures, shaders, vertex buffers, framebuffers, etc.
<civodul>so i guess you'd have constructors like 'texture', 'shader', etc., no?
<davexunit>yes, those are not part of the monadic interface.
<davexunit>but there are procedures that set those in the OpenGL state machines
<davexunit>setting a texture is currently called 'texturize'
<davexunit>shaders have 'shade'
<davexunit>binding and drawing a 2d/3d model is unidealy named 'geometry'
<davexunit>texturize, shade, those are imperative verbs
<davexunit>which may make it appear as though it's actually performing those actions
<davexunit>which isn't true.
<davexunit>since it just returns a rendering procedure
<davexunit>I may prefix all of these with 'brush-' or some other kind of metaphorical thing.
<davexunit>moving a brush around on a canvas sounds like a reasonable metaphor.
<mark_weaver>davexunit: oooh, sounds excellent!
<davexunit>mark_weaver: yes, it's the best functional rendering interface for Sly I've been able to implement.
<davexunit>and I've been through several other implementations
<davexunit>(part of the reason Sly seems to be in a state of almost-ready for release for over a year)
<davexunit>it ends up working similar to SICP's picture language, but lower-level.
<davexunit>I think implementing the picture language on top of this would be a good exercise. I'd like to see 'square-limit' in action.
<mark_weaver>I'd love to have both emacs-based and web-based REPLs for Guile where if the result is one of these 2d/3d models, the rendered result is shown inline in the REPL transcript.
<mark_weaver>an emacs-based one should be quite easy to adapt from imaxima, which does something very similar when plotting graphs.
<mark_weaver>imaxima is an emacs-based REPL for the Maxima computer algebra system (which is written in common lisp)
<mark_weaver>as I recall, the way it works is that there's code within the Maxima process that saves the image to a file in disk and then outputs a specially formatted message containing the filename. imaxima detects that specially formatted message in the output and replaces it with the image in the emacs buffer.
<mark_weaver>math output is also rendered using TeX and the nice rendered formulas are displayed in emacs.
<davexunit>mark_weaver: yes that would be awesome
<davexunit>Geiser can already render images from Racket
<mark_weaver>ah, in that case I guess it's even better to add support in geiser!
<mark_weaver>and I'd guess it would probably be fairly easy
<davexunit>I'm just not sure what the implementation details are
<davexunit>cairo could be used under-the-hood to generate images in guile
<davexunit>and then somehow get that rendered to a file on disk for emacs to read.
<mark_weaver>yes, I've used guile-cairo to render png files before, it's straightforward
<mark_weaver>although I didn't know if there was a better way to create the image files in the opengl world.
<davexunit>OpenGL isn't ideal for non-realtime graphics
<mark_weaver>in what way is it deficient for that purpose?
<davexunit>well, there's the complicated API because it's built for efficiently rendering things at, say, 60FPS
<davexunit>and then there's no built-in way to write a framebuffer to disk.
<davexunit>you have to use some other library for that part.
<davexunit>but it is powerful, it could be used for one-off 3D scenes.
<davexunit>for static 2d scenes, cairo seems better suited, especially if you want to deal with vectors.
<davexunit>OpenGL is very low-level and does not deal with vector graphics.
<rlb>mark_weaver: indeed, and I've made progress, but all the data structures are currently just wrappers of pfds. Which might or might not be sufficient. The vector, in particular is quite a bit different in clojure/jvm iirc.
<rlb>mark_weaver: fwiw, I was working on agents, but someone's suggested to me that agents aren't a good idea (given core.async now), so I've shifted to other things while pondering the argument.
<paroneayea>so I'm at a breakpoint, and I'd like to evaluate things at the breakpoint given current variables
<paroneayea>how do I do that?
<paroneayea>I figured from the manual it was
<paroneayea>,inspect foo
<paroneayea>but doesn't seem to be...
<paroneayea>what ,locals makes available to me isn't always enough...
<mark_weaver>paroneayea: I don't think we have the ability to run arbitrary code in the local context like that.
<paroneayea>mark_weaver: oh :(
<paroneayea>mark_weaver: that seems surprising though!
<mark_weaver>well, we have this thing called 'local-eval'
<paroneayea>mark_weaver: all the data is surely there right?
<paroneayea>mark_weaver: oh, let me see what that is
<mark_weaver>but that requires having a lexical environment object, which requires changing the code.
<paroneayea>ah, hm
<paroneayea>mark_weaver: it seems like something that could probably be added though, when variables have not been somehow optimized out?
<mark_weaver>you write (the-environment) and it returns a lexical environment object that can be passed to 'local-eval' along with an expression to evaluator
<mark_weaver>paroneayea: well, maybe for some subset of operations, but not in the general case
<paroneayea>mark_weaver: hm, if it's not a pain to ask, why not?
<paroneayea>I would think the vm would hav eall that data
<mark_weaver>for one thing, if a variable is never 'set!' within its lexical scope, then it can be represented differently, inlined, etc.
<paroneayea>optimizations are destructive I guess? :)
<mark_weaver>and that's very important for optimization. mutable variables are much more expensive
<mark_weaver>also, macro expansion requires the full compile-time environment information
<paroneayea>ACTION nods
<mark_weaver>now, when you put (the-environment) in there, then all accessible lexicals magically become mutable.
<mark_weaver>and all of the needed lexical information is stored away
<mark_weaver>(the-environment) actually expands into a rather large thing, and mostly inhibits optimizations within that scope
<mark_weaver>so it's a heavy mechanism, but perhaps sometimes useful
<paroneayea>oh neat
<paroneayea>mark_weaver: so if I need all that, just dump in (the-environment)
<paroneayea>and break becomes more useful?
<paroneayea>mark_weaver: that's cool to know
<paroneayea>mark_weaver: great, thanks so much :)
<mark_weaver>if you bind (the-environment) to a variable, then you can get that value in the debugger, and run 'local-eval' to execute arbitrary expressions from the lexical environment where (the-environment) appeared
<paroneayea>mark_weaver: cool!
<mark_weaver>and by the way, I really mean the lexical environment where that text appeared, even if that's within the definition site of a macro that contains (the-environment) within the macro template.
<paroneayea>mark_weaver: very helpful, thanks
<paroneayea>ACTION files this info into orgmode :)
<taylanub>wleslie: I should've taken your advice. :) I decided to call it 'hashtable-reduce' now, after input from Shiro Kawai
<taylanub>apparently the semantics for "reduce" in SRFI-1 isn't exactly authoritative.
<ArneBab>mark_weaver: no prob, I’m glad you like it :)
<ArneBab>mark_weaver: the wish to provide the update instructions over twitter brings nice ideas :) (that’s why it’s that short, and shell globbing is fun ☺)
<ArneBab>davexunit: I initially wanted to use screenplay-syntax, but that uses indented person-headers. Shakespeare however uses something which looks so close to wisp that it’s hard to argument that the code is not how one would write that naturally ☺
<ArneBab>though the appeal to authority on how to structure a play (“Shakespeare did it”) feels a bit too neat… the feeling is hard to explain ☺
<davexunit>ArneBab: Shakespeare is one of those appeals to authority that everyone is OK with ;)
<davexunit>^ alien smiley face
<ArneBab>>] ← evil grin
<ArneBab>taylanub: would it be possible to add a rule that hashtable-reduce and such must work in a specified order?
<ArneBab>that would add a runtime cost for implementations without ordered hashtables but would be free with ordered hashtables.
<mark_weaver>ArneBab: what order should be specified?
<davexunit>hash tables are an unordered structure, if you want order then you should use a different structure.
<ArneBab>mark_weaver: damn, that question goes deeper than I expected
<taylanub>davexunit: well, apparently PyPy supports ordered hash tables via an additional layer of indirection
<mark_weaver>taylanub: when I browse with emacs-eww at least, several of the messages in the SRFI-126 list archives are blank, e.g. all the recent posts from Shiro.
<taylanub>ArneBab: I think that would be quite difficult both to specify and implement well
<mark_weaver>I wonder if the archival program is choking on non-ascii characters or something
<taylanub>hmm, it could be that they're in HTML. in Firefox, they're displayed but the style looks different from other mails.
<mark_weaver>do you see Shiro's message at that URL?
<taylanub>yes. it uses some fancier formatting than the plaintext in other mails. e.g. a vertical line instead of vertical row of ">" characters for quoting.
<mark_weaver>bah, so it requires javascript to even read the text :-(
<ArneBab>taylanub: I agree. Would it be possibly to specify that if the hashtables are ordered, these operations must preserve the order of the hashtable?
<taylanub>in Gnus I noticed them not wrapping correctly
<mark_weaver>that's disappointing
<taylanub>indeed, disabling JavaScript makes his text not render when loading the page :\\
<davexunit>yet another annoying change by the new SRFI maintainer
<taylanub>ArneBab: hmm, it might be unnecessary prose, since an implementation with ordered hash tables would surely do that anyway
<paroneayea>ArneBab: wisp does seem pretty cool
<paroneayea>ArneBab: we need a guix wisp package! :)
<paroneayea>all the cool guile libraries need guix packages so I can play with them easily ;)
<ArneBab>paroneayea: agreed :)
<ArneBab>I just need to do it…
<ArneBab>(and that means getting guix running again)
<paroneayea>this json-ld implementation is working more and more bit by bit
<paroneayea>and my guile skills are growing rapidly
<paroneayea>this has been a nice stress test project of my abilities
<paroneayea>I'm already going back and refactoring code to use things like call/ec where otherwise I had extremely gnarly code to be able to return certain values only by conditionals
<paroneayea>and now that I realized (compose) makes good use of multiple value return
<paroneayea>redoing sections that were previously very nested but all are processing 2 values
<paroneayea>to be multiple passes