IRC channel logs

2015-04-25.log

back to list of logs

<mark_weaver>please_help: you could generate gensyms, but the problem is that any later references to the binding would also need to be mapped to that gensym.
<mark_weaver>which kind of defeats the purpose
<mark_weaver>if everything is going to be remapped to the gensym, then why not just use the original name?
<mark_weaver>somehow you'd need to look at (register-op (s:+ (s:+ derive 'params) (s:+ integrate 'params) x . xs) '+) and know to map some of those occurrences of 's:+' to the gensym, and some not.
<mark_weaver>please_help: can you help me understand why you want to give the same name to both of these distinct procedures?
<please_help>mark_weaver: simply so that I can adjust the function definition to properly generate the <proc>. In the first pass, many ops are registered. In the second pass, the "exec" part of the "partial" ops are used during the description of the derivative or integration part of said ops.
<please_help>e.g. d/dx x + x -> 1 + 1 such that I need to generate a plus op that looks like <<proc> fn: + args: (x x) vars: (x) d: <<proc> fn: + args (1 1) vars: () d: <const value: 0> ...> ...>
<please_help>so I was trying to first generate + ops in the first pass without the derivatives, then in a second pass generate the ops with the derivatives.
<mark_weaver>well, in Scheme, each module has just one mapping from identifiers to variables (mutable locations in the store). this is needed to support recursive definitions obviously.
<mark_weaver>so I suggest that you use different names for the preliminary first-pass bindings.
<mark_weaver>maybe just add a single character like '~' or something.
<please_help>That's what I was going for with the gensym idea
<please_help>just adding a symbol can cause conflicts which is why I wanted a full gensym
<mark_weaver>well, then the problem is deciding which later references should map to the gensym, and which should be left alone.
<mark_weaver>not to mention that we don't provide any mechanism to transform all later expressions
<mark_weaver>but even if we did, I'm not sure how you'd use it
<please_help>Unhygienic macros, maybe?
<mark_weaver>it seems that what you'd need is for 'register-op' to add some transformation function to some list of transformations that we would apply to all later expressions (in the same module ?). and these would all pile up, and look at things like (register-op (s:+ (s:+ derive 'params) (s:+ integrate 'params) x . xs) '+) and decide which occurrences of 's:+' should be mapped to the gensym and which should be left alone.
<mark_weaver>and even that wouldn't be enough, because how would 'register-op' know whether this one was the final definition or some preliminary one.
<mark_weaver>?
<mark_weaver>that way lies madness
<please_help>yup
<mark_weaver>this is a bad design
<mark_weaver>the preliminary and final bindings are different things... they should have different names.
<mark_weaver>you shouldn't try to pretend they are the same and expect the system to guess which one you want for each reference.
<mark_weaver>even from the point of view of being able to read the code and understand it, you should not conflate the two bindings, IMO.
<mark_weaver>please_help: another possible option is to use laziness (see SRFI-45) to delay the computations that must be deferred.
<please_help>But the bindings need to be the same after the second pass because otherwise, e.g., (x^2 + x^2)'' is undefined.
<please_help>does this also include lazy binding?
<please_help>e.g. (define expr (delay (+ 1 2))) (let ((+ -)) (force expr)) ;=> -1?
<mark_weaver>the result of that would be 3
<mark_weaver>because the delayed expression is evaluated in the lexical environment where it was delayed.
<mark_weaver>but my idea is that the parts of the operator object that would have to be deferred to the second-pass could just be wrapped within 'delay', and then forced when needed.
<please_help>Then it would fail because the timing would be incorrect: (x*y)' = (x'*y + y'*x) such that the +proc has to be properly defined at this point, but that requires waiting until the second pass.
<mark_weaver>but it would be properly defined, and the 'exec' part would be available.
<please_help>The problem is that, while the +proc generator can be described in a single pass, the +proc creating as part of the description of its derivative causes recursion without an exit condition.
<mark_weaver>it's only the information about derivatives and integration that would be delayed until it was first needed
<please_help>s/creating/created/
<mark_weaver>well, I don't understand enough about how your system works, and I don't have time to study it right now.
<please_help>That's because we can't know in advance with regard to which variable we're going to be deriving I think.
<please_help>OK, I'll think about it on my side too and maybe I'll come up with a solution tomorrow.
***Guest61661_ is now known as Guest61661
<dujmidd>Hi all! I read something about planned support for Lua and Python, but I didn't find any information about the state of this planning.
<dujmidd>Is it still planned or already rejected?
<mark_weaver>it's not rejected, it's just that no one is working on it right now.
<ft>It's not rejected as far as I know, it's lackeing manpower
<mark_weaver>I toyed with the idea of working on python-on-guile, but realistically it's too far down on my priority list now and I'll never get to it.
<mark_weaver>but it would be good to have. same for lua.
<dujmidd>Thank you for the answer!
<dujmidd>Another question: Is it planned to improve the support for C++? It is not too much fun, dealing with all the type conversion errors if you use Guile in C++...
<dujmidd>And before everybody writes his own wrapper it might be better to support it out of the box.
<mark_weaver>I would support improved support for C++, if we can agree on the details. However, I would probably oppose implementing any of Guile core in C++ or having it use aspects of the C++ runtime internally.
<mark_weaver>instead, IMO, the improved C++ support should be in the form of wrappers or conversions that happen at the interfaces between libguile and C++ code.
<dujmidd>mark_weaver, Yeah. Is anybody already working on it? Or should I just start from scratch if I am interested in it?
<mark_weaver>there was a recent post about this on guile-devel, specifically about exception handling and C++, but I haven't yet had the time to post my thoughts on it to the list.
<mark_weaver>if you wanted to work on it, I think that would be great, but please let's talk about the details of the design on guile-devel before you spend much time implementing it.
<dujmidd>All right. Thank you!
<mark_weaver>you're welcome!
<mark_weaver>thanks for your interest in helping us improve these things :)
<please_help>I want to capture the specific case where an expression matches (quote x), but not (whatever x), in a macro. Is that possible?
<please_help>Also, is there something like a "macro-map"? (macro-map the-macro '(the things to transform))
<please_help>And it seems that a macro definition like ((_ (fn arg1) (if (eq? (quote fn) 'quote) arg1 ((the-same-macro fn) (the-same-macro arg1)))) kills the stack, I'm assuming expansion happens unconditionally despite the if? Yet, (the-same-macro fn) isn't supposed to be valid unless fn has a very special form in my real case, but I get a stack overflow no matter what fn is.
<please_help>And how come the interpreter randomly? either gets aborted or displays a vmerror? Is that a bug in the interpreter or is it OS-related?
<taylanub>please_help: (syntax-rules (quote) ((_ (quote x)) ...)) would match (the-macro (quote x)) or (the-macro 'x). same with syntax-case. that is, you need to put `quote' in the "literals" list of syntax-rules/syntax-case
<taylanub>macro-map could be implemented like: http://sprunge.us/ZRDg (this will more generally map any expression onto any list of expressions, e.g.: (syntactic-map display (foo bar)) => (begin (display foo) (display bar)).)
<taylanub>one could also generalize that to any number of operands per operator-call: http://sprunge.us/jNFa example: (syntactic-map + ((2 3) (4 5))) ;useless example since the addition results are unused but you get the idea
<taylanub>you could also make it configurable whether it uses "begin" or "values" or whatever for combining the map result: http://sprunge.us/KdSR
<taylanub>please_help: macro expansion happens strictly before any kind of code execution, and 'if' is not a macro (but a "special form") so yes, that recursive macro will try to create infinitely deep nested code
<taylanub>you have to use an equivalent of 'if' in macro logic, which is of course only possible if the test-expression does not rely on data that will only exist at execution-time. think of the macro system as a meta-program that runs during the compilation of your code, and after that the compiled code is written to a file (this also means that e.g. the macro system cannot insert a procedure object into
<taylanub>the code it generates; it can only insert a lambda code-snippet that will generate a procedure when the program executes. likewise for any data type that can't be serialized.)
<taylanub>please_help: the random abort/error might depend on the exact point it runs out of stack memory or something like that
<please_help>alright, thanks for all the answers, taylanub
<taylanub>HTH :)
<please_help>It looks like this syntactic-map will also overflow the stack unless the number of arguments are known at macro-expansion time, unfortunately.
<please_help>I have a macro of the form (define-syntax-rule (macro expr bindings) (let bindings (local-eval expr (the-environment)))), but this doesn't properly capture let-like bindings (instead complaining that "x" is not defined), why is that?
<taylanub>please_help: I don't really know but, I would avoid usage of local-eval as much as possible. maybe I would fear it less if I fully understood it, but my opinion so far is that it's a horrible hack that fundamentally goes against lexical scoping.
<ijp>and that specific macro is easily written without it
<taylanub>normally one can debug a macro by using ,expand at the REPL. when (the-environment) is involved, this becomes a little difficult, because try it and see :-)
<please_help>heh, I see
<please_help>so, how do I go about not using local-eval in this case, given that I need the local environment?
<please_help>from ,expand I see that the binding looks correct, but somehow it still complains about the symbol. Doing it explicitly works, however.
<taylanub>please_help: you can create and manage environments procedurally; they won't interact with any of the lexical environments of the code that is creating and managing them, but in general that should be unnecessary
<mark_weaver>please_help: can you show me a self-contained example where it complains that 'x' is not defined?
<taylanub>mark_weaver: (macro '(+ x y) ((x 0) (y 1))) reproduced it for me
<please_help>thanks, taylanub
<mark_weaver>thanks
<mark_weaver>please_help: okay, the problem there is that (the-environment) captures the lexical environment whereever that actual expression is present, by default.
<mark_weaver>and in this case, that is the lexical environment of the macro _definition_, not the macro use.
<please_help>ah I see
<mark_weaver>however, 'the-environment' can accept a syntax object
*mark_weaver refreshes his memory before saying more, hold a bit please
<please_help>e.g. (datum->syntax 'x)?
<mark_weaver>please_help: so, this optional operand is not actually documented, but we should do so.
<mark_weaver>just pass it a syntax object corresponding to an identifier, and it will use the environment where that identifier was introduced.
<mark_weaver>so, in the case of a macro use, pick some identifier from the macro use, usually. the macro name itself might be best, but you can only do that if it's a syntax-case macro.
<mark_weaver>so, you could do this: (define-syntax macro (lambda (form) (syntax-case form () ((macro expr bindings) #'(let bindings (local-eval expr (the-environment macro)))))))
<mark_weaver>the reason you have to use 'syntax-case' here is that in 'syntax-rules' macros, the pattern variable in the "operator" position (in this case 'macro') is not bound to the actually operator in the macro use, but is rather bound to fresh identifier.
<mark_weaver>and that holds for 'define-syntax-rule' as well.
<mark_weaver>if you want to actually capture the identifier that was present in the operator position of the macro use, you have to use 'syntax-case'.
<please_help>seems to work
<please_help>I still need to bind the symbols before giving them to (the-environment) right?
<mark_weaver>I don't understand the question, you'll have to be more clear
<mark_weaver>one way to think about it is this: imagine that you took the expression passed to 'local-eval', and put it in place of the identifier that's passed to 'the-environment'.
<please_help>here's the current full definition: http://paste.lisp.org/+35T6 in that case, the let is indeed mandatory, right?
<mark_weaver>and then imagine if it was evaluated in that position, would it get the bindings you want.
*mark_weaver looks
<mark_weaver>please_help: 'the-environment' only accepts up to one operand. it can't accept arbitrary numbers of them.
<mark_weaver>so that's incorrect
<mark_weaver>it has to evaluate the expression within a *single* lexical environment, so you have to pass it exactly one identifier, whose lexical environment will be used.
<please_help>Is there a way to communicate multiple bindings then? Also, why is it necessary to have a lexical in the syntax-case even if it ends up not being used?
<please_help>ah
<please_help>that sounds a lot more tricky then
<mark_weaver>in that paste, you aren't using the pattern variable in the operator position, so you needn't use syntax-case there.
<mark_weaver>please_help: I don't understand the question about "multiple bindings". (the-environment) captures its entire lexical environment, and (the-environment id) captures the entire lexical environment where 'id' was introduced.
<please_help>new version: http://paste.lisp.org/+35T6/1
<please_help>yeah, I get it now, I didn't know that before I asked my question, you answered it at the same time I asked it.
<please_help>should I instead add a fake binding inside the let? I don't think I have any other symbols that are in the correct scope.
<mark_weaver>that wouldn't work
<mark_weaver>(the fake binding)
<please_help>why not?
<mark_weaver>the name of the fake binding would presumably be introduced by the macro, correct? (as in, not present in the macro use)
<mark_weaver>in which case it would capture the lexical environment of the macro definition, because that's where that identifier was introduced.
<please_help>I see
<please_help>but is it then impossible to have a construct like (let ((fake-name (gensym))) #'(let ((,fake-name #f) (b v) ...) ...)) ?
<mark_weaver>this is not going to really be hygienic, because of course '%resolve' is probably going to introduce other identifiers, and there will be an assumption that those have the expected bindings where 'b1' was introduced.
<mark_weaver>that wouldn't work
<please_help>It is the user's fault if they do not give us the bindings for variables referred to in %resolve's evaluation, but %resolve itself does not introduce any new bindings (for now).
<mark_weaver>instead of %resolve returning a normal s-expression, the proper way to do this is to have %resolve accept and return syntax-objects.
<mark_weaver>and then it would be hygienic
<mark_weaver>syntax-objects are like s-expressions except that they preserve hygiene by default (unless you specifically override it in particular cases)
<mark_weaver>syntax-objects are the representation of expressions and code that is used by the macro expander.
<mark_weaver>'syntax-case' is how to destructure syntax-objects, and 'syntax' (usually written in the shorthand #'... form) is how to construct syntax objects.
<please_help>resolve must not be a macro, however. can syntax objects be created procedurally?
<mark_weaver>yes, absolutely
<mark_weaver>syntax-case macros are just procedures.
<mark_weaver>when you write (define-syntax macro (lambda (x) ...)) that's a normal procedure
<please_help>right
<mark_weaver>it's a procedure that accept the syntax object of the entire macro use.
<please_help>I see.
<mark_weaver>usually, we start by doing 'syntax-case', but you can do whatever you want.
<mark_weaver>similarly, in the right-hand-side of every "case" of a 'syntax-case', we usually start with 'syntax' (written as #'...), but you needn't.
<mark_weaver>you can have a whole bunch of procedures that pass around these syntax objects, iterating over things, etc.
<mark_weaver>please_help: take a look at our definition of 'cond' in module/ice-9/boot-9.scm, for example. search for "define-syntax cond" to find it.
<mark_weaver>'syntax' is analogous to 'quote', except that it preserves the lexical environment information of the identifiers within.
<mark_weaver>actually, that definition of 'cond' makes some slightly inappropriate assumptions about the representation of syntax-objects, so it's best not to emulate that code.
<mark_weaver>(outside of guile)
<mark_weaver>please_help: see also 'syntax->datum', 'datum->syntax', and 'with-syntax'.
<please_help>So (syntax (+ 1 2)) will give me the syntax object associated with 3, but not (+ 1 2)?
<mark_weaver>so, it will give you a syntax-object corresponding to the expression (+ 1 2)
<mark_weaver>and the '+' there will refer to the lexical binding of '+' that was in scope where that 'syntax' form was present.
<mark_weaver>so, for example, if that syntax-object is then put somewhere that '+' has a different binding, you will still get the '+' that was active where you wrote that 'syntax' expression.. and that's an example of how hygiene is maintained.
<mark_weaver>that's why I can write (define-syntax-rule (double x) (+ x x)), and it will still work even if this macro is used somewhere that '+' has a different binding.
<mark_weaver>because that turns into (define-syntax (lambda (form) (syntax-case form () ((_ x) (syntax (+ x x))))))
<mark_weaver>and the 'syntax' doesn't just store the symbol '+' in that syntax-object, but rather the *identifier* '+', which remembers the lexical environment where that '+' was introduced.
<mark_weaver>"identifier" means a syntax-object corresponding to a symbol.
<mark_weaver>(as opposed to more general syntax-objects that might correspond to any s-expression)
<mark_weaver>there's a predicate 'identifier?' which is often used in syntax-case macros, which you should use in place of 'symbol?' for example.
<please_help>how can a syntax object then be evaluated?
<mark_weaver>well, you can't evaluate it in general because this is all happening at macro-expansion time, which is part of compilation. it happens before the actual expanded code is run.
<mark_weaver>what you would do instead is to splice the syntax-object into the expanded code, which will later be run.
<mark_weaver>and you can do that in a couple of different ways.
<mark_weaver>you can use 'quasisyntax', which is analogous to 'quasiquote', something like: #`(let bindings #,expr) where 'expr' is bound to a syntax object.
<mark_weaver>or you can use 'with-syntax' to bind the syntax object to a 'pattern variable', which are the same as the pattern variables bound by 'syntax-case'. (in fact 'with-syntax' is just a macro that expands into a use of 'syntax-case')
<mark_weaver>and then pattern variables are substituted in any use of 'syntax'.
<mark_weaver>so (with-syntax ((expr <syntax-object>)) #'(let bindings expr))
<mark_weaver>the 'expr' in the 'syntax' form will be replaced with the syntax-object returned by the expression <syntax-object>
<mark_weaver>so <syntax-object> might be something like (resolve-graph #'graph)
<mark_weaver>where 'graph' was a pattern variable from the original 'syntax-case'.
<mark_weaver>so, putting this all together, something like:
<mark_weaver>(define-syntax graph-eval (lambda (form) (syntax-case form () ((_ graph ((b v) ..)) (with-syntax ((resolved-graph (resolve #'graph))) #'(let ((b v) ...) resolved-graph))))))
<mark_weaver>whre 'resolve' is a procedure that accepts a syntax-object and returns an optimized syntax-object.
<mark_weaver>and then, it could be done hygienically.
<mark_weaver>if you want to write an expansion-time optimizer, this is the much better way to do it.
<please_help>Your solution yields "reference to identifier outside its scope in form graph". I tried other formulations but either get reference to pattern variable outside syntax form in form graph (when not syntaxing graph), or unsyntax: Invalid expression in form (unsyntax (%resolve (syntax test-proc)) where test-proc is a proc-object that described an op.
***FracV_ is now known as FracV
<mark_weaver>please_help: I typed in the exact expression I typed above (but with '..' replaced with '...'), and then (define (resolve x) x), and then (graph-eval (+ x y) ((x 1) (y 0))) worked and returned 1
<mark_weaver>please_help: show me your exact code?
<please_help>I think the problem is with my resolve, it's returning a list of syntax elements, not syntax elements.
<mark_weaver>'syntax' is the way to create syntax objects, and 'syntax-case' is the way to destructure them.
*mark_weaver goes afk for a bit...
<please_help>Here's what I have now: http://paste.lisp.org/+35TE I don't know what I'm missing but I simply can't figure out how to get from where I am to the expected result. I obviously tried the syntax-case version (graph not syntax'd) of graph-eval first, to no avail. I take it that resolve must absolutely operate on syntax. It also seems like unsyntaxing can only be done within a syntax context. How then do I describe conditional
<please_help>computations based on the identity of the object wrapped in the syntaxobject?
<mark_weaver>please_help: I see several problems there
<mark_weaver>%iresolve --> %resolve, presumably?
<mark_weaver>qeval needs to be a syntax-case macro
<mark_weaver>so (define-syntax qeval (lambda (form) (syntax-case form () ((_ what ((b v) ...)) (with-syntax [...]
<mark_weaver>do you intend to require at least one variable to be bound here in this form?
<mark_weaver>but anyway, the deeper problem here is that when the 'qeval' syntax transformer is run at compile-time, 'what' is not a normal variable holding a <proc> object, but rather a syntax-object representing the expression 'test-proc' in this case.
<mark_weaver>so you can't use 'proc?' meaningfully in '%resolve'
<mark_weaver>sorry, I should have said that #'what is a syntax object.
<mark_weaver>'what' is actually a pattern variable, so it only has meaning within a 'syntax' form as a thing to substitute in.
<mark_weaver>'syntax' forms are actually somewhat like quasiquote forms in this respect.
<mark_weaver>where 'quasiquote' forms do substitutions on 'unquote' forms (or comma shorthand), 'syntax' forms do substitution whereever a pattern variable occurs.
<please_help>hm
<mark_weaver>'%resolve' needs to be rewritten. hold a sec..
<mark_weaver>well, it would be something more like this:
<please_help>looks like it needs to do something like (define-syntax-rule (%resolve graph bindings) (if (proc? graph) ((define-syntax (syntax-case...)) args-for-this-macro))) ?
<mark_weaver>no
<mark_weaver>%resolve is a normal procedure, called during macro expansion (part of compilation)
<mark_weaver>I'm sorry, I can't rewrite this. the whole concept is wrong
<mark_weaver>'test-proc' is a normal variable that is being initialized at *run-time* and contains a record object after this module is loaded at run-time.
<mark_weaver>'qeval' is a macro, which gets expanded at *compile-time*, long before this <proc> record object is created.
<mark_weaver>can you give me some example uses of 'graph-eval' you were hoping for, and what you'd like them to expand into?
<mark_weaver>or rather, 'qeval'
<please_help>qeval was a simplified version of graph-eval that was meant to resemble the definition you gave (as I mentioned, the syntax-case form couldn't work because what is not a thing, so to speak). Graph-eval would be used as:
<please_help>(graph-eval <<proc> fn: '- args: '(x y 5) cover: '(x y)> ((x 10) (y 20))) or (graph-eval <<proc> fn: '- args: '(<<proc> fn: '+ args: '(x 3) cover: '(x)> y) cover: '(x y)> ((x 10) (y 20)))
<please_help>Aside from graph-eval, (derive graph wrt) and (integrate graph wrt) would yield a graph of the same form that represent the computation of the derivative or anti-derivative with regard to wrt.
<please_help>Moreover, it has to be possible to manipulate the graph to optimize it, such as replacing multiple instances of the same operation with the same arguments by the same node, but that's for much later.
<please_help>The procs themselves are generated by macros based on a set of pre-registered ops so that (s:- (s:+ 'x 3) 'y) yields the second example proc.
<please_help>graph-eval should expand to (- x y 5) for the first example, and to (- (+ x 3) y) for the second example.
<mark_weaver>please_help: <<proc> ...> is not a supported syntax of the guile reader, so it can't appear in ordinary source code like that.
<please_help>test-proc
<please_help>$70 = #<<proc> fn: - args: (x y) cover: (x y)>
<please_help>so prepend a #
<please_help>proc is an srfi-9 record.
<please_help>(its type is <proc> of course)
<mark_weaver>well, that's a write-only syntax, but sure, you could add a syntax extension to guile's reader to construct these things at read-time.
<mark_weaver>although it's a bit of a pain.
<mark_weaver>perhaps better is to make 'make-proc' a macro
<mark_weaver>and then there's no need for those ugly quotes (and you don't want to strip the syntax-object information)
<please_help>The syntax-object information can be part of the fn argument, but there's no need to bother if it's not possible to make the thing work as anything but a local-eval'd quoted sexp.
<mark_weaver>but this of course assumes that <proc> is a compile-time object.
<please_help>The definition of <proc> is available at compile time, and a make-proc macro could probably be made if we were to remove the use of records, but I don't know what that would add or solve.
<mark_weaver>btw, I should also mention that 'local-eval' is a rather heavy operation that kills the compiler's ability to optimize any variables that are visible to 'the-environment' forms.
<mark_weaver>all of that would be done at run-time, not compile-time.
<mark_weaver>it's a question of whether you actually intend these things to be done at compile-time or run-time.
<please_help>from a whole-system point of view I don't think there are any runtime dependencies except maybe for registering ops, since their definition might change based on available hardware.
<mark_weaver>please_help: well, whatever you end up with, I strongly recommend that you make heavy use of the ,expand REPL command and maybe also ,optimize to see what the run-time code actually looks like.
<please_help>ok.
<mark_weaver>if you are using 'local-eval' and 'the-environment', that turns into a rather heavy thing at run-time.
<please_help>I suppose eval has much of the same performance implications?