***guile-guest0 is now known as sbwhitecap
<mark_weaver>kwrooijen: it's bad for all of those type tests and 'appends' to be inserted into the generated code, to be run on every execution of the code that uses the macro. also, it's bad to use 'primitive-eval' on quoted bits of the code, which loses hygiene and can result in unintended variable capture. in scheme, we would write those macros like this: http://paste.lisp.org/+6JZU <kwrooijen>mark_weaver: Oh wow that is so much cleaner! I wasn't aware you could pattern match like that <kwrooijen>Also I see you use parens instead of square brackets like I did, is there any reason to do that other than preference? <mark_weaver>kwrooijen: you can use either. most of us prefer to use parens, but it's mostly a matter of preference in guile. <mark_weaver>but note that usig square brackets as parens is not standard scheme, and some extensions might use square brackets for other purposes. <kwrooijen>Ah ok, then I'll avoid square brackets from now on <mark_weaver>actually, R6RS made square brackets the same as parens, but R7RS (which mostly rejected the R6RS changes) continues to reserve square brackets for future extensions. <mark_weaver>basically it was a fork in the scheme community, which is a shame <peterbrett_work>I confess my Scheme is R5RS + a few SRFIs and I have pretty much ignored the R6 & R7 stds <mark_weaver>that's probably good policy until it becomes more clear which fork is going to "win". <mark_weaver>peterbrett_work: btw, although I haven't yet responded yet, I've looked into the LLP64 issue <mark_weaver>in summary, it looks like it won't be too hard to fix this up, but with the somewhat unfortunate fact that immediate integers (fixnums) will be limited to 29 bits + sign bit even on LLP64 <mark_weaver>and actually, that's already the case, because the code that decides how many bits fixnums can be assumes that 'long' is the same size as a pointer <peterbrett_work>Well — since bignums are still available it's not a big deal really unless you're trying to write extremely optimised integer-banging code <mark_weaver>the reason we can't easily do better than that is that the GMP API uses 'long' in most of the functions that accept C integers as one of the arguments. <mark_weaver>and there are no variants that accept larger int types <mark_weaver>so I guess what needs to be done is to explicitly cast the 'scm_t_inum' values to 'long' before passing to the GMP functions that expect long -- and there are several examples of this besides 'scm_i_inum2big' -- and then to remove that #if check in 'scm_i_inum2big' <mark_weaver>the explicit casts may be not even be strictly needed <mark_weaver>numbers.h includes this code: #define SCM_I_FIXNUM_BIT (SCM_LONG_BIT - 2) <mark_weaver>numbers.h also includes this: #define SCM_MOST_POSITIVE_FIXNUM ((SCM_T_SIGNED_BITS_MAX-3)/4) <mark_weaver>and I guess that should be LONG_MAX instead of SCM_T_SIGNED_BITS_MAX <mark_weaver>there may be other problems, since it seems to be assumed in several places that 'long' is the same size as a pointer. <mark_weaver>I don't have an LLP64 machine on which to test, so you might need to take the lead on this <mark_weaver>actually, I guess we might as well avoid the explicit casts. <mark_weaver>there should be prototypes for all of those functions, and iirc none of them are varargs functions. <mark_weaver>occasionally, I've entertained the idea of avoiding the mpz_* API in favor of the mpn_* API, in order to take control over the allocation of the bigints, avoiding the extra pointer redirection, and perhaps most importantly: enabling a reduction in the number of code paths in numbers.c by using the mpn_* functions in all cases, and thereby merging the fixnum and bigint cases for most numeric procedures. <mark_weaver>so fixnums would be copied to a temporary buffer on the stack and then passed to mpn_* functions directly. <mark_weaver>but it's not yet clear when/if I'll find the spare time to make such a change. <mark_weaver>but as bonus, such a change would avoid the issue of the 'long' arguments in the mpz_* API and thus enable LLP64 systems to have 60-bit fixnums. <mark_weaver>(we could in theory support 62-bit fixnums in LLP64 systems even in the current numbers.c, but it would probably involve a non-trivial efficiency loss in several functions, and also make the code a lot uglier) <mark_weaver>kwrooijen: actually, using 'primitive-eval' on quoted bits of code in macros like that is far more problematic than I alluded to above: the most fatal problem is that the code passed to 'primitive-eval' would not be able to access any of the lexical (i.e. non-toplevel) variables in scope at that point. <mark_weaver>as a general rule, if you find yourself using 'eval' or 'primitive-eval' in your code, you're probably doing it wrong :) <mark_weaver>(unless you're writing something like a REPL or IDE or something) <lokli>hello guilers, is it possible to do (+ (non-existent) 10) , where "non-existent" will be some arbitrary not defined identifier? Idea is, the error will be caught, and there could be a handler, which will return, say 42 for the errored expression, and we get 52 as a result <davexunit>lokli: I have to go afk now, but it seems like you're looking for a nondeterministic computing system or maybe just want to use continuations. <peterbrett_work>mark_weaver: Maybe the correct test is sizeof(long) < sizeof(void *) <lokli>davexunit, it is a dsl with user defined expressions embedded into scheme code, was wondering if it is possible <davexunit>lokli: it's certainly possible, but what you've described so far isn't making a lot of sense. <lokli>davexunit, ok, thanks for the links, will study closely <davexunit>you mention domain specific languages. those often involve using Scheme's macro system. <lokli>davexunit, indeed, the idea is to make macro system available, but i dont know of a solution to non-defined identifiers with macros <davexunit>I think you would need to better explain why you feel the need for such a thing. <davexunit>if we can better understand the end goal, we could make better recommendations. <wingo>lokli: it's possible using the debugging apis but not implemented right now <wingo>i think you should be able to ,frame to the frame in question and ,return 42 <wingo>lokli: and since the debugging apis have a scheme interface it can be done programmatically too <wingo>but ,return isn't implemented right now <lokli>wingo, to my knowledge it would require debug vm and would result in degraded performance <wingo>lokli: debugging vm isn't that much of a performance difference, and no it actually wouldn't require the debugging vm at all <davexunit>lokli: it's trivial to make a printf procedure. we already have 'format' which is like printf. <wingo>e.g. gdb can return from frames without requiring the program to be recompiled, and it has a much harder job to do than guile would :) <lokli>davexunit, yup, but the big idea is to have any identifier to be used, without the need of any wrappers or functions to exist <davexunit>I'm completely in the dark as to what you are trying to do. <lokli>davexunit, i am trying to make a source-to-source compiler <davexunit>lokli: okay, so the process at a high-level is: <davexunit>where that list is an sexpression in the format of your choosing <lokli>that would require lots of hand parsing <davexunit>you don't have to parse the expression because you used a list representation. <davexunit>so now you apply pattern matching to that list <lokli>but i want to construct the list programmatically <lokli>say, i want to do (printf "hello %d" (+ 10 20)), which will be translated to printf("hello %d",30) <lokli>and with that scheme representation i can apply macro and all the stuff scheme <lokli>but if i construct it out of scheme source, say (list printf arg1 arg2), i feel like loosing all macro <lokli>more like `(printf "hello %d" ,(+ 10 20)) <lokli>but it is already overcomplication for me <davexunit>but (+ 10 20) could be an expression of your language. <lokli>but could be any scheme code <davexunit>it has to be a form that your compiler can understand. <lokli>it would be scheme, if printf was defined <taylan>I think I sort of understand what lokli wants to do <lokli>so, the first question was on how to trick scheme to think that printf in fact is defined <lokli>by error handling or any other mean <taylan>something between a DSL and an EDSL <davexunit>lokli: you can easily define a 'printf' Scheme procedure, but that doesn't help you. <davexunit>how can you take a Scheme procedure and produce C code from it? <taylan>AFAIUI it would be some object representing a C function, not a Scheme procedure <lokli>davexunit, i dont need C code, i need a procedure that will produce a string of C <lokli>and the output of all such fake procedures will be combined in a block, { printf("hello"); puts("world"); } <lokli>and then we just put it to a file <taylan>(maybe a Scheme procedure after all, but not one that simply does what the C function it represents would do, but rather one which when applied to some arguments, returns a C grammar snippet or something like that) <davexunit>lokli: I think you are confusing the various layers of a compiler. <taylan>davexunit: I think lokli wants something else than a typical compiler <lokli>davexunit, could be! but i want it to look that way, and no other =) <davexunit>lokli: it's trivial to write a procedure that returns a string. <taylan>specifically, some sort of compiler EDSL so that the local macro system can be used... <lokli>taylan, you totally get the idea <taylan>(well, rather foreign-language EDSL) <davexunit>lokli: you can make it look however you want if you only could slow down for a minute and understand the various layers of abstraction. <lokli>davexunit, i am open to anything you guys can offer, thats why i came here <taylan>davexunit: it won't be possible to use define-syntax/syntax-rules if the whole code is represented as a sexpr DSL separated from Scheme <davexunit>taylan: I'm having trouble explaining myself because no one seems to get it. <taylan>if you instead define a zoo of procedures that evaluate to composing grammar snippets, then you can use define-syntax/syntax-rules and such with it <lokli>taylan, yes, but it is way tedious, so it would be nice to do it on-the-fly <davexunit>taylan: you can certainly create Scheme macros that *expand to* the s-expression format that the compiler uses. <taylan>davexunit: I believe I understand what you mean, it's just that that typical method of representing another language as a sexpr DSL means it doesn't integrate with any of the language semantics of the lisp used to write the compiler <davexunit>this sexp-to-C thing is *no different* than SXML. <lokli>well, i kinda have a working prototype, i can show if it helps <taylan>davexunit: the one you have in mind isn't but the one lokli has in mind is :) <davexunit>the key is to understand that the syntax you are looking for is just sugar <lokli>total integration with scheme, yes. It could be anything that works and preserves the desired form. <davexunit>but what are these Scheme procedures supposed to produce!? <taylan>davexunit: objects that represent statements and expressions in the C grammar, or some such <lokli>strings for starters! it does not really matter, it depends on other parts of the compiler. But string is a good start <davexunit>taylan: that's *exactly* what I've been trying to say! <lokli>if (printf "hello") will produce printf("hello") without printf defined i am a happy panda <davexunit>'printf' is not defined. you must define it. <lokli>so the question is how to make it work <davexunit>if you want to refer to an arbitrary symbol itself, you must quote it. <davexunit>because printf has meaning in C, not Scheme. <lokli>taylan, ok, so you totally get the idea, is there any distant chance of getting it right? <taylan>lokli: well I can't help about the part with undefined symbols <lokli>taylan, right now i am using (c prinf "hello %s" "world"), with 'c being a macro that does all the work. But is kinda sucks <taylan>at best I can imagine one could declare all such symbols at the top of the file, but it will be annoying to do the book keeping <taylan>(this "declaration" of the symbols would actually define them as variables) <taylan>(holding some object saying "to be actually bound") <lokli>yes, was thinking of that, even auto parsing of header files <lokli>but seems to much work in the wrong direction. Only if i could recover from 'not defined' error in the same place with my own procedure that would do the trick <taylan>yeah, you probably want to follow wingo's lead there <lokli>yup, will try. time for some experiments then <davexunit>wanting to refer to something that has meaning in another language, but not Scheme, is the *exact* intent of quoting. <davexunit>trying to use exception handling to allow reference to the names of C identifiers is not a very good road to go down. <davexunit>I plan to write a "source to source" compiler myself, for the OpenGL Shader Language (GLSL). <mark_weaver>peterbrett_work: I think the better approach here is to say that fixnums need to fit in both a long and a pointer, so their width should be the minimum of those widths. <mark_weaver>I guess maybe that SCM_I_FIXNUM_BIT could be (min (SCM_CHAR_BIT * sizeof(long), SCM_CHAR_BIT * sizeof(scm_t_bits) - 2)) <mark_weaver>in which case fixnums could be 32 bits wide on LLP64 <mark_weaver>although it's possible that we might need to shave another bit off of that, I'm not sure. <lokli>davexunit, can you show example code of source? will you use quoting all over the place for identifiers? <davexunit>lokli: I haven't started my project yet, but what I can say that it's really important to separate the various layers of the compiler tower. <davexunit>the source code data structure will indeed be s-expressions, which one could create directly via quoting <lokli>davexunit, so, how about quoting the piece of code and then with macro establish missing bindings and eval it? <davexunit>lokli: there's nothing to eval. it's a source->source compiler. <davexunit>you could write an interpreter for this high-level lispy language, but that's another story. :) <lokli>davexunit, ok, i see your point. i think i am getting close btw, need one thing to clarify. Is it possible to create top level binding from inside functions (not from top level itself)? <davexunit>the exception here is the 'begin' form, which hoists definitions to the environment 'begin' was used it. <lokli>davexunit, you wont like the idea =) but here it goes: (let ((x (lambda () (printf "some")))) (catch #t ...) (when (undefined-variables) (establish-bindings) (try-again)) <davexunit>all of the confusion and complexity here comes from conflating the host language (Scheme) and the language you are building. <lokli>and i actually accept the idea that i wont make it. as i mentioned before, i will probably settle for (call printf "hello"), where 'call' is a macro that will do the proper work <davexunit>a macro could be used to hide quoting or whatever for certain cases. <davexunit>in my as-of-yet unimplemented sexp->glsl compiler, I plan to have a macro like this: <lokli>exactly what it will do here, notice printf is not quoted, but still not defined <davexunit>(define-shader foo-shader (some glsl expression here that is not quoted)) <davexunit>lokli: sure, I get it. the macro would expand "printf" to "'printf" <lokli>davexunit, are you going to use square brackets in it? things like '(argv[5]) wont work <davexunit>lokli: written using only parens that is '(argv (5)) <lokli>or they will, but in the wrong way <lokli>davexunit, i am still thinking on a way to implement (raw ...) <davexunit>you can't change how Scheme reads s-expressions (well, you can, but that's very dangerous) <lokli>but no idea how to get to it <davexunit>without reader macros, which are extremely dangerous. <davexunit>but what's the point? you've just made the entire usage of s-expressions moot. <lokli>i use reader macros, they seem ok for easy stuff. For example i have #<stdio.h> <lokli>which expands to something like "#include <stdio.h>" <lokli>davexunit, point of (raw)? to make some quick dirty hacks, i guess. Now that you ask it, i wonder if i actually need it <davexunit>you're mixing together a lot of abstraction layers that makes for a messy result. <lokli>i hope to get a profit from it <davexunit>I wish I could my points across better, because what you're trying to do just isn't going to work well, I'm afraid. <lokli>davexunit, no, you present your points just fine, i am just that stubborn. I have my final image and want to go there no matter what <lokli>davexunit, btw, are you going to put your code up somewhere, when you are done? or maybe even wip? would like to see it. Maybe you have github account or something similar? <davexunit>so if I ever get around to it, I will put it there. <lokli>nice, will be checking it then <davexunit>lokli: the big flaw with having Scheme procedures that produce strings of C code directly is that your compiler will be unable to perform any analysis on the complete program *before* producing C code. <davexunit>for example, one feature that a hypothetical sexp->c compiler could have is type inference. type inference is examining the context of an expression to determine the types of its inputs and outputs so that the programmer doesn't explicitly have to specify types everywhere. <lokli>davexunit, well, i got through this phase already. Now i am about to present everything as objects of proper classes, and they will be transformed to strings at final stages. Will get nice bonuses from it, like some type inference <davexunit>lokli: OK, well from you're above examples it seemed like that was not what you were going for. <lokli>davexunit, oh, i tried to use strings for simplicity, because the big picture did not seem very important with discussing error-recovery scenarios <davexunit>you had a macro that expanded (call printf "hello") to "printf(\\"hello\\");" <lokli>davexunit, yes, i do now have such a macro <davexunit>now, putting aside that a macro expander wouldn't produce a string literal, I took that to mean that the result of that expression was a string of C code. <davexunit>so there seems to be a good deal of miscommunication here. <lokli>why would not it produce literal? <davexunit>lokli: I think you may be calling something a macro that isn't actually a Scheme macro. <davexunit>(call printf "foo") couldn't expand to a string literal, but it could expand to this: (string-append (symbol->string 'printf) "(" "foo" ");") <lokli>davexunit, well, macro is needed to make printf into 'printf, that for sure, the rest of the work is done by helper functions <lokli>davexunit, oh, ok, not scheme string literal, no. i didn't get it right <mark_weaver>procedural macros could do it via 'datum->syntax', but don't take this as an endorsement of lokli's approach. (I haven't been following closely, but I suspect I agree more with davexunit) <davexunit>mark_weaver: sure, I'm trying to steer clear of some things that are possible but not appropriate, such as reader macros. :) <lokli>well, anyway, strings are in the past, now classes and objects <davexunit>lokli: when you say classes, do you mean GOOPS classes? <davexunit>you're of course free to design how you would like, but I would be hesitant to use object-oriented programming for this. <davexunit>preferring a purely functional style instead. <lokli>davexunit, speaking of shady solutions, is it possible to shadow 'if? like (let ((xx if)) ...) but of course let does not work for this <mark_weaver>yes, reader macros *definitely* should be avoided. they don't compose, and there's no good way to ensure that they will be loaded before the code is read, in general. <lokli>davexunit, i dont like oop in itself, classes give types, thats what i really want <davexunit>lokli: for making new types, I recommend looking at the 'define-record-type' macro in the (srfi srfi-9) module. <lokli>mark_weaver, can i shadow a macro? <davexunit>lokli: the first question I have is: why do you want to redefine what 'if' does? <mark_weaver>lokli: you could do (let-syntax ((xx (syntax-rules () ((_ . operands) (if . operands))))) ...) but the fact that you want this makes me very skeptical of your approach <mark_weaver>proper use of hygienic macros generally makes this kind of thing unnecessary. <mark_weaver>and I agree with davexunit you should avoid OOP unless your problem is really well suited for it, which most things aren't. <lokli>mark_weaver, the thing i am trying to do here is very unlike-scheme, but i really want to cram it into scheme in order to have full macro power <mark_weaver>OOP is massively overused, and I suspect that you have been overly influenced by the OOP-craze that inappropriately takes the position that OOP should be used for every problem. <lokli>fully agree on oop, but method dispatched on type is a very nice feature. One can use oop without inheretance for example <mark_weaver>in fact, stateful objects are far less easy to work with and reason about than raw data. <mark_weaver>raw data can be copied, cached, transferred over the network, etc. these things are very problematic for stateful objects. <davexunit>lokli: I think you can take advantage of macros, but I think it's important to remember that macros provide syntactic sugar. you can design your system to work without it first and get the really important compiler stuff right. then, think about where some syntax sugar makes sense. <mark_weaver>but I don't expect to be able to convince you of this, and I don't have time to try. <lokli>mark_weaver, i never use oop in fact. But i dont feel like scheme gives me enough possibility for typed objects. I will totally check define-record-type for that <davexunit>lokli: I think define-record-type will do what you need. <lokli>thanks for all the feedback and advices, really helps to get a better picture <davexunit>writing source to source compilers in Lisp is a common enough phenomenon because s-expressions are easy to work with. <lokli>maybe there are some study materials around for that? <davexunit>the designers of them first choose how they would like to represent their target language using a quoted list as a syntax tree. <davexunit>in C, you may see this expression "int foo = 1" <davexunit>a language designer might translate that to Lisp like so: <lokli>well, that is easy enough. Troubles you get, when int turns into unsigned long long, so you need to make another list, for type <davexunit>lokli: sure, you have to think about all of these things when designing your language. <davexunit>since there's a fixed number of those modified number types, you could create identifiers for each one: unsigned-long-long, unsigned-long, unsigned-short, etc. <lokli>davexunit, so, in the end what you have is '(define int foo 1), and you actually cant do macro transforms on it, right? <lokli>true to that, didnt think about it <davexunit>lokli: right, that's a list, not Scheme source code. <davexunit>this is not to say that macros cannot be used! <davexunit>you can use all of the facilities that Scheme provides to produce your syntax tree. <lokli>davexunit, i dont get it. Are you gonna parse then quoted list into some other structure? Or are you gonna make transforms on list with list transforming procedures? <davexunit>lokli: that quoted list is the source code that you pass to the compiler to turn into C code. <davexunit>example: (lispy-c->c '(define int foo 1)) might return "int foo = 1;" <lokli>well, that is source to source, right. I find it a bit limiting for now, but will sleep on it, maybe it is the way to go about it <davexunit>you can write Scheme programs that generate your syntax trees. <davexunit>and introduce procedures/macros as you like. <lokli>davexunit, hm, thats true. I see your way more and more clearly <lokli>davexunit, i am still feeling, that some power is going away. You will need to do some extra work. Consider (foo x (bar y z)), where foo and bar and x,y,z all defined. So, if it is all valid scheme, then guile will make all the work for me, calling functions and evaluating arguments. If you do it compiler-way, you will do it by hand. And on the output we both get the string of target language <davexunit>lokli: what do you mean "all defined"? it's a completely different language. ***heroux_ is now known as heroux
<davexunit>if 'foo' was a procedure that returns a value that is valid in your sexp C representation, then you could use it. <lokli>davexunit, they are defined in scheme terms. (define (foo x) (string->append "my-fun" x)) <davexunit>but again I feel like you are conflating various layers of the system. <lokli>is seemt like it to me too =) <davexunit>the above expression refers to two different "foo"s <davexunit>the second is the Scheme procedure called "foo" <lokli>i am smart enough to get quotation :P <lokli>no problem. But it seems not smart enough yet to have clear separation of layers. Not surprising though, i dont have any compiler experience so far <davexunit>fwiw, I don't really know much about compilers, but I do have experience writing s-expression representations of other languages. <lokli>davexunit, do you use record types for that? <davexunit>no, because the types that can be represented in those languages can be represented using built-in types <lokli>davexunit, so, how do you represent result of (@ ...)? <lokli>thats expected, but intermediate representation is what i am after. Before it gets too far, just after @ is executed, what is the output? simples string? <davexunit>but the list is traversed and written as XML <lokli>ok, i need to see the compiler then <davexunit>the cool thing about SXML is that I can write templates using Scheme procedures <davexunit>(define (paragraph text) `(p (@ (class "foo")) ,text)) <lokli>davexunit, yes, that is nice. so, can i see the compiler source? because i would totally make 'p and '@ scheme procedures <davexunit>lokli: no no, the important thing is that they *aren't* Scheme procedures <lokli>davexunit, i get it now. are you ignoring request for the translator from sxml to html? <davexunit>lokli: no, sorry. I believe you'll find that in the sxml->xml procedure in the (sxml simple) module. <lokli>davexunit, checked the source. In fact for such translator it is easy enough to parse, destructure the input list, and run appropriate code for parts. But for the language, that itself has types, and more complicated structure, wont this procedure (csexp->c) will turn into a huge monster? <lokli>but i certainly see the beauty in separation, it is more simple to reason about and implement <davexunit>lokli: did you think writing a compiler would be trivial? <lokli>davexunit, in fact i did! =) <davexunit>at its simplest, you could write a compiler that does nothing fancy at all, that just translates sexps into C code. <lokli>davexunit, that i did for my toy examples. Was not satisfied too much <davexunit>right, it's reasonable to want something cooler! <davexunit>implementing type inference requires some compiler machinery. <lokli>yeah, that seems right. It would be way easier to match sexp, than some hand made ast <davexunit>you have lots of great tools at your disposal for working with sexps <davexunit>there's a pattern matcher available in (ice-9 match) <lokli>davexunit, but i can still manage to use record types in order to have lots of info attached to sexp ast nodes, right? <davexunit>lokli: sure, the nodes of the tree can be any type you'd like. <davexunit>I'm partial to using only simple types for these sorts of things, but you don't have to do the same. <lokli>davexunit, maybe you just saved my project from being a disaster =) Anyway, it is past 2 am for me, time to sleep. Thanks for all the help. But you will most likely now have a newbie asking lots of questions on the topic ;) <b0f>To answer my own question - GNUTLS has a guile API