<didi>Is there some mechanism in the FFI that converts strings to arrays of chars using C functions declarations? e.g. (pointer->procedure void "foo" (list string)) <didi>davexunit: Right, right. But I want to call the procedure with a string as an argument. <didi>(foo "bar") and not (foo (string->pointer "bar")). <davexunit>so write a wrapper function that does the conversion? <davexunit>if foo calls C code, then you need to do some sort of transformation. ***haroldwu_ is now known as haroldwu
<Televangelor>In what way can one run a command and capture the output of that command in guile? <davexunit>Televangelor: what do you mean by "capture the output" <davexunit>write to a file? display on screen? assign to a variable? <davexunit>for what you're trying to do there, you probably want a macro instead. <didi>davexunit: Macros, bad. Procedures, gud. :^P <davexunit>well they're good when you're generating code! which it looks like you're trying to do. <didi>davexunit: Nah, (lambda args ...) take care of it for me. <didi>davexunit: You are probably right tho. <didi>davexunit: Hum, `pointer->procedure' looks like a procedure to me. I wonder how does it generate a lambda with a fixed number of arguments. <didi>How strange. I'm not finding the definition of `pointer->procedure'. <didi>Ah, it's a naughty C function. <didi>AFAICS it's a procedure but, as it's implemented in C, it can output another procedure with the right amount of parameters. <mark_weaver>Televangelor: I guess you probably want 'open-output-pipe' from (ice-9 popen) <mark_weaver>Televangelor: sorry, I meant to say 'open-input-pipe'. <mark_weaver>though it's more robust to avoid the shell (with all its complications of proper quoting of arguments) and instead use 'open-pipe*' with OPEN_READ as its first argument. <mark_weaver>Madsy: if you were going to do another test of Guile on Windows, you might want to hold off a bit. I'm about to add more gnulib modules and make more things work on Windows. <didi>Crude. I need to use `syntax-case', don't I? <mark_weaver>I'm pretty sure this can be done with syntax-rules macros <mark_weaver>it's a little tricky though, best done with the help of 'ck'. <mark_weaver>recent versions of guile include 'ck' in (system base ck) <mark_weaver>the basic problem you have to confront is that macros are evaluated from the outside in, and they are only evaluated in places where expressions are expected. <mark_weaver>although it's small enough that it wouldn't be a problem to include it in your own code. <mark_weaver>in this caes, it might be easier to just use syntax-case. <didi>mark_weaver: I'm still considering if it's worth the effort just to avoid a rest parameter. <mark_weaver>well, there's also the fact that you're doing more work at each call than you need to. <mark_weaver>your wrappers are not as efficient as they could be. <didi>mark_weaver: I can pre-compute more stuff. <mark_weaver>didi: use ,expand at the REPL with various things to see what it expands into. <didi>mark_weaver: Very cool. Thank you. <mark_weaver>I haven't actually tested it, other than to look at the expansions... <mark_weaver>btw, this macro assumes that the list of arg-types will not have a quote in front of it. instead it should just be a parenthesized list of expressions, e.g. (int string '*) <mark_weaver>for example, try: ,expand (pointer->procedure* int blah (int string '*)) <didi>Ah, `generate-temporaries'. <mark_weaver>one thing: I haven't actually evaluated the GC-safety of this, and I wonder about it. <mark_weaver>well, I think the string could be freed when the SCM foreign-pointer object is no longer accessible. <mark_weaver>and it's not clear to me that the SCM foreign-pointer object returned by 'string->pointer' will necessarily stay alive until the C function returns. <didi>mark_weaver: I've been worrying about it lately. <didi>AFAICT, Guile's FFI philosophy is to not let others alloc/free memory themselves. <didi>So sometimes I don't know for how long something will be available. <didi>But your remark about `string->pointer' worries me. I thought it would last at last until we return from C. <mark_weaver>I could do the research, but he probably already knows without having to research it. <didi>I've been happily passing pointers to strings to C functions. <didi>Maybe I might need to wrap them with `let's. <didi>(let ((string-ptr (string->pointer "foo"))) (call-c string-ptr)) <mark_weaver>well, 'let' won't necessarily solve the problem, if indeed there is a problem at all (i'm not sure) <mark_weaver>my impression is that the code you just wrote with 'let' is likely to be optimized into the current code anyway. <nalaginrut>alas, after reading this article, I'm considering to drop ORM to implement a new SQL-mapping, let alone ORM took me long painful time to write too... <mark_weaver>at present, I believe that arguments to procedures are kept alive until the procedure returns, but I don't know if that's guaranteed to remain the case in future guile or not. <didi>mark_weaver: I wonder what would be the solution then. <didi>Also, there is the problem of C functions wanting strings but refusing to copy it, so one has to hold a reference to it as long as the C library demands. <mark_weaver>well, the solution that comes to mind for me is to pass the objects that you want to keep alive to a top-level dummy procedure that does nothing. <mark_weaver>well, yes. maybe we should add a core procedure for this purpose. <didi>We would be easier with we could alloc memory. <didi>`foreign-alloc' and `foreign-free'. <mark_weaver>what about exceptions? and first-class continuations? <didi>I don't know enough to know how these impact allocation of memory. <didi>But as a general rule, FFIs suck. <mark_weaver>well, if an exception in thrown, you'd want to free this object. <didi>I still don't understand. <mark_weaver>if an exception in thrown between 'foreign-alloc' and 'foreign-free', then 'foreign-free' would never be called. <didi>`foreign-alloc' would probably allocate memory outside GC reaches. <mark_weaver>I think this is so obvious that I don't even know how to clarify. <mark_weaver>(let ((x (foreign-alloc ...))) (call-c x) (foreign-free x)) <mark_weaver>I think that calling some (keep-alive x) at the end is better, for several reasons including this. <didi>Well, I'm up for any solution, really. The allocation of memory is the one I'm familiar with from CFFI. <mark_weaver>it also means that if the continuation is captured somewhere, and a reference to 'x' still exists, that the memory won't be freed. <mark_weaver>anyway, maybe it's better that we should just decide to guarantee that if you pass a foreign-pointer object to an FFI-wrapped C function, that the pointer will stay alive until the C function returns. <didi>mark_weaver: That's a start. But it's not sufficient. <mark_weaver>and we could decide to make that a promise going forward. <mark_weaver>if you need the string to stay alive longer than that, then you need to keep a reference somewhere else, like in a global hash table. <didi>Which is not ideal. But FFIs aren't either, so. <mark_weaver>I don't think we should keep these objects alive forever by default, because that's usually not needed and would be a major space leak. <nalaginrut>mark_weaver: isn't dynamic-wind useful in that memory-leak case? or it can't catch the proper continuation at this situation? <mark_weaver>btw, I want to again point out that if you configure libgc to handle interior pointers, then all these problems go away and are handled most satisfactorily. <didi>Crashes are an uglier problem than memory leaks, but we can crash FFIs without the help of the GC anyway. <mark_weaver>the actual 'char *' pointer to the string data, or the C pointer to a 'make-c-string' object, are interior pointers. <mark_weaver>if libgc is configured to detect those, then it will keep those objects alive regardless. <mark_weaver>well, actually I'm not 100% of that either, I confess. I don't know off hand the set of things that libgc scans. <mark_weaver>maybe we should continue this discussion on guile-devel? <didi>Well, I don't think I know enough to sustain a discussion, really. <mark_weaver>I don't actually mix Scheme and foreign C code all that much. The C code that I do write and use with Scheme are typically specifically written to work nicely with GC, so I don't face these nasty impedance mismatches. <didi>That's *precisely* the money quote: "nasty impedance mismatches" <didi>I'm doing some experimentation with GTK+. <Televangelor>mark_weaver, how would one go about executing a command in guile and capturing its stdout in a string? <didi>Televangelor: `with-output-to-string' <mark_weaver>Televangelor: as I said, use 'open-input-pipe', or better yet 'open-pipe*' with OPEN_READ as the first argument. <mark_weaver>'with-output-to-string' won't actually work here, because a subprocess needs a proper POSIX file descriptor as its stdout, and a Guile string port isn't backed by a real POSIX file descriptor. <mark_weaver>Televangelor: both of those return a port that you can read the output from. <mark_weaver>when you're done, use 'close-pipe' on the returned value to close the pipe and shutdown the subprocess. <mark_weaver>for robust error handling, check the exit value with something like: (unless (zero? (status:exit-val (close-pipe port))) (error "subprocess didn't exit cleanly")) <Televangelor>so open-pipe* returns a port? the docs don't seem to clear on that <mark_weaver>to read everything from the command as a string, you could use 'read-string' from (ice-9 rdelim) <mark_weaver>you could also use 'read-line' if appropriate, which has the benefit of stripping the line terminator by default (sometimes desirable). <mark_weaver>I should mention that 'read-string' was added in Guile 2.0.9 (the latest release). 'read-line' is much older. <mark_weaver>didi: I agree that we could do better here, but fwiw, making that work as expected would require creating a new pipe to act as a bridge between the subprocess stdout and the internal string port, launching a thread to copy the data from the pipe to the string, and arranging to shut down this pipe and thread at the appropriate time. <mark_weaver>maybe some day we'll have a much more elaborate subprocess module, but it's a big job. <Televangelor>mark_weaver, well, r6 supports get-string-all which seems to work in guile. <didi>mark_weaver: oic. Well, it's "surprising" but not much more than that. You already presented another way of accomplishing the task. <mark_weaver>Televangelor: that's true, and we've had that for much longer. <Televangelor>Honestly, I think read-line is a bit of awkward nomenclature, 'read' to me as always implied parsing external repraesentations of data <Televangelor>read-line would imply 'get all the data on one line as values <mark_weaver>I often consider switching to mostly R6RS code, and importing selected procedures from (guile) and elsewhere when needed. <Televangelor>Scheme nomenclature in general shows how old the language is, it's almost like English, it evolved enough to no longer make sense. :") <Televangelor>I like how r6 calls it list-sort instead of simply sort for sorting on lists seemingly breaking the tradition of not giving any function operating on lists the name of it in front of it. <didi>Televangelor: I would call it `sort-list'. <Televangelor>r6 + srfi + whatever modules you want gets you quite far and portably so. <Televangelor>I really wouldn't mind if you'd just use list:ref and vector:ref where list: and vector: are praefixes. <mark_weaver>given how often lists are used, I'd find it a bit too verbose to prefix all of those procedures with "list:". <mark_weaver>but I agree that the nomenclature is not very consistent. <nalaginrut>Televangelor: I think the name 'read-line' is to avoid conflict to GNU Readline involved in Guile <Televangelor>mark_weaver, well, you can choose to import the list libraries and not praefix them. <mark_weaver>indeed, we can all make that choice independently :) <Televangelor>This is very common in OCaml, open up the list stuff so map is for lists but you get Vector.map for vectors, unless you're mostly working with vectors. <Televangelor>I've honestly dabbled into the idea from time to time to completely re-write a feasible standard library from scratch which does away with most of the errors of the past <Televangelor>The single best thing about is of course that it currently has [<function> <= <expression>] clauses in conds rather than just [<expression> => <function>] <mark_weaver>I can sympathize with your desire to clean up everything, but I suggest that you resist that temptation :) <mark_weaver>it's bad enough that we've split into R6RS and R7RS camps. <Televangelor>Yeah, the reason I'll probably never finish it is because no one will use it and they'd be wise to do so. <Televangelor>R6 added a module system over R5, there was a big need for portable modules but R7 seems to just be "new standard, because we can" <mark_weaver>well, it depends what you're doing. most programmers want more than R5 + modules, and if there's no standard for the extra stuff then those programs end up being non-portable. <mark_weaver>well, quite a bit of portable R6RS code has been written. <mark_weaver>but it's true that most people don't bother trying to make their code portable. <mark_weaver>anyway, it's worth pointing out that many other languages have only one popular implementation, and yet they don't get bashed as much as Scheme does. <zRecursive>It seems it is easy for a program to support different impls in CL. But it is VERY hard in Scheme <mark_weaver>that's true. CL is a much more comprehensive standard. <mark_weaver>but why doesn't Ruby get bashed for being non-portable? <mark_weaver>somehow, the fact that there are tons of Scheme implementations is counted against us. <mark_weaver>okay, so if you are content with having just one implementation, then forget about "Scheme" and instead think about "Guile" or "Racket". <mark_weaver>well, if you're concerned about that, I'd recommend writing your code mostly for R6RS and the SRFIs, and limit your use of Guile-specific interfaces where needed. <Televangelor>mark_weaver, I feel srfi provides a better solution to standard libraries than a core operational standard like R5 <Televangelor>I wouldn't mind if Rx just gave the bare minimal operational semantics not even including things like map, but that's sor tof what R7 small is trying to do anyway <Televangelor>zRecursive, you have some small options to leave guile, you don't have an option to leave the one ruby implementation at all. <Televangelor>That said, I write my code portyably enough to be able to run it on multiple different implementations <mark_weaver>Televangelor: if you prefer R7RS, then you might be interested to know that we have a mostly complete implementation of R7RS-small in a branch in the git repo, and hopefully it will be in Guile 2.0.11. <Televangelor>Currently testing it on racket and eventually compiling it on larceny or ikarus works fine for me, r6's portable module system really helped <mark_weaver>zRecursive: also, I think you'd find that thanks to modern scheme macros, it wouldn't be very hard to implement whatever Guile features you need in terms of some other modern Scheme implementation. <Televangelor>And just put the non portable small parts into differnet module load paths <didi>mark_weaver: Thanks for the macro again. I'm growing new ugly mutated expressions on it. <mark_weaver>didi: fwiw, I'm going to propose that we guarantee that any SCM foreign-pointer passed to an FFI-wrapped C function be guaranteed to be kept alive until the C function returns, and I'm reasonably sure that this has always been the case so far in Guile. <didi>mark_weaver: Cool. Thank you. <didi>mark_weaver: I think it's fundamental for a FFI. <mark_weaver>I half expect that the response to this proposal will be "well, duh, of course :)" <mark_weaver>but we should probably make it more clear in the documentation, at least. <didi>After that we can work forward for a way for keeping it longer. Some C functions expect "buffers" to leave for long periods of time. <mark_weaver>didi: in cases where the buffer should be kept alive longer, I would prefer to hold a reference to it during that time instead of explicitly freeing. <didi>mark_weaver: Sure. I was concerned about you "optimization" remark. <mark_weaver>yeah, a proper solution will involve some way to avoid any possibility of the variable being optimized out in the future. <mark_weaver>but for the foreseeable future, adding the object to a mutable data structure bound to a module top-level variable should be fine. <mark_weaver>it's hard for me to imagine that Guile could ever optimize away such a thing, given the existing body of Guile code. <mark_weaver>but I agree that we should have primitives that make this more explicit. <didi>mark_weaver: Incidentally, how would you expand that macro to call (pointer->string (proc ...)) if `return-type' is `string'? <didi>Also, will the C string be freed eventually? <mark_weaver>yes, it will be freed when the foreign-pointer object is no longer reachable. <mark_weaver>oh, do you mean the C string passed to 'pointer->string' ? <didi>So I need to pointer->procedure the "free" C function. <mark_weaver>'pointer->string' doesn't (and shouldn't) assume that the pointer passed to it was allocated using malloc. <didi>That's what I was getting at. <didi>See? Memory leaks even without `foreign-alloc' and continuations. <didi>mark_weaver: Ah, so the symbol `proc' remains untouched. <didi>I was worried about hygiene (funny). <mark_weaver>The 'proc' identifier is introduced by this macro. It will be a unique identifier, but all occurrences of it generated by this macro invocation will be the same. <didi>Oh right. It's called "identifier" inside macros. <mark_weaver>it won't conflict with any other 'proc' elsewhere, or even a 'proc' introduced by this same macro but a different invocation of it. <mark_weaver>in practice, the hygiene just works automatically, unless you specifically work to avoid it. <didi>mark_weaver: I thought I had to do the `it' trick from the manual. <mark_weaver>an identifier is more than just a symbol. it includes information about the lexical environment where it occurs, as well as a set of marks that is unique for each macro invocation. <didi>It goes without saying that I have a very crude understanding of macros. <mark_weaver>yeah, understanding macros deeply (enough to implement them properly) is one of the most difficult parts of scheme. <mark_weaver>but the good news is that in most cases, they can be written and used properly without understanding them, and the hygiene just works automatically. <didi>Now I'll expand it to free the `char *' returned. <Televangelor>mark_weaver, scheme makes it easier than most languages though. <mark_weaver>didi: fwiw, I'd recommend making a new procedure that works the same as 'pointer->string' but frees the pointer, and then just changing the macro to use that new procedure. <mark_weaver>remember that you are generating code here, so it's best to minimize the size of the generated code. <didi>mark_weaver: Sleep tight. <didi>Can I generate a keyword with a macro? Like 'foo => #:foo <mark_weaver>yes, it can be done. what are you trying to do exactly? convert a symbol in the macro input into a keyword in the macro output? <didi>mark_weaver: Yes. Specifically, I'm writing a GOOPS slot definition and I want the macro to write a `#init-keyword' keyword for me. I found `symbol->keyword' but I don't know if it's the answer yet. <mark_weaver>well, if 'id' is the pattern variable that matches the identifier, then you want something like (with-syntax ((kw (datum->syntax #'id (symbol->keyword (syntax->datum #'id))))) ...) <didi>`syntax-case' keeps haunting me. <mark_weaver>it takes some getting used to. these macros are a bit trickier than most because they are not merely copying/rearranging their input. <mark_weaver>I guess you'll have more than one 'kw' and more than one 'id', so it'll be more like (with-syntax (((kw ...) (map (lambda (x) (datum->syntax x (symbol->keyword (syntax->datum x)))) #'(id ...)))) ...) <mark_weaver>that last '...' is actually where you fill in code, not a real ellipsis. <didi>Hum. GOOPS doesn't like my macro as one of the slots of a `define-class'. <mark_weaver>didi: macros are only expanded when they are in expression position. <tupi>didi: may i ask what you are trying to do? why your slot(def) has to be the result of a macro? <didi>I'll try writing my own `define-class'. It would be better to use metaclasses but I don't understand them. <didi>tupi: Because the slot definition is pretty regular and long. And I'm lazy. <tupi>i could then create a class that has this regurlar and long slot name and inherit it ... <didi>tupi: Oh, there are many of these slot definitions. In many classes. But they all amount to an identifier, a getter and a setter. <mark_weaver>didi: it would be much better to use metaclasses to access the gobject information directly, like guile-gnome does. <didi>mark_weaver: Yeah. I agree. <didi>mark_weaver: Nothing in particular. Like I said, I'm just experimenting with it. <tupi>mark_weaver: speaking og guile-gnome, yesterday, for the first time, glade3 3.6.7, the latest version that reads/writes *.glade file, required by our guile-gnome-2, systamatically seg fault. so unless i would have access to an old machine, i can not evolve my app [kisê for example], which is kind of a serious [like a disaster :)] problem <tupi>[it seg fault because what glade 3.6.7 'needs' is not part of the lower level lib(s) any more... ] <mark_weaver>do you have any idea what changed on your system that might have broken glade3? <tupi>not really, things removed in the gtk2 libs probably <tupi>but as i already wrote quite a lot :), we really really need guile-gnome-3 <mark_weaver>I know. I'm just overwhelmed with important things to do. we need more help. <tupi>i know. but it becomes really a problem now: i can not edit kise's glade file, it is kind of a disaster, which needs 'urgent' measures, i think. i understand that guile core ... but this will never ever end, it's been a couple of years now that i ask [even contributing financially], so i am very concerned <mark_weaver>tupi: what system are you running? (distro, version) <mark_weaver>well, wheezy will be supported for a while longer, so I guess that's your best option at the moment. but I agree that it's becoming increasingly critical. <tupi>mark_weaver: how about working on this after 2.0.10 ? <tupi>otherwise it will never be done, i think <mark_weaver>I'll think it over. I have to go offline for a bit. ttyl! ***linas__ is now known as linas
***cluck` is now known as cluck
<didi>Oh wait. It /does/ work. *sigh* I suck at it even more. <didi>It's as if `define-class' expands before `my-slot'. <turbofail>that's not define-class's fault, i'm pretty sure that's just how macros work <didi>turbofail: Can't a change the flow somehow? <turbofail>well if you call my-slot as a function on syntax instead, that might work <didi>You mean, turn `my-slot' into a function? <didi>Same error. I /think/ `define-class' is not letting `(my-foo id)' eval, so it tries to interpret it as a slot expression. <didi>#` and #, to the rescue. <didi>turbofail: Cool, your idea worked. Thank you. <didi>Good ol' `(...) expressions. <turbofail>an alternative for my-slot could be (lambda (x) (syntax-case x () (id #'(id #:init-value 42)))) <turbofail>you'd probably want it to use the syntax-case destructuring if you were doing something more complicated with it <didi>turbofail: Cool. Thank you. .oO( search for "syntax-case destructuring" ) ***u_l-lap is now known as unknown_lamer
<taylanub>didi: Destructuring is what syntax-case templates do, like when you have input (foo (bar) baz) and in your template you do (x y z), you get x=foo, y=(bar), z=baz <taylanub>didi: The `match' macro from (ice-9 match) can do the same thing on arbitrary data. In Common Lisp it's called `destructuring-bind', which might be where the term comes from. <mark_weaver>my-slot shouldn't be a macro, given the way you're using it. it should just be a normal procedure. <didi>We can use `with-syntax' out of `syntax-case'? o.O <mark_weaver>(define (my-slot id) (with-syntax ((kw (datum->syntax id (symbol->keyword (syntax->datum id)))) (id id)) #'(list id #:init-value 42 #:init-keyword kw))) <mark_weaver>you just haven't taken the time to study syntax-case macros. you're operating in trial-by-error mode. <didi>mark_weaver: Nice. Thank you.