***linas_ is now known as linas
<didi>Is it more common to use `throw' or srfi-35 to signal errors in Guile programs? <didi>davexunit: I'm hesitating because its type is always `misc-error'. <mark_weaver>didi: I must steer you away from Guile's implementation of srfi-35, I'm afraid. <mark_weaver>guile's implementations of both srfi-34 and srfi-35 are very poorly integrated with every other exception-throwing or catching primitive in guile. <mark_weaver>however, if you want to use something like srfi-35, I can recommend R6RS conditions. <didi>mark_weaver: Well, it's settled then. <didi>mark_weaver: I will look into those, thank you. <mark_weaver>at some point, I'll change srfi-34 and srfi-35 to be based upon the R6RS exceptions and conditions in guile. <mark_weaver>didi: well, okay, I have to add a caveat. in current guile git, and in 2.0.10 when it's released soon, rnrs exceptions are well integrated with native guile exceptions. <didi>mark_weaver: Oh, you mean they aren't at 2.0.9? <mark_weaver>well, the issue is that the (rnrs exceptions) catching primitives such as 'guard' can only catch exceptions generated by the (rnrs exceptions) throwers. <mark_weaver>and similarly for srfi-34 and srfi-35. each of these kinds of exceptions lives in their own separate worlds, I'm afraid. <didi>That's fine. I'll then wait for the upcoming release to use rnrs exceptions. <mark_weaver>in guile git, and in 2.0.10, if you catch a native guile exception using something from (rnrs exceptions), then it the native exception will be converted into a condition object automatically. <mark_weaver>fwiw, I think that R6RS conditions is the most nicely designed system for classifying errors that we have in the scheme world. <mark_weaver>so if you want to look to the future, I would base my code on those. <didi>OK, you convinced me. I'll look into it. <mark_weaver>the only thing to be aware of is that native guile exceptions will not be caught by 'guard' or 'with-exception-handler'. <mark_weaver>also, you don't have to wait. in current guile git (both stable-2.0 and master) R6RS exceptions/conditions are fairly well integrated with the native exceptions. <didi>Nah, you're awesome. Anyway, I'm too lazy and spoiled by Debian. ;^) ***ozzloy_ is now known as ozzloy
<Onslauth>Hi guys, I am sorry to bother you again, I have a few questions regarding guile and encoding. <Onslauth>Currently I am using guile-dbi and guile-dbd-mysql to read from a mysql database. <Onslauth>I was busy testing character encodings and I ran into an issue with a utf8 character. <Onslauth>I think the problem might be on the dbd-mysql side, and I cannot seem to set any charset options, I also see that any options after the database location seem to be ignored. <Onslauth>Has anyone done anything similar to this, or can point me in the right directions? <Onslauth>I know the character in question is a utf-8 character, "e2 80 99", and it gets inserted into the database correctly, however when reading it back out, guile then errors out with the following: <Onslauth>(scm_to_stringn cannot convert wide string to output locale 88 #f #f) <Onslauth>Doing a quick 'show variables like 'character_set%' using the guile client, produces the following: <Onslauth>((Variable_name . character_set_results) (Value . latin1)) <Onslauth>That is the complete error message I am receiving <Onslauth>Is there an option to turn on more debugging output? <Onslauth>The computer in question has the following locales: <ijp>guile uses iso 8859-1 by default for IO <Onslauth>Is there a way to get it to handle utf8? <Onslauth>Or must I somehow specify it must decode it to utf8 ? <ijp>Onslauth: see set-port-encoding! and %default-port-encoding <Onslauth>Thanks, is there a way to catch the encoding error and then try to convert? <Onslauth>Heh, how much of a performance hit are we talking about? <ijp>I'm not even talking performance, it's just not sensible <Onslauth>The program starts up, and reads values from a database <ijp>you know the encoding, so set it at the start of the program <Onslauth>And then every so often new values are entered, so its not heavy usage for that <Onslauth>so you say just set it at the beginning? <nalaginrut>setlocale to "utf-8" in the very beginning of you program if you really need non-latin chars <nalaginrut>I guess there's a bug who cut locale name string, but I can't confirm it with the error msg you provided <Onslauth>Is there a way to get further error information? <nalaginrut>I don't know, what's version of your guile-dbi/dbd? <Onslauth>Thats all thats being outputted to stderr <Onslauth>But that error message is from guile itself. <nalaginrut>I think it's OK because I use these two packages in my web framework <Onslauth>And you are reading from mysql databases? <nalaginrut>if I want to use non-latin chars, I'll set locale in the very beginning <Onslauth>Is there a way to get the current locale in guile? <Onslauth>scheme@(guile-user)> (setlocale LC_ALL "") <Onslauth>I think the error is somewhere in the mysql-dbi/dbd <Onslauth>nalaginrut - but doesn't mysql return the results in latin1 then? <Onslauth>Or do you set the results encoding before doing the queries? <Onslauth>Ok, but I can see the error in question. <Onslauth>So the problem is between writing and reading <Onslauth>0000160: 6675 7274 6865 7220 534d 53e2 8099 7320 further SMS...s <nalaginrut>even if your locale is not correct, it won't throw errors, but shows ???? <Onslauth>Now before parsing the values read from the database, I logged them, and I get the following: <Onslauth>0000120: 2e20 416c 6c20 6675 7274 6865 7220 534d . All further SM <Onslauth>0000130: 533f 7320 7769 6c6c 2062 6520 6368 6172 S?s will be char <nalaginrut>Onslauth: try (setlocale LC_ALL "en.US-utf8") in the very beginning of you code <nalaginrut>for you current system, I think it's "en_US.UTF-8" <Onslauth>I'll look into the error message and see what the 88 corresponds too. <Onslauth>Thanks again for all the help nalaginrut <Onslauth>If you ever in SA, you are more than welcome to come for some beer, or whiskey or kaoliang :D <Onslauth>(encoding-error iprin1 cannot convert to output locale 0 #<output: r6rs-bytevector-output-port 1792680> ) <ijp>presumably south africa, because you can't get a beer in .sa (saudi arabia) <Onslauth>nalaginrut - does setlocale set the default port encoding? <mark_weaver>Onslauth: if you haven't set the default port encoding in some other way, then yes. <Onslauth>Is the best solution to set %default-port-encoding at the beginning of the program? <mark_weaver>I recommend calling (setlocale LC_ALL "") at the beginning of a program, unless there's some reason to not do it. <Onslauth>mark_weaver - I have the (setlocale LC_ALL "en_US.UTF-8") <Onslauth>its a program that reads some triggers/templates from a database, and then receives input and sends out sms's <mark_weaver>sure, if you'd prefer to hardcode it, that might be better. just beware that if you call any other programs via subprocesses, they are probably going to set their locale (and encoding) according to the environment variables like LANG, LC_ALL, etc, so in that case you'd better set those too. <Onslauth>now one of the templates has a unicode character in it, and the program was breaking on reading the one template from the database, which using setlocale has now solved <Onslauth>Except that now the sms sending is breaking <Onslauth>2014-02-25T12:40:45 W 07 SMSC-TX: read/write failed (encoding-error iprin1 cannot convert to output locale 0 #<output: r6rs-bytevector-output-port 1792680> ) <mark_weaver>ah, I see. the problem here is that you're writing text to a binary port, which is technically outside of the standard (in R6RS you're simply not allowed to do that). <Onslauth>Ok, and how would I go about solving that? <mark_weaver>in guile, the default port encoding of binary ports is iso-8559-1 <Onslauth>Ok, so set the %default-port-encoding to UTF-8 at the start? <mark_weaver>but in guile, you could set the port encoding of that port after you open it. <mark_weaver>no, that won't be enough. for binary ports, it won't honor that. you have to call 'set-port-encoding!' after you open it. <Onslauth>Ok, perfect. I was wondering why the setlocale wasnt setting it for the port <mark_weaver>the reason we use iso-8559-1 (latin-1) for binary ports is that any arbitrary sequence of bytes has a text mapping in latin-1, so if you read+write using the text I/O procedures, it will always work and be lossless. <Onslauth>mark_weaver - Sorry, I have two more questions from a colleague, if you dont mind? <Onslauth>1. What is the correct way to write binary data to a socket? <Onslauth>If I open a port, and I just want to get raw bytes in an out, what would the encoding be? <mark_weaver>the best way is to use the binary I/O procedures, in which case the encoding doesn't matter. <mark_weaver>e.g. get-us, get-bytevector-n, get-bytevector-n!, get-bytevector-some, get-bytevector-all <Onslauth>He says if you are ever in SA he'll buy you some whiskey <Onslauth>Andrew says thanks so much, everything is ow so clear. <mark_weaver>nalaginrut, Onslauth: btw, I saw it mentioned earlier in the log that (setlocale LC_ALL "") gets the current encoding, but that's incorrect. <mark_weaver>that sets the locale according to the environment variables, and then returns that locale as a canonicalized string. <mark_weaver>to get the locale, you want to leave off that third argument, so somethign like (setlocale LC_ALL) <mark_weaver>but I recommend setting the LC_ALL environment variable to a UTF-8 locale for all the programs that are part of this system, so that everything is talking the same UTF-8 encoding. <Onslauth>Thanks for all the help, really have cleared a lot up today. <didi>IIUC, the way to use rsrn conditions is to create a compound one using a combination of base types to communicate the meaning of the condition, right? <mark_weaver>the idea is that handlers can extract just the parts they understand. <mark_weaver>and there are a set of standardized conditions, but you can add your own too. <didi>I see. So I can raise a condition like (raise (condition (make-assertion-violation) (make-irritants-condition bad-argument))) and the caller can use `condition-irritants' to fetch the `bad-argument'. <Onslauth>What is the equivalent using bytevectors for write-string/partial ? <Onslauth>Or is put-bytevector the equivalent of write-string/partial ? <mark_weaver>although I'm not sure 'equivalent' is quite the right word. <Onslauth>Sorry, not sure if you saw the question or not <mark_weaver>you should transition away from the (ice-9 rw). that's legacy cruft from 1.8 when we didn't have proper i18n support. <mark_weaver>I'm afraid that we don't really support non-blocking I/O at this point. <Onslauth>What would the new functions to use instead of write-string/partial ? <mark_weaver>hmm, I see that the docs claim that write-string/partial works properly with non-blocking ports. perhaps that's true. well, put-bytevector should work as well as write-string/partial did. <mark_weaver>write-string/partial returns a count, and put-bytevector doesn't. *mark_weaver adds to his TODO list. <Onslauth>Sorry, we just working through our code to see what we doing here. <Onslauth>We have a single thread processing several sockets using (select). <Onslauth>If there's a way to do it with non-blocking I/O, then we can go with that. <Onslauth>*If there's a way to do it with blocking I/O, then we can go with that. <mark_weaver>well, for now, I guess the best option is to do (write-string/partial (bytevector->string bv "ISO-8859-1") ...) <mark_weaver>you'll need the (ice-9 iconv) module for 'bytevector->string'. <mark_weaver>but it would be good for most of your code to transition to using bytevectors, and to isolate the use of strings for binary I/O to a few wrappers like this. <mark_weaver>we are planning to change our internal string representation to UTF-8, at which point these procedures from (ice-9 rw) will no longer be nearly as efficient. using strings to represent binary data is just a bad idea. <mark_weaver>I'm sorry I don't have a better answer for you. we'll get it fixed soon. <mark_weaver>the other option is to use multiple threads, but of course then you have to worry about concurrent access to data structures, limiting the number of threads somehow, etc. <Onslauth>Its been a problem as they never specified the encoding and told us it would be 7bit ascii, and then today they went and broke it using unicode cause someone decided to copy/paste. <Onslauth>What we are planning is converting all the incoming data to use bytevectors, convert to utf8 when we need to process the strings against the triggers/templates, and then convert back to bytevectors <Onslauth>with the exception of the SMSC which might have to be UCS-2 <mark_weaver>so the only missing pieces are these low-level non-blocking I/O procedures for bytevectors, which we don't yet have. <Onslauth>when using put-bytevector, will it error out if it cannot write? <Onslauth>And if we put it in a tight select loop? <mark_weaver>if the socket is in non-blocking mode. I'm not sure what it would do. <Onslauth>Ok, the other option is to write a single byte at a time and put it in a select loop <mark_weaver>it would probably raise an error yes, but you wouldn't be able to find out how much was written. <mark_weaver>that would be quite inefficient, but maybe it doesn't matter for your application. <mark_weaver>I think it's probably better to make your own 'write-bytevector/partial' procedure, implemented as I wrote above. <mark_weaver>and hopefully soon we'll have something in core guile that you can transition to. <Onslauth>Sorry, missed the non-blocking reads question <Onslauth>(write-string/partial (bytevector->string bv "ISO-8859-1") ...) <Onslauth>But the port we are writing to has an encoding of UTF-8 set <mark_weaver>write-string/partial ignores the encoding. it always uses ISO-8859-1, and requires that the string has nothing outside of that. <Onslauth>For the non-blocking, if we try to read and it gives whats available back, even if its less than requested, thats fine. <mark_weaver>write-string/partial is really a binary I/O procedure, although it uses strings because that's all that was available in guile 1.8. <Onslauth>Do you mean if that if the bytevector has characters outside of the 8859 encoding, it will raise an encoding-error when trying to write? <mark_weaver>it doesn't make any sense for a bytevector to have characters outside of 8859. it doesn't have characters at all, just bytes. <mark_weaver>anyway, (write-string/partial (bytevector->string bv "ISO-8859-1") ...) will never fail because of that. <mark_weaver>what kind of ports do you have to do non-blocking reads from? <Onslauth>We dont control all the architecture, and they have specified they follow the standards, but they dont so their systems have caused problems. <mark_weaver>anyway, to do non-blocking reads, you have two choices: read one byte at a time using 'get-u8', or read a chunk using 'get-bytevector-some'. <mark_weaver>however, you must avoid calling either of those procedures unless there's at least one byte available. <mark_weaver>in other words, you can call either of those procedures when 'select' tells you there's input available. <Onslauth>Input appears to be OK with this approach. <Onslauth>We could write one byte at a time, i.e. when select tells us the port is writable. <mark_weaver>okay, just make sure the port is unbuffered, or else what will really happen is that you'll put bytes into a buffer and write it as a block when the buffer is full, which could raise an error and data would be lost. <mark_weaver>ah, so you're only reading one byte anyway. so yeah, just use get-u8 <Onslauth>After everything we ahve learnt today we are planning on changing that to get-u8 or get-bytevector-some <Onslauth>To make the port unbuffered, do we do (setvbuf port _IONBUF) <mark_weaver>however, I should say that it will be more efficient to write chunks using (write-string/partial (bytevector->string bv "ISO-8859-1") ...) ***Fuuzetsu is now known as Guest48731
<Onslauth>What representation does Guile use internally for strings? <mark_weaver>currently, we use ISO-8859-1 when possible, or else UTF-32. however, we plan to change it to UTF-8. <mark_weaver>I should mention that "when possible" isn't quite the same as "when it contains only ISO-8859-1", because of the fact that 'substring' (and some others) do copy-on-write, and thus share the underlying buffer. <didi>Are `make-c-struct' eventually garbage collected? <didi>If they are, can I protect them? <mark_weaver>didi: from looking at the code, I think that when the SCM pointer object is garbage collected, then the object will then be free to collect as well. <mark_weaver>so it looks like you need to hold a reference to the SCM pointer object for as long as the object needs to live. <didi>mark_weaver: Righto. As I'm passing them on to C callbacks, I guess I'll have to make a protective hash table. Not a big deal. <mark_weaver>didi: I suppose another option would be to set GC_ALL_INTERIOR_POINTERS=1 in the environment before the garbage collector is initialized. <mark_weaver>though that would make the GC more likely to hold onto garbage (more conservative) and possibly less performant. I'm not sure how much (if any). <didi>mark_weaver: Hum. I prefer not messing with environment variables. <didi>Collecting stuff in hash tables is not an uncommon occurrence to FFI code. <Onslauth>mark_weaver - if you dont mind me asking, where about do you live, what country? <Onslauth>Ok, if we ever around we'll come buy you a beer :D <Onslauth>Thanks for all the help and the answers. Its really made things a lot clearer and easier to deal with. <didi>Just so I know I'm looking at the right place, if I want to protect the stack from blowing up in a callback, `dynamic-wind' is my best bet, right? <didi>mark_weaver: I'm in a callback, i.e. a Scheme procedure called by C, and some error occurs. I can't just unwind the stack because who called the Scheme procedure probably has some state it needs to take care when the callback returns, so I must try my best to return to C. <didi>mark_weaver: OK. Can I make a continuation barrier too? <mark_weaver>yeah, 'with-continuation-barrier' is a good idea too. <mark_weaver>'with-continuation-barrier' is enough by itself, but if you want to handle the errors in another way, then put a catch inside of it. *mark_weaver goes afk for a while <didi>How do I set! a slot of a make-c-struct? <didi>There is a typo at (info "(guile-2.0) Void Pointers and Byte Access"). "(rnrs bytevector)" => "(rnrs bytevectors)". A missing "s". <didi>OK, I think I figured it out. It's a combination of `pointer->bytevector' and `bytevector-uint-set!' in my case. ***Guest48731 is now known as Fuuzetsu
<didi>Does GOOPS have method combination? More specifically, can I write an after `initialize' method? <ijp>it does not have method combination <tupi>didi: see [look for] call-next-method [in the manual], which the way to go for what you want, i think <tupi`>next-method actually [not call-next-method] ***wingo_ is now known as wingo
<stis>Hmm guile-log is pretty heavy. 26500 loc! <stis>hi wingo: I have a question! <stis>is it ok to steal some of the install scripts in the figl repo? <stis>I reused configure.ac Makefile.am etc <stis>for guile-log and guile-syntax-parse <mark_weaver>lloda: why did you use 'int32' and 'uint32' as the types in your fftw bindings, instead of 'int' and 'unsigned-int' ? <wingo>stis: go ahead, you might need to ask daniel hartwig, or you might not, depends on the history <mark_weaver>the header file declares the corresponding arguments as int and unsigned int, so I'd think that you should use 'int' and 'unsigned-int'. *davexunit is going to write an SDL2 wrapper. <davexunit>found a JSON file with the SDL2 spec. now to generate Scheme with it. *mark_weaver goes offline a bit; back soon. <mark_weaver>well, it's free software, isn't it? at the very least he should have the right to use it if he keeps your copyrights on it, no? <stis>Cool anyhow it's open source, but it's nice to give a hint for the real coders location. <stis>This means that the install procedure is pretty much automated right now in guile-log <stis>But I do want help in testing if it builsd on another server then my. <stis>If you do this please do not install it yetjust make if you have not checked for clobbering of namespaces <stis>some kind of advice for name space handling is also appriciated <tupi`>wingo: i also have a Q :) in sqlite3.scm, [line 296 in the copy I have here], you call (dynamic-func "sqlite3_bind_blob" libsqlite3), but I think it should be (dynamic-func "sqlite3_bind_double" libsqlite3) <wingo>i wonder how to allow other people to commit to that project, do i have to make a new team or something in gitorious? <tupi`>i even forgot where originally holded actually, i took a copy and sent you once a patch ... <tupi`>wingo: [and mark possibly] they is a problem that i don't know how to solve it: floats coming from sqlite3 lands in guile with extra decimals <tupi`>here is an example [give a sec and i'll paste it] <wingo>i think i tried to make a group but failed somehow, humm <stis>exiting times guy's with all wingomagics and flying bullets etc etc, but I have to go now, cheers! <tupi`>in an sqlite3 terminal: select duration from kise where who = 'bryony'; -> 0.3 <wingo>tupi`: do you have a gitorious user? <tupi`>(db-kise/select-some "who = 'bryony'" #f) -> ... 0.30000000000000004 "f" ... <tupi`>so, i thought that the only way [maybe?] is to allow sqlite-row [and callers] to accept a function [as a key arg maybe] so that we could force a round operation at row build time <tupi`>if not, i'll have to process rows twice ... <wingo>tupi`: want commit access to sqlite3 to fix this bug? <davexunit>if I redefine a variable at the REPL, does that block other threads from accessing it until the assigment has finished or could there be a threading issue? <tupi`>wingo: if it helps, surely could, i know you're busy, more and more ... and as usual would not commit without askig first ... <wingo>davexunit: in theory there could be issues. in practice on x86 i don't know that there are any <tupi`>[gitorious is where i hold kisê...] <tupi`>ok will do it but maybe not today, if that's ok <tupi`>the float stuff is guile, not sqlite3, i thibk. and i'd love to get some advice on what people think is the best aproach to solve the 'problem' <lloda`>mark_weaver: I looked up the foreign type list and I didn't see int, if you can believe it. Thanks for noticing, I'll fix.