IRC channel logs

2014-10-25.log

back to list of logs

***fangism is now known as fangism-hongry
<naboo>can someone answer my question? http://lists.gnu.org/archive/html/guile-user/2014-10/msg00035.html
<nalaginrut>naboo: have you put (setlocale LC_ALL "utf-8") at the very beginning of your program?
<rekado_>I want to use guile to write a small XMPP pubsub implementation with SXML.
<rekado_>Now I'm looking for persistence mechanisms in Scheme that would be a good fit for XML/SXML documents.
<rekado_>Is there a list of guile bindings to NoSQL databases somewhere?
<nalaginrut>rekado_: maybe guile-redis?
<naboo>nalaginrut: utf-8 is incorrect locale
<naboo>nalaginrut: says guile
<nalaginrut>naboo: in your case, maybe "UTF-8"
<naboo>nalaginrut: as well as UTF-8
<nalaginrut>naboo: it's related to the name in your "locale -a"
<naboo>nalaginrut: (setlocale LC_ALL "pl_PL.UTF-8") script works but error still occur
<nalaginrut>naboo: did you put it very beginning of you program?
<naboo>nalaginrut: yes
<nalaginrut>hmm...
<naboo>nalaginrut: before modules
<nalaginrut>use-modules?
<naboo>nalaginrut: yep
<naboo>nalaginrut: i mean yes
<nalaginrut>I see
<nalaginrut>I'm using guile-dbd-mysql too, but never encounter the similar problem
<nalaginrut>usually, setlocale magic works
<naboo>nalaginrut: meby my compilation is bad
<rekado_>nalaginrut: thanks, I'll try that
<nalaginrut>rekado_: np ;-)
<nalaginrut>naboo: maybe not, #vu8(80 65 87 69 163) error for me too
<naboo>nalaginrut: hmm
<nalaginrut>(bytevector->string $1 "iso8859-1") works
<nalaginrut>but I don't think it's suit for you
<nalaginrut>anyway I think that bytevector is not utf8
<nalaginrut>maybe mysql is not set correctly?
<naboo>nalaginrut: what you mean?
<nalaginrut>naboo: I mean maybe mysql didn't convert it to utf8?
<naboo>nalaginrut: database is in iso-8859-2 but i use set names utf-8
<naboo>nalaginrut: in terminal it works
<nalaginrut>yes, you set it in terminal, but did you set it with guile-dbi?
<naboo>nalaginrut: yes, with this (db-query "SET NAMES utf8 COLLATE utf8_general_ci")
<naboo>nalaginrut: (db-query "SET NAMES utf8")
<nalaginrut>hmm
<nalaginrut>naboo: ok, I think there's a way for that
<naboo>nalaginrut: i listening
<nalaginrut>naboo: you have to convert it to iso8859-1 then convert it to utf8, works for mr
<nalaginrut>for me
<naboo>nalaginrut: yyyy but how?
<nalaginrut>(string->bytevector (bytevector->string $1 "iso8859-1") "utf8")
<nalaginrut>naboo: you have to import (ice-9 iconv)
<naboo>nalaginrut: ok i will try
<nalaginrut>naboo: well, in your case, maybe just convert it to iso8859-1 is enough
<nalaginrut>but I'm afraid there maybe info was dropped according to encoding..
<naboo>nalaginrut: problem was in other place
<naboo>nalaginrut: i wrote some converter for charset and there was problem
<naboo>nalaginrut: it is not related to dbi so sorry for your time, and thanks for help, now it works fine
<nalaginrut>naboo: no problem, I'm glad to help ;-)
<gjanssens>Hi, I've got a question related to loading compiled scm files
<gjanssens>I have a bunch of scm files
<gjanssens>Some are loaded with use-modules, some with load-from-path
<gjanssens>I have now compiled all these files with guild compile and added the path to the compiled files to the environment variable "GUILE_LOAD_COMPILED_PATH"
<gjanssens>Then cleared the autocompiled files from the default cache dir
<gjanssens>When I now load one of my modules using use-modules, the module is loaded without triggering an autocompile
<gjanssens>However when I load a file with load-from-path this always triggers an autocompile of the file first
<gjanssens>Why is that ?
<gjanssens>The documentation on load-from-file says it should try to load the compiled file if it finds one
<gjanssens>Hmm, it seems to be more complicated than that...
<gjanssens>When I set up a simple test case it works
<gjanssens>It doesn't work when integrated in gnucash
<gjanssens>I'll have to go debug some more...
<gjanssens>Ok, got it: if I want load-from-path to work I must make sure its argument doesn't have a file extension
<gjanssens>To clarify: if I want to load "test.go" which is a compiled version of "test.scm" I have to use (load-from-path "test")
<gjanssens>(load-from-path "test.scm") will always trigger an autocompile
<gjanssens>I have also tried to create a compiled file "test.scm.go", but it's ignored.
<gjanssens>(load-from-path "test.scm") will always trigger an autocompile
<gjanssens>I can work around this, but should this be considered a bug ?
<taylanub>gjanssens: I'd say feel free to file a bug report; at worst it will serve as future documentation.
<wingo>so, ignorant question. what sorts of mutations can i do in parallel in guile without locks?
<wingo>for example can i write to different parts of a bytevector in parallel
<wingo>and if so are there restrictions -- like not bytes in the same word, or what
<wingo>assuming that the program doesn't read from the bytevector until after it's synchronized on all of the writers
<wingo>i suspect mark_weaver probably knows :)
<mark_weaver>hi wingo!
<wingo>heya mark_weaver :)
<mark_weaver>I liked your rant about setting up ssl, btw :)
<wingo>haha thanks :)
<wingo>i wrote most of a jpeg decoder in scheme, for no good reason
<wingo>and am wondering what i can do to parallelize it
<mark_weaver>nice! :)
<wingo>it's a bit silly but i don't want to have libjpeg on my server :)
<wingo>irrational things.
<mark_weaver>it's a good question, and I probably knew at some point, but that information seems to have left me.
<mark_weaver>hmm.
<wingo>well, i'll do this piece serially, then see what happens if i swap in parallel-for-each. not a principled approach but i think the damage if any would be limited to the bytevector.
<wingo>(though is that necessarily true, even? my instincts are always so suspect when it comes to parallelism)
<mark_weaver>yeah, I think the damage would be limited to the relevant cache lines.
<mark_weaver>as I (vaguely) recall, technically the standards say that races lead to undefined behavior, i.e. _anything_ could happen, but of course we know it's not quite *that* bad.
<wingo>right
<mark_weaver>naboo: (80 65 87 69 163) is not valid UTF-8
<nalaginrut>mark_weaver: he said it's the bug of his convert function, so unrelated to anything of Guile or dbi ;-)
<mark_weaver>that last byte (163, or #xA3, or #b10100011) is the problem. since the top two bits are "10", that's supposed to be a "continuation byte", and it should have been preceeded by a byte with at least the top two bits set (indicating the start of a multi-byte sequence).
<mark_weaver>nalaginrut: ah, okay, thanks.
<mark_weaver>gjanssens: the argument you pass to 'load-from-path' should end with ".scm". conceptually, you're loading the source file, and the .go files are just a cache.
<mark_weaver>gjanssens: if 'load-from-path' works and 'use-modules' doesn't, my first guess is that it's because the module has already been loaded. 'use-modules' is a no-op if the given module already exists in the session.
<gjanssens>mark_weaver: it's actually the other way around
<gjanssens>use-modules works fine
<gjanssens>load-from-path name.scm works but will always use an autocompiled .go file, never my self-compiled one
<gjanssens>load-from-path name (without the extension) will use my self-compiled .go file
<gjanssens>So something odd is going on with the extension parsing
<mark_weaver>is the *.go file newer than the *.scm file?
<gjanssens>definitely
<mark_weaver>well, that's the problem.
<gjanssens>How is that ?
<mark_weaver>the reason is works when you leave off the extension is that, for historical reasons, we allow source files with no extension at all, but they map to the same .go file.
<mark_weaver>if the *.go file is newer than the *.scm file, then the .go file is assumed to be fresh, and so it uses that instead of recompiling the *.scm file.
<gjanssens>I understand, and that's what I want
<gjanssens>I *want* to use my self-compiled .go file
<mark_weaver>oh, I see what you mean.
<gjanssens>But I can't if I add an extension in (load-from-path test.scm)
<gjanssens>I'm talking about the same source file test.scm and its (self-compiled) test.go counterpart
<mark_weaver>I understand the problem.
<gjanssens>Ok
<mark_weaver>wingo: do you remember off-hand how this is supposed to work?
<gjanssens>FYI this is with guile 2.0.11 on fedora 20
<nalaginrut>btw, anyone can run 'guild' successfully? not "guild compile"
<nalaginrut>I can't even run guild for guildhall
<mark_weaver>wingo: gjanssens has foo.scm in his load path and foo.go in his compiled load path, but when he tries to load foo.scm with 'load-from-path', it doesn't use the .go file in his compiled load path. it auto-compiles anyway, apparently only looking for the .go file in some auto-generated cache directory.
<mark_weaver>nalaginrut: guild works for me. I suspect it does for most of us.
<nalaginrut>mark_weaver: alright, I'll check the code of master
<nalaginrut>I can't do that under master
<gjanssens>(load-from-path "foo") uses foo.go in the compiled load path
<mark_weaver>gjanssens: right, but that's a hack that would be best avoided.
<gjanssens>Sure. I'm just mentioning to make sure the problem is narrowed down sufficiently
<mark_weaver>gjanssens: what is the value of GUILE_LOAD_COMPILED_PATH, the exact argument to 'load-from-path', and the exact location of the *.go file?
<gjanssens>I run my test from a directory which has a subdirectory "modules"
<wingo>mark_weaver: don't you have to leave off the .scm for that to work?
<gjanssens>Then I start guile as follows:
<wingo>i.e. (load-from-path "foo/bar")
<gjanssens>GUILE_LOAD_PATH=$(pwd)/modules GUILE_LOAD_COMPILED_PATH=$(pwd)/modules guile
<gjanssens>(both test.scm and test.go are in the modules subdirectory)
<wingo>;;; Load a Scheme source file named NAME, searching for it in the
<wingo>;;; directories listed in %load-path, and applying each of the file
<wingo>;;; name extensions listed in %load-extensions.
<wingo>(define (load-from-path name)
<wingo> (start-stack 'load-stack
<mark_weaver>wingo: well, I think that's happening there is that, since we currently accept "no extension" as an extension, it sees "foo" as a source file that maps to "foo.go", and since the source file doesn't exist, it just uses the .go file anyway.
<wingo> (primitive-load-path name)))
<gjanssens>(load-from-path "test") uses my test.go
<gjanssens>(load-from-path "test.scm") will autocompile and use the autogenerated test.scm.go
<mark_weaver>wingo: it seems like it would be better if it looked in the load compiled path for a fresh .go file.
<mark_weaver>and we might want to deprecate (and later remove) the empty extension thing.
<mark_weaver>or no?
<wingo>i don't know, this stuff can get very tricky
<gjanssens>well the file being searched does have a .scm extension
<gjanssens>The question is whether it should be added in load-from-path or not
<wingo>so is the issue that the interface is confusing or that it's doing the wrong thing?
<gjanssens>From an external point of view I would expect load-from-path to work for both "test" and "test.scm" as argument
<mark_weaver>wingo: is the intent of the 'load-from-path' API that the extension should be omitted?
<wingo>if you use load-from-path without specifying an extension, and a file with an extension is found, then a .go file is looked for consisting of the stem (without the .scm extension) plus .go
<wingo>mark_weaver: yes
<wingo>in fact the no-extension case is last in the list
<mark_weaver>ah, okay.
<wingo>s/.go file is looked for/.go file is looked for in the load-compiled path/
<mark_weaver>wingo: okay, sorry, I didn't realize that 'load-from-path' was supposed to add the extensions as well as the directory prefixes.
<mark_weaver>gjanssens: so, I guess you're *supposed* to leave off the extension when using 'load-from-path'. my mistake.
<mark_weaver>sorry about that :-/
<daviid>mark_weaver: will removing empty extension provide sripts to be named like 'bouncer' ? I like to name my scripts without extension, from a user perspective it is a command...
<mark_weaver>daviid: I do the same. Removing the empty extension wouldn't prevent that.
<daviid>ok
<mark_weaver>it would just mean that when loading a module with 'use-modules' or similar, the file would need to end with *.scm
<gjanssens>Ok
<gjanssens>That's what I figured so far
<gjanssens>I could not make that up from the documentation though
<gjanssens> https://www.gnu.org/software/guile/manual/html_node/Load-Paths.html
<gjanssens>It doesn't mention this.
<mark_weaver>yes, I see that the docs for 'load-from-path' are quite misleading, which is why I don't feel quite so embarrassed for not knowing this.
<mark_weaver>we'll have to fix that.
<wingo>i think load also will add extensions
<gjanssens>"When primitive-load-path searches the %load-compiled-path for a corresponding compiled file for a relative path it does so by appending .go to the relative path."
<wingo>...not that that is mentioned either
<gjanssens>This would suggest that is my relative path = test.scm the corresponding go file would be test.scm.go
<gjanssens>But that doesn't work
<wingo>gjanssens: right, that's a doc lack of clarity issue
<wingo>the thing it adds the .go to is the argument to load-from-path
<gjanssens>If I self-compile into test.scm.go from test.scm, and then do (load-from-path "test.scm") it will still autocompile
<gjanssens>Which is different from what you just claim
<gjanssens>Or I misunderstood what you are telling me :(
<wingo>so there are two things
<wingo>one is the load-compiled path
<mark_weaver>wingo: given that the docs for 'load' call the argument "filename", that seems surprising to me. also, I suspect other scheme implementations usually treat the argument to 'load' as a filename.
<wingo>the other is the automatic compilation facility
<wingo>mark_weaver: i think there you'd be wrong :) istr some things will append .so for compiled files, etc
<mark_weaver>but given how muddled this has all become, we may never be able to remove the empty extension :-(
<wingo>mark_weaver: i don't have a lot of experience with other scheme implementations tho
<wingo>gjanssens: so the load-compiled path is used only when loading a file via load-from-path (or primitive-load-path)
<wingo>gjanssens: and in that case we have the stem, and append .go to the stem when looking for the compiled file in the load-compiled path
<wingo>the other case is if we don't find a compiled file and look in the auto-compiled directory
<wingo>this second case can be reached from more places, for example from normal "load"
<mark_weaver>wingo: take a look at the definition of 'load' in R5RS. I think it's fairly clear.
<gjanssens>Sorry if I'm slow, but how should I expect these to work:
<wingo>we don't strip extensions in that case because the important thing is not really to have a pretty name -- the important thing is to have a 1-to-1 mapping between auto-compiled and source files
<wingo>and that includes files without extensions, etc
<mark_weaver>it's clear enough that portable code might reasonable write code to a file and then load from the same file.
<gjanssens>I have test.scm test.go and test.scm.go in my modules directory
<wingo>gjanssens: why do you have test.scm.go in your modules directory?
<gjanssens>Both GUILE_LOAD_PATH and GUILE_LOAD_COMPILED_PATH include this directory
<gjanssens>Just for testing sake
<wingo>hmm :)
<gjanssens>So
<gjanssens>(load-from-path "test") -> uses modules/test.go
<gjanssens>(load-from-path "test.scm") -> autocompiles ignoring modules/test.scm.go
<wingo>that's, um, strange
*mark_weaver goes afk for a few hours
<gjanssens>Indeed :)
<wingo>sounds like a bug unless test.scm.go is out of date
<gjanssens>It really looks inconsistent
<gjanssens>no, I just ran guild compile to generate both .go files
<gjanssens>test.scm hasn't been touched since earlier today when I started my experiments
<wingo>sounds like a bug then
<gjanssens>Should I file one on savannah ?
<gjanssens>Oh, and moving the .go files in an independent ccache directory (and adjusting %load-compiled-path accordingly) doesn't change this
<gjanssens>So there really is some inconsistency in how this is implemented
<gjanssens>Which bugtracker does guile use ?
<gjanssens>I looked at Savannah but the most recent open bug seems to be from 2011
<gjanssens>Or has guile been bugfree ever since ? ;)
<wingo>bug-guile@gnu.org
<wingo>sending a mail there makes a debbugs.gnu.org request
<wingo>see the manual for details on submitting a bug report :)
<wingo>the "reporting bugs" section
<daviid>wingo: while you will be looking at goops, i'd like to remind you of this one as well: https://lists.gnu.org/archive/html/bug-guile/2014-04/msg00015.html
<daviid>this bug makes it mandatory to always redefine setters for all subclasses, uncomprehensible for end users and quite a pain for us :)
<gjanssens>Hmm, The manual tells me to run build-aux/config.guess
<gjanssens>This file is not available in the repository under tag v2.0.11
<gjanssens>Downloaded from the link instead...
<gjanssens>Ok, bug filed
<rekado>Is there an implementation of SHA1 available for guile?
<rekado>or should I just use the FFI to call a C function in some C library?
<daviid>rekado: searh the web for guile-ssh
<daviid>there is a module but i don't erecall where...
<ijp>there might be one in guile-lib
<sneek>Welcome back ijp, you have 1 message.
<sneek>ijp, dsmith-work says: yeah, seemed like a gavino sighting
<rekado>ijp: a search for sha in guile-lib didn't return any results.
<ijp>industria will have one, but I can't remember if it has any weird dependencies
<daviid>rekado: here is the annouce https://www.mail-archive.com/guile-user@gnu.org/msg07116.html
<ijp>rekado: I must have been thinking of the md5 code
<mark_weaver>rekado: weinholt's industria library has a SHA-1 implementation in portable R6RS scheme. It is most easily available for Guile via ijp's guildhall repository.
<mark_weaver>sneek: guildhall?
<sneek>Someone once said guildhall is https://github.com/ijp/guildhall/wiki/Getting-Started
<mark_weaver>the specific module name is (weinholt crypto sha-1)
<mark_weaver>alternatively, you could use Guile's Dynamic FFI to access a C function (e.g. in libgcrypt), as you suggested.
<mark_weaver>rekado: it might be helpful to look at http://git.savannah.gnu.org/cgit/guix.git/tree/guix/hash.scm , which uses libgcrypt to generate sha256 hashes. it could probably be modified to generate SHA-1 with very few changes.
<rekado>@all: thanks.
<taylanub>BTW I'll be slowly making R7RS SRFI libraries; using the reference implementations where I can: https://gitorious.org/taylan-scheme/srfi/