IRC channel logs

2014-03-11.log

back to list of logs

<parolang>Hello. Trying to get going with guile but have a question about ,apropos at the REPL. Documentation says I can use to find bindings/modules/packages. But it seems it can't find a module unless it's been loaded, or am I just not understanding it's syntax?
<parolang>Also ,binding just returns: %module-public-interface #<variable f9ae20 value: #<interface (guile-user) f95cf0>>
<parolang>I can use the info manual to get by, but I was hoping to be able to explore what was available through the repl.
<nalaginrut>morning guilers~
<lloda>hello, the guile docs give this line for autoconf,
<lloda>PKG_CHECK_MODULES([GUILE], [guile-2.0])
<lloda>but on one computer I have guile-2.2 instead. How do I write this so that it works with either?
<nalaginrut>Can I change the bindings in an arbitrary closure from outside?
<taylanub>nalaginrut: No, that would inhibit optimization even more.
<taylanub>Currently procedures can be said to be our atomic/opaque compilation units, so their insides can be optimized as much as we want as long as they behave the same. The same can't be said for modules for example, whose even private variables can be changed from the outside.
<nalaginrut>taylanub: thanks for explain! In spite of optimization, is there any possible to do so?
<taylanub>Maybe by messing with the generated object code or so, I don't know :) In any case it would be a very unusual thing to do, it wouldn't work well across Guile versions, the binding you're looking for might have been deleted entirely because all uses of it have been inlined, etc. etc.
<nalaginrut>ok, I think there's no generic way to do so, the variable lacks of definition maybe eliminated during compilation.
<nalaginrut>anyway, I give up for it
<taylanub>Remember that you can always use parameters, boxes, cons cells, etc., those would probably be the correct way to do it
<nalaginrut>yes I can do that, but I'm thinking a way to define something for users in the procedure they passed in. But it's just something sugar and unnecessary
<wingo>moin
<ijp>morning
<ArneBab_>moin
<ArneBab_>nalaginrut: sugar is all the difference between Guile and a Turing Tape ☺
<Fuuzetsu>This is a bit like saying that all Turing complete languages are the same, just with different syntax.
<nalaginrut>ArneBab_: some people will love the sugar I just added to artanis, it's convenient for building web
<nalaginrut>but anyway, we shouldn't eat too much sugar...
<ijp>...you'll get cancer of the semi-colon
<nalaginrut>fortunately, I'm not trying to change syntax of Scheme, which is dangerous because it's addictable
<nalaginrut>just some magic of function calling...
<ijp>(perlism epigram #3 http://www.cs.yale.edu/quotes.html)
<nalaginrut>yes I've read Perlis ;-)
<nalaginrut>I have to go home for supper, see you guys tomorrow
<ArneBab_>nalaginrut: :)
<ArneBab_>cu
<Onslauth>Hi guys
<Onslauth>Me again with one more hard to track down problem.
<mark_weaver>hi Onslauth
<Onslauth>I have a memory leak, and its not a fast one
<Onslauth>hey mark_weaver
<Onslauth>I came to ask what the best way was to track down a memory leak.
<wingo>do you use weak tables?
<wingo>is it gc-managed memory or some other memory?
<wingo>do you use c extensions?
<wingo>Onslauth: ^
<Onslauth>The program is exclusively in guile
<Onslauth>it uses guile-dbi and guile-dbd-mysql
<Onslauth>to read nad write to mysql database.
<ijp>are you secretly ferreting memory to the chinese via the secret guile backdoor?
<Onslauth>Otherwise it reads in files, and accepts connections on socket, and then sends values out an another socket. It also writes files.
<mark_weaver>haha
*wingo orders more icepicks
<wingo>Onslauth: do you use weak tables?
<Onslauth>So it will be gc managed memory I believe
<Onslauth>I'll have to ask, as I didnt write all of it myself.
<Onslauth>If there are weak tables, is this a known issue?
<wingo>be sure if you have a weak-key table that the values don't reference the keys
<mark_weaver>iirc, Onslauth is the one who hit the assertion failure in hashtab.c regarding weak tables.
<Onslauth>Yeah, thats never happened again by the way mark
<wingo>Onslauth: where is the source to guile-dbd-mysql?
<Onslauth> http://home.gna.org/guile-dbi/
<mark_weaver> http://bugs.gnu.org/16641
<Onslauth>Yeah, thats it
<Onslauth>We havent hit that assertion again
<mark_weaver>Onslauth: that's good to hear. I think the weak tables implementation in 2.0 has some flaws, but the one in 2.2 will be a lot better. hopefully the problem hits rarely enough that you can wait until 2.2.
<Onslauth>Yeah, hopefully, and its also only exhibited on a test computer, never in production
<mark_weaver>(well, the improvements are already in guile git (master branch), but I wouldn't recommend running it in production yet.
<mark_weaver>)
<Onslauth>Also, one other question, is there any chance that guile would just exit
<Onslauth>no error, no core, no anything, just exit?
<wingo> http://svn.gna.org/viewcvs/*checkout*/guile-dbi/trunk/guile-dbd-mysql/src/guile-dbd-mysql.c?revision=147
<wingo> if(scm_equal_p(scm_string_p(dbh->constr), SCM_BOOL_F) == SCM_BOOL_T)
<wingo> is gross ;)
<mark_weaver>ick
<ijp>what is this I don't even
<Onslauth>I tested the mysql code
<Onslauth>created a select test and left it overnight, no memory loss
<Onslauth>did a insert, select, write file as well, left it overnight and they all stabilise
<Onslauth>I was writing timestamps into mysql, but the strings in question werent more than say 40 bytes total.
<wingo>there are small leaks in __mysql_make_g_db_handle(gdbi_db_handle_t* dbh)
<taylanub>Since I never used the Guile C API, what would be the correct version of the above ? Using scm_false_p() or so ?
<Onslauth>I dont know if there is some limit where the string creation changes if its over a certain length?
<wingo>the sretn values
<wingo>taylanub: scm_is_string
<taylanub>heh
<mark_weaver>is this case: if (!scm_is_string(dbh->constr))
<Onslauth>as in sretn isnt cleaned up?
<Onslauth>Ok, maybe I should provide more context
<Onslauth>the process will startup, and connect to mysql via a unix socket, and only ever connect on startup.
<Onslauth>The db connection is held and never closed.
<wingo>other than those things, guile-dbi and guile-dbd-mysql looked fine to me, memory-wise
<wingo>i would look for weak tables or object properties or something to see if you can find a value referencing a key
<wingo>i would also check (gc-stats), printing them out every so often
<Onslauth>Ok, thanks.
<Onslauth>Will try the above.
<wingo>depending on fragmentation it is possible for memory use to rise over time, to some constant level
<wingo>sometimes it's difficult to distinguish fragmentation effects from memory leaks
<wingo>but if it's a constant-rate rise it's usually a memory leak
<Onslauth>Yeah, its not a lot
<Onslauth>its really minimal, but will check it out
<Onslauth>I mean, the computer itself has something like 160GB of ram
<wingo>you could try valgrind's massif, but that's going down the rabbit hole
<wingo>160 GB!
<Onslauth>and with the rate of loss, in 10 years it wouldnt have reached even 2GB yet
<wingo>how much is "minimal" to you?
<Onslauth>well its only at 60MB now, and I think the output says its losing 4KB every couple of hours or so
<mark_weaver>davexunit: can you resubmit the 'hash-for-each' patch with just the small formatting change we discussed? (line break before the operator)
<mark_weaver>wow, that's a lot of RAM :)
<Onslauth>Yeah, the output is in KB
<Onslauth>yeah, the cell operators like to overspend on RAM.
*mark_weaver goes afk for a while
<Onslauth>Thanks for the help, i'll try see what I can find.
<davexunit>mark_weaver: yes, I can. I won't be back at my own computer until late tonight but I'll take care of it.
<Onslauth>Are weak tables only created via make-weak-* ?
<ijp>object properties make them too
<Onslauth>which uses make-object-property?
<ijp>yes
<Onslauth>Yeah, doesnt seem to have been used at all
<Onslauth>Any other possible ways?
<wingo>Onslauth: you are using the latest guile 2.0, no?
<Onslauth>2.0.9 i believe is the version
<Onslauth>You said there was a 2.0.10 coming out sometime soon?
<wingo>Onslauth: i think that covers most common weak table uses
<Onslauth>So if I dont use make-weak and make-object-property, they shouldnt be being created?
<wingo>correct
<wingo>afaik :)
<mark_weaver>Onslauth: looking more at http://svn.gna.org/viewcvs/*checkout*/guile-dbi/trunk/guile-dbd-mysql/src/guile-dbd-mysql.c?revision=147 I see a leak that might be the one hitting you.
<Onslauth>Where about?
<mark_weaver>nevermind, I was mistaken.
<mark_weaver>sorry for the noise.
<Onslauth>No worries
<cluck>:)
***Molkolev_Zed is now known as FaxIsTheFuture
***linas__ is now known as linas
<mark_weaver>I wrote a portable version of SCM_I_INUM that uses a conditional to avoid relying on non-standard behavior w.r.t. casting unsigned integers to signed integers. GCC was smart enough to turn that into a simple arithmetic shift instruction. Clang 3.4 was _not_ smart enough.
<mark_weaver>anyway, it turns out that GNU C specifies the behavior we want, so I'll do the simple thing #ifdef __GNUC__, and that should cover Clang as well (since it pretends to be GCC).
<wingo>neat
<mark_weaver>wingo: one thing to be careful of: the argument to SCM_I_INUM could be evaluated more than once. that's been the case forever, on platforms where SCM_SRS does the fall-back thing. I found that the "halt" instruction in vm-i-system.c performed a side-effect in the argument to SCM_I_INUM.
<mark_weaver>(took me a while to track that down :)
<wingo>why are we not using static inline functions?
<mark_weaver>good question, dunno :)
<mark_weaver>one issue is that when we compile with -O0, e.g. for debugging, that could make things very slow.
<mark_weaver>but I suppose we could make the macro call an inline function only when it would have to evaluate its argument more than once (i.e. when not using GCC or Clang). I suspect that would give us the best of both worlds.
<wingo>-O0 doesn't matter i think
<wingo>you expect it to be slower
<wingo>we should use more static inline functions :)
<mark_weaver>I generally agree with that, but my perspective is somewhat different since I use a much slower machine than you do.
<mark_weaver>speaking of macros that should be inline functions, there are several in goops.c that look painfully inefficient, e.g. SCM_GNS_INSTANCE_ALLOCATED_P and SCM_GNS_INDEX.
<mark_weaver>I'm not sure if the compiler would collapse all of those redundant memory reads (via SCM_CDDR etc) or not.
<ArneBab_>mark_weaver: did you check -Og for debugging? You can actually get back traces for tail recurisive functiions in C with that: http://draketo.de/light/english/free-software/tco-debug#sec-3-4 g++ -g -Og -foptimize-sibling-calls
<ArneBab_>I’m no expert on GCC, though
<ArneBab_>(I just happened to hit on that once)
<mark_weaver>ArneBab_: good idea, I should experiment more with -Og.
<mark_weaver>in my case, the usual problem is variables and arguments being "optimized out", so I can't see what's going on.
<ArneBab_>the link shows how to get nice backtraces even with recursion: http://draketo.de/light/english/free-software/tco-debug#nice-backtraces
<ArneBab_>and the performance of -Og wasn’t too bad
<mark_weaver>if -Og does what I need and would still inline static functions, then I guess that would address my concern.
<ArneBab_>maybe you can manually add the inlining
<mark_weaver>I certainly don't mind specifying the 'inline' keyword, where it's available.
<ArneBab_>I meant as GCC option
<ArneBab_>-finline-small-functions
<ArneBab_>gcc --help=optimize | grep inli
<ArneBab_>then you can use -Og with just the additional optimizations you need
<mark_weaver>ah, sure.
<mark_weaver>makes sense
<ArneBab_>(similar to the -foptimize-sibling-calls in the example)
<ArneBab_>(and maybe add a gccdebug target or so, that people can reuse your work without having to read up on gcc options themselves)
*ArneBab_ hopes he does not overstep with that advice: I’m no guru, but I don’t want to not pass on what I learned just because it’s likely that it’s not new. That’s how experience disappears…
<ArneBab_>(means: sorry if I say obvious stuff here…)
<mark_weaver>I don't know about adding a new target, partly because my autofoo is not good enough.
<ArneBab_>for a sorted list of options for GCC: http://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html
<ArneBab_>do you have a Makefile.ac?
<mark_weaver>I'm busy with something else right now, but if you'd like to propose a patch, feel free.
<mark_weaver>(and of course test it first :)
<ArneBab_>(I fear that the guile autotools setup is wizard-level…)
<mark_weaver>to be honest, I'm not even sure that a target is a good idea for this.
<mark_weaver>it would more likely be a configure option, I think.
<mark_weaver>to the extent that it's handled automatically at all.
<mark_weaver>although in practice, usually I would pass CFLAGS arguments to make.
<mark_weaver>and maybe only rebuild one file that way.
<ArneBab_>ok
<ArneBab_>… your solution looks like the better one for autotools: It seems that adding a new target is a can of worms: http://stackoverflow.com/a/4680578/7666
<ArneBab_>mark_weaver: what you could do is adding AM_CFLAGS for optimizations guile absolutely requires - but then you need to add these for all compilers guile wants to support…
<mark_weaver>this is a can of worms. I'd rather just document it in HACKING.
<mark_weaver>anyway, if you could help by producing patches, that would be great. I'm severely overloaded.
<ArneBab_>no probs - I just wanted to note that you were right on the target
<ArneBab_>(so you can cross it off in your mind)
<mark_weaver>thanks
<mark_weaver>civodul: I have a question for you. the recent gnulib update brought in part of gnulib's threadlib, which in turn added the following to our ./configure --help: "--enable-threads={posix|solaris|pth|windows} specify multithreading API"
<mark_weaver>but of course, we don't actually support those other options.
<mark_weaver>what can/should we do about this?
<mark_weaver>or maybe nothing, dunno.
<mark_weaver>(it wasn't added because of one of the modules I recently imported; it was added by simplying updating)
<mark_weaver>*simply
<civodul>mark_weaver: "our" option is called --with-threads
<civodul>but still, that sucks
<civodul>the Gnulib folks suggested that we implement the glthreads API in terms of Guile's API
<civodul>but that seems tedious
<civodul>do you know which module pulls glthreads?
<mark_weaver>I don't remember, sorry.
<mark_weaver>fwiw, I've been thinking lately that we might want to use the glthreads API instead of the pthreads API. It might be needed to support threads on windows.
<mark_weaver>not to mention solaris and maybe others.
<mark_weaver>but I haven't looked closely enough to know whether its feasible or desirable.
<civodul>hmm ok
<civodul>in the short term, though, it'd be nice to see if we can get rid of it
<mark_weaver>but bdwgc for mingw seems to need to use the native win32 threads, anyway.
<mark_weaver>the last time I ran gnulib-tool, I could have sworn that it was listed indented under sys_time, but I just checked and glthreads is not listed as a dependency for sys_time.
<mark_weaver>and it doesn't really make sense anyway.
<mark_weaver>(and I thought so at the time as well)
<civodul>maybe 'lock'?
<mark_weaver>lock does depend on it.
<mark_weaver>(I just searched for 'threadlib' in gnulib/modules)
<mark_weaver>but we don't import lock
<mark_weaver>or at least I don't see it in gnulib-cache.m4
<mark_weaver>oh, threadlib isn't in there either, so we're not pulling in the whole module, just part of it.
<mark_weaver>so we do import lock. I guess I don't know enough about how gnulib works
<mark_weaver>well, I found one dependency chain. we import 'regex' which depends on 'lock' which imports part of threadlib.
<mark_weaver>civodul: and indeed, this new dependency was added last August in ade95dda0df7e42c80c5e6d7cb51fe44d0663719 of gnulib.
<mark_weaver>"regex: port to non-glibc/lock-using systems"
<civodul>oops
<civodul>that's actually precisely because of my bug report
<mark_weaver>heh
<civodul>so we're kinda screwed :-/
<civodul>lemme look at it
<mark_weaver>thanks!
<civodul>mark_weaver: i'm trying what Paul Eggert suggested back then: providing our own "lock" module impl
<civodul>mark_weaver: what Gnulib commit did you use?
<civodul>i don't get it
<civodul>gnulib-tool reports:
<civodul> locale
<civodul> localeconv
<civodul> lock
<civodul>but 'locale' depends neither on 'localeconv', nor on 'lock'
<mark_weaver>yeah, I also found that gnulib-tool was giving me bizarre dependency reports.
<civodul>grr
<mark_weaver>I wonder if it's because of those bash bugs that we've been living with under guix?
<civodul>what?
<mark_weaver>all of the patches for bash that fix various bugs, since 4.2 was released.
<mark_weaver>fortunately, we have 4.3 now :)
<mark_weaver>I guess it's unlikely.
<civodul>yeah, and my /bin/sh is an even older Bash
<civodul>seriously:
<civodul> announce-gen
<civodul> arpa_inet
<civodul> autobuild
<civodul> binary-io
<civodul>something is completely broken now?
<mark_weaver>yeah, I think the script is broken.
<mark_weaver>like I said before, I could have sworn that it told me threadlib was pulled in by sys_time, which made no sense either.
<civodul>ok, lemme send an email
<mark_weaver>it's fairly common nowadays for /bin/sh to be dash. maybe dash would run it better.
<mark_weaver>(since debian uses it as their /bin/sh now)
<jemarch>it would be much better to fix bash instead
<jemarch>it is GNU software
<mark_weaver>if the problem is in bash, then yes, of course. but at the moment, we don't know where the problem is.
<mark_weaver>(my first guess is that the script itself is broken)
<civodul>yeah, most likely
<civodul>although i don't think it has changed recently
<civodul>for your own entertainment: http://lists.gnu.org/archive/html/bug-gnulib/2009-01/msg00034.html
<civodul>:-)