IRC channel logs

2013-09-09.log

back to list of logs

<ArneBab>mark_weaver: (+) ⇒ 0, that’s an argument I buy — I’ll have to remember to add this to my guile-notes tomorrow!
<mark_weaver>cool :)
<mark_weaver>in the scheme community, we generally prefer for a given procedure to have a single meaning. numeric addition and string concatenation are conceptually very different. to think they are similar is a result of imprecise thinking, IMO.
<mark_weaver>now, list concatenation (append), string concatenation (string-append), and vector concatenation *are* conceptually the same thing. combining all of those things into a ++ operator would be much more defensible.
<mark_weaver>although you'd still have a problem of what to return when passed zero arguments.
<mark_weaver>that reminds me of another place where there'd be a conflict with combining these two concepts into one: one might reasonably decide to extend numerical addition to vectors and lists, by adding successive elements together, e.g. (lambda (x y) (map + x y))
<mark_weaver>this is in direct conflict with the idea of concatenation.. again, it's a different concept.
<foeniks>that a concatenation operation should work on zero arguments is also conceptually weird
<foeniks>I never new that the hardest part about lisp-parsing was the literals
<mark_weaver>I don't find that conceptually weird, but then I guess I went down the rabbit hole long ago :)
<mark_weaver>foeniks: which literals do you find difficult?
<dsmith>The leaks are not as bad, if they exist at all. The cron job to restart runs every 4 hours now instead of every hour.
<mark_weaver>dsmith: if you think the leaks might no longer exist, it might be useful to verify that by removing the cron job that restarts sneek.
<dsmith>mark_weaver, I've been thinking of doing that.
<dsmith>I don't remember how I measured that it was leaking, however..
<dsmith>It was an astounding amount before. About 750k for every IRC line, iirc.
<mark_weaver>if there's a leak, I'd expect gradually increasing memory usage to appear in 'ps' output.
<mark_weaver>('ps' is not the best way to measure memory usage, but it should be sufficient to detect a leak like this)
<dsmith>Yes
<dsmith>Arrg! I forgot the user name on my rpi!
<dsmith>mark_weaver, DO you remember?
<dsmith>(pi, btw)
<dsmith>hey
<dsmith>sneek, botsnak
<dsmith>sneek, botsnack
<sneek>:)
<dsmith>Ok, that cause the rss to go from 8816 to 8824
<dsmith>sneek, seen rlb?
<sneek>rlb was here Aug 10 at 01:17 am UTC, saying: mark_weaver: ok, right -- I'll hold off for further discussion, and then we can either change the topic or the approach. Thanks..
<dsmith>And that to 8860
<dsmith>hmm
<dsmith>and then 8876 with my lines after that
<dsmith>I think that's in K bytes
<mark_weaver>well, increases in memory use early on are probably inconclusive.
<mark_weaver>it'll be interesting to see if memory use continues to increase without bound.
<dsmith>The last restart was at 1600.
<dsmith>SHould have been one at 2000 a few min ago
<dsmith>if I still had the cron running..
<mark_weaver>also, rss might not be the best thing to watch. virtual memory might be better
<dsmith>Hmm.
<mark_weaver>well, leaked memory might or might not remain resident.
<dsmith>Added vsz
<dsmith>Looks like 2-4 pages for every line.
<dsmith>The code does a LOT of consing I think.
<dsmith>Lot's of splitting strings to lists and manipulating those.
<dsmith>We will see how it does. I'm logging every hour
<dsmith>there is no mem consumption when idle. Only for messages.
<mark_weaver>certainly, memory leaks are much more likely to happen when interfacing with C/C++ code.
<mark_weaver>I'm sending sneek private messages now.
<mark_weaver>sneek: seen wingo
<sneek>wingo was here Sep 08 at 05:05 pm UTC, saying: hehe.
<mark_weaver>dsmith: how much has memory usage increased in the last few minutes?
<mark_weaver>sneek: psyntax?
<sneek>Someone once said psyntax is still compiling
<mark_weaver>hehe
<mark_weaver>sneek: botsnack
<sneek>:)
<mark_weaver>(I've sent him many messages privately also)
<mark_weaver>sneek: who is wingo?
<sneek>I've heard wingo is awesome!
<mark_weaver>:)
<nalaginrut>morning guilers~
<cky>dsmith: Maybe I should spend some time reviewing the bobot code.
<cky>dsmith: That may help me identify the location of the memory leak.
***banannagram is now known as bananagram
<nalaginrut>is there anyway to send mail without any lib binding?
<nalaginrut>I want to add mailsend in Artanis
<mark_weaver>it's not trivial to send email.
<mark_weaver>mainly because of spam filtering.
<nalaginrut>I want to add a function for mail activation
<mark_weaver>if you can count on the machine having a properly configured local MTA, then you can use the local 'sendmail' command on POSIX systems.
<nalaginrut>no any other easier way? call 'system' is not the best way I think
<mark_weaver>that's really the only reasonable solution for you.
<nalaginrut>or I have to write bindings?
<mark_weaver>sorry, there's no easy way to send email. in practice, you depend on a properly configured MTA.
<nalaginrut>OK, I'll check it out, thanks
<mark_weaver>it's not just a question of having a library. you need configuration from the user of where to send the email, what authentication method to use, credentials, etc.
<mark_weaver>nalaginrut: on an internet server, I think it's probably reasonable to assume that there's a properly-configured local MTA.
<nalaginrut>so 'system' is reasonable
<mark_weaver>yes, although I'd use 'system*'
<mark_weaver>to avoid potential security holes from shell metacharacters, etc.
<dsmith>mark_weaver, I've only been logging every hour
<nalaginrut>I wonder how does web.py handle it
<dsmith> 9628 16044
<dsmith> 9628 16044
<dsmith> 9632 16044
<dsmith>That would be 9,10,11 pm
<dsmith>rss and vsz
<dsmith>Probably should have timestamped each
<dsmith>nalaginrut, I agree with mark
<dsmith>Mail *is* complex
<mark_weaver>dsmith that looks good so far.
<dsmith>fwiw, for systemy things in scripts that I need to send mail, I typically use the "mail" command on a properly configured system.
<mark_weaver>especially now with spam filtering. it's actually possible in theory to send email directly to a recipient's MTA without any configuration, but nowadays that email is likely to go straight into the spam folder (or simply dropped) if you don't jump through hoops.
<dsmith>Piping the message into stdin, and putting the "to" adderess on the command line, along with the subject using -s
<mark_weaver> http://www.codinghorror.com/blog/2010/04/so-youd-like-to-send-some-email-through-code.html
<dsmith>mark_weaver, Ya, I was surprised how nice it looked (mem usage)
<dsmith>only went up 4k
<mark_weaver>nalaginrut: in general, I'd recommend avoiding 'system', 'open-pipe', 'open-input-pipe', 'open-output-pipe', and 'open-input-output-pipe'. all of these pass a single string to the shell for interpretation, which is a likely source of security holes.
<mark_weaver>instead, use 'system*' or 'open-pipe*'.
<nalaginrut>these two are already in guile-2.0.9 right?
*dsmith has even more agreement with mark_weaver
<mark_weaver>nalaginrut: yes, they are quite old.
<nalaginrut>ok
<nalaginrut>even if I add actor-model or greenlet later, there won't be any problem?
<mark_weaver>also, for commands that support it, it's a good idea to put a '--' argument after the options and before any non-option argument, to guard against arguments that you *intend* to be non-options, but that might start with '-'.
<mark_weaver>nalaginrut: hmm, you haven't given me enough detail to answer that question.
<nalaginrut>mark_weaver: yes, sorry for such a question
<mark_weaver>though I should mention that (ice-9 popen) is not thread-safe currently.
<mark_weaver>I hope to fix that before 2.0.10, but for older versions of Guile, you should ensure that only one POSIX thread calls any of the (ice-9 popen) procedures.
<nalaginrut>mark_weaver: I have to go for lunch, see you later
<mark_weaver>okay, ttyl!
<mark_weaver>nalaginrut: fwiw, I agree with dsmith's suggestion to just pipe the message to "mail -s <subject> -- <email-addr> ..." on an internet server.
<mark_weaver>(he didn't mention --, but I add it :)
<dsmith>(last paste) and is 9-12
<dsmith> 9628 16044
<dsmith> 9628 16044
<dsmith> 9632 16044
<dsmith> 9664 16176
<dsmith>Yeah, -- is a good idea, especially when it's a user-supplied address.
<dsmith>I've never used --, but then, I've always been sending to a constant hard-coded address.
<wingo>i thought i made ice-9 popen threadsafe a while back...
<wingo>that was the point of rewriting open-process* in C
<wingo>is there a problem with the scheme code?
<wingo>i got it to the point that it worked well for me in my multithreaded forking environment
<dsmith>The nice thing about "mail" is you don't need to worry about port numbers, and if sendmail or postfix (or something else) is installed. Or if you've got the right headers.
<mark_weaver>tupi had problems with it. the pid/table is not protected, nor is the guardian.
<mark_weaver>wingo: ^
<wingo>ah.
<mark_weaver>I already have a patch to fix those things, but that led to other problems, which I posted about on guile-devel.
<wingo>the port/pid-table is atomic in master at least
<mark_weaver>subject: something like "asyncs and mutexes"
<wingo>because it is a weak table, and weak tables are atomic in master
<wingo>seems to me guardians should be internally atomic
<wingo>i.e. no mutex need be taken
<mark_weaver>in that case, there will be deadlock issues because of the async (after-gc-hook)
<wingo>anything finalization-related has to be minimally threadsafe...
<wingo>could be
<mark_weaver>can't talk now, ttyl!
<wingo>ciao
<mark_weaver>wingo: if you wanted to read the messages I wrote to guile-devel about these deadlock issues, and to contribute your thoughts, that would be most welcome :)
<wingo>it's on the list :)
<wingo>i should have lots of guile hacking time in a week or two
<nalaginrut>I confirmed that we don't need WSGI, since we have an unified extensible server interface, which is (web server), though we don't have logger. And for Ruby, they have Rack as a unified server interface for various framework in the future.
<mark_weaver>wingo: fwiw, I don't yet see any reply from you about the async+mutex deadlock issues. Your mesg here "it's on the list" led me to expect otherwise.
<wingo>mark_weaver: on my todo list, i meant
<mark_weaver>wingo: glad to hear that you'll have some hack time :)
<mark_weaver>ah, okay :)
<wingo>sorry for being ambiguous
<mark_weaver>no worries.
<mark_weaver>wingo: fwiw, I've gotten distracted by some other things (Guix, and the NSA revelations), so I won't be looking at prompts in RTL for a while. if you're inspired to work on that, feel free :)
<wingo>cool, thanks for the note :)
<mark_weaver>in Guile land, I'm more likely to work on source tracking for bare symbols in the near future.
<wingo>via some sort of read-syntax function, I guess?
<mark_weaver>well, my original plan was a variant of read that produces syntax objects, yes..
<mark_weaver>however, there's a slight complication.
<mark_weaver>the guile syntax objects include a 'module' field, and the code assumes that the module is already baked into any syntax object.
<wingo>humm, i see.
<mark_weaver>so now I'm a bit torn between going through all the code looking for that assumption, vs making a separate type for source-annotated datums.
<mark_weaver>but it's all up in the air right now. your thoughts are welcome :)
<wingo>no idea :) sounds like you're thinking of all the right things though :)
<mark_weaver>okay, I'll mail the list (and seek your input in particular) before committing one way or the other.
<wingo>sure
<wingo>for reader changes, ludo probably knows & cares more about that bit, fwiw
<mark_weaver>okay, I'll make sure to ask him also.
<nalaginrut> total heap usage: 23,780 allocs, 23,230 frees, 101,592,755 bytes allocated
<nalaginrut>100M was allocated for Guile?
<wingo>-ECONTEXT ;)
<nalaginrut>well, I just tried "valgrind guile" then quit directly
***jijijij is now known as anderson
<nalaginrut>now Artanis support return string as HTML content, (get "/new"
<nalaginrut> (lambda ()
<nalaginrut> "hello world"))
<nalaginrut>(get "/new"
<nalaginrut> (lambda ()
<nalaginrut> "hello world"))
<nalaginrut>but it's just a trick to cheat newbies, since if you want to custom status/headers/time, you have to use more powerful response-emit
<nalaginrut>anyway, the code looks cool on main-site
<Arne```>mark_weaver: I have a question to be really, really sure (as in “I’m certain that I can misunderstand anything if it even has a minuscle amount of ambiguity”): When I add a method with GOOPS, does that affect libraries I use? When I release a module which adds a method with GOOPS, and someone uses that module, does that affect other modules he or she uses?
<mark_weaver>when you add a method to any generic function (such as +) with GOOPS, you are effectively mutating that generic function. Anything thing else in the system that uses that generic function is affected, yes.
<mark_weaver>in this case, if you add string concatenation to +, that will affect + for all users of + throughout the guile process.
<Arne```>ouch! then I agree that it’s dangerous…
<Arne```>mark_weaver: thanks!
<mark_weaver>np!
<Arne```>mark_weaver: may I quote you?
<mark_weaver>sure
<Arne```> thanks!
*mark_weaver --> zzz
<Arne```>cu
<Arne```>mark_weaver: idea for funny dreams: why not just adjust /append/ to also append strings, and /+/ to apply to lists of lists and do vector addition?
<Arne```>(no need to answer ;) )
<nalaginrut>Arne```: just overload '+'
<nalaginrut>I overload '+' for string every time, maybe we should add it
<nalaginrut>alright, web.py use /usr/bin/sendmail for sending mail, I'll take mark_weaver 's advice
<Arne```>nalaginrut: potentially breaking error handling in unrelated code sounds quite dangerous to me…
<nalaginrut>Arne```: what? sendmail?
<wingo>you can make new, scoped generics
<wingo>module-scoped anyway
<Arne```>wingo: “you can” as in “(use-modules (scoped generic))” or as in “here’s the advanced scheme manual”?
<Arne```>module scoped would be nice
<Arne```>nalaginrut: (+ "ab" "c")
<nalaginrut>so what's wrong with this?
*Arne``` actually prepared for this :)
<Arne```>(let ((foo 1)(bar "2")) (catch 'wrong-type-arg (lambda () (+ foo bar)) (lambda (key . args) (format #f "foo or bar is not a number!"))))
<Arne```>nalaginrut: ^ code sample someone might have used to check whether the arguments to some function are numbers while doing the computation.
<wingo>make a new binding (perhaps by making a pure module, with the binding for + renamed to guile:+), and call guile:+ as the + method with unspecialized args
<nalaginrut>that's the problem of overload, even in Python you got such exception
<nalaginrut>or you may throw '(format #f "unsupported op type for +: ~s and ~s" (class-of foo) (class-of bar))
<nalaginrut>which is more clear
<Arne```>nalaginrut: the exception is not the problem I think of. The problem is that if I were to import a module which adds a generic + which works on strings, the exception in the other (unrelated!) code would be silenced befor it can be caught.
<Arne```>wingo: I don’t think I really understand yet… So I would rename + to guile:+ and then define a private function named +, which I do not export?
<wingo>you would export +
<wingo>or alternately you define "add", and export it as "+"
<wingo>probably easier that way
<Arne```>but since it is not the same object as the standard +, goops would not affect libraries which do not use the module?
<wingo>correct
<Arne```>Nice!
<nalaginrut>ah nice
<Arne```>Now we’re getting to the kind of magic I like: The explicit kind :)
<Arne```>wingo: Thanks!
<wingo>np :)
<nalaginrut>I can't help to use 'format' again since it's easier to express
<dsmith>Oi. Who knew that % was magic in crontab files?
<dsmith>An can only be escaped with \\
<ijp>read-hash-extend is global right?
<ijp>ewww
<dsmith>ijp, I bet read-hash-extent predates when guile got modules
<ijp>I'm just looking at the code for emacsy
<ijp>he is really liberal with that
<ijp>no offense to shanecelis, but I'm not sure he could have hit any more of my pet peeves if he tried :)
<ijp>huh, we had gavino on here a short while back
<ijp>wingo: what was that OS you linked to a few days ago?
<ijp>ah ethos
<ijp>I was grepping for "http:/" but since you didn't put in a full uri I missed it
<mark_weaver>wingo: I think you probably confused poor Arne```
<mark_weaver>I thought I'd just about convinced him that combining numeric addition and string append was a bad idea.
<mark_weaver>as you know, there's no way to combine them together into the same op without either (1) globally affecting + for the entire Guile process, or (2) creating a new + that's quite slow.
<mark_weaver>I suggested ++ for 'append' operations, which I thought was something he might be able to live with.
<mark_weaver>nalaginrut: you too? you add a string append method to '+'?
<mark_weaver>ugh
<mark_weaver>ijp: well, 'read-hash-extend' makes global changes to 'read' in the process.
<mark_weaver>apologies, I can be cranky in the morning sometimes, heh.
<ijp>well, read-hash-extend is not the most problematic thing in the world, since it's not a very general extension
<ijp>but it suffers from a problem that goops also does, which is multiple modules fighting for the same extension
<mark_weaver>GOOPS overloading of primitive generics is the only sane way I know to build something like a computer algebra system, but it has to be used responsibly.
<mark_weaver>as for 'read', it would be nice if we had an easy way to make custom readers without mucking up the system reader.
<mark_weaver>I guess 'guile-reader' (which I've looked at in detail) is maybe the right thing for that, dunno.
<mark_weaver>s/which I've/which I've never/ (oops)
<ijp>it wouldn't be hard (probably) to make module specific readers, but I haven't thought about how it interacts with modules in non-scheme languages
<mark_weaver>module specific readers are easy enough, yes. what's a bit harder is arranging for a custom reader to read the _source_ code of your module, which is the problem shanecelis has.
<ijp>in racket, it's easy enough, since they have a language declaration at the top of ever source file
<mark_weaver>One idea I was thinking of was to have #!FOO look up a reader defined in the (reader FOO) module, or something.
<mark_weaver>well, I guess a general #!FOO has to be able to somehow modify the existing reader, such that more than one of those things can be used together. a thornier problem.
<ijp>the way we did it for the guile executable was to make it a flag
<mark_weaver>in particular, I want to be able to support things like SRFI-110 and Arne's WISP without building them into Guile's reader.
<ijp>for languages, that is
<ijp>a noweb language could possibly make sense, dunno
*dsmith-work hands mark_weaver a decrankifying hot beverage of choice
<mark_weaver>hehe, thanks dsmith-work :)
<ijp>but for things like the #.\\ in emacsy, it's too much work
<mark_weaver>biab...
<wingo>it wasn't clear to me that Arne``` cared about speed
<ijp>arne is the guy who wrote wisp?
<mark_weaver>yes
<mark_weaver>wingo: *nod* I can appreciate that it's hard to know when to try to convince someone to drop their bad habits, and when to show them that they can do what they want in Guile, no matter how foolish :)
<mark_weaver>ijp: when you say "the way we did it for the guile executable was to make it a flag", who's "we"? and what is this flag? I'm confused.
<mark_weaver>was this before I was involved?
<dsmith-work>mark_weaver: I think he is referring to --language
<ijp>we is wingo, I think
<ijp>and I was talking about --language
<mark_weaver>ah, okay
<mark_weaver>thx
<ijp>which reminds me, I am finally going to rebase lua and apply those months old patches today
<mark_weaver>nice! :)
<ijp>and then I'll reserve some time later in the week for crying over the parser :)
<mark_weaver>heh, I was tempted to ask how you decided to address the end-of-line issues, but decided that given my current mood, maybe I better not :)
*mark_weaver tries to focus on paging psyntax back in
<Arne`>x
<mark_weaver>hi Arne`
<Arne`>Hi mark_weaver
<mark_weaver>Arne`: I saw your irc exchange with wingo. I wanted to make sure you knew that making a new addition operator, for the purpose of combing 'string-append' into it without affecting other modules, has significant performance implications.
<dsmith-work>ijp: gavino was here? Did he make a fuss?
<mark_weaver>and it still doesn't solve the (+) problem.
<Arne`>mark_weaver: how much does performance degrade?
<wingo>measure if you care...
<mark_weaver>Arne`: well, '+' is an important operator, as you can imagine. the core '+' compiles to a single VM instruction.
<Arne`>Knowing that I can do the local operator essentially just means that I can define it if I need it for a given task.
<mark_weaver>using your customized version means that all of those additions will turn into procedure calls, and inhibit many optimizations around any code that uses '+'.
<ijp>dsmith-work: mark_weaver noticed him, but it looks like nalaginrut never knew about him
<mark_weaver>Arne`: I guess I was hoping to convince you that it's better to use a different operator name for append, since it is conceptually a different thing.
<Arne`>does it affect code from modules I use? If I run a function from another module, and I used the module scoped generics, does that affect the code in the modules.
<ijp>mark_weaver: did you see my silly generalisation of mean the other day?
<mark_weaver>Unlike most other languages, Scheme allows you to choose nice names like ++ for that.
<Arne`>mark_weaver: I think for a general modul it would be more useful to have (append) operate on lists and strings alike.
<mark_weaver>Arne`: okay, well, then you have the problem of what (append) [no arguments] will return.
<ijp>well, you could just not support it
<mark_weaver>but I suppose you could make your combined generic append not accept zero args.
<Arne`>but there are some applications which do lots of string concatenation, and for these it would be useful to bind + locally.
<Arne`>yepp
<Arne`>or to actually turn that into an error case.
<Arne`>though it would be better for not breaking expectations to have (append) return a list.
<mark_weaver>Arne`: so ++ is too long/ugly for you to consider?
<shanecelis>Any tips on finding memory leaks when using guile? Use valgrind?
<Arne`>(++) looks to me like it does addition inside the stuff i pass it.
<ijp>how about &
<Arne`>(++ '(1 2 3) '(2 2 3)) ⇒ '(3 4 6)
<mark_weaver>there are so many symbols to choose from. if you're willing to use Unicode, then the set of possibilities get even bigger.
<ijp>inb4 PILE OF POO function
<Arne`>mark_weaver: For any case where string-concatenation isn’t the most important task, I would just overload (append)