IRC channel logs
2017-05-08.log
back to list of logs
<OrangeShark>amz3: is the guile-git project just a package of guile from a recent checkout of guile's git repo? <paroneayea>amz3: it should be able to support both probably, at least if you interpret like / dislike as upvote/downvote <daviid>CharlieBrown: wrt your guile-gnome attempt, you need to import the module (gnome gtk) <daviid>CharlieBrown: (use-modules (gnome-2 (oop goops) (gnome gtk) ...) [where ... are other gnome modules if needed, like gdk-event, glib, ... see the doc for a coplete list <daviid>sorry, I said (gnome glib) I ment (gnome gobject), see also (gnome gdk) <daviid>CharlieBrown: you effectively need (gnome glib) to create and manage a main loop, g-main-loop-new, g-main-loop-run, g-main-loop-quit ... <daviid>CharlieBrown: what version of guile-gnome are you running? <daviid>CharlieBrown: in the source tree, there are examples, here: gtk/examples <daviid>some do not work (they were written 10y or so :), needs love.. but most and the mst important do work <CharlieBrown>daviid: cal@komputilo ~$ guix package -s guile-gnome | grep version <daviid>look in the source tree, then gtk/examples <daviid>also here: gtk/examples/guile-gtk-demo <daviid>you can run the srcipt: guile-gtk-demo.scm (in that directory) and see what example work... the apps definitely works <CharlieBrown>daviid: Will the source tree be in the location where guile-gnome was installed? <daviid>CharlieBrown: i don't use guix, I don't know <daviid>I'm working on it though, it needs love wrt its goops 'usage' and guile 2.2, I don't think there is a guix package yet, help welcome :) <CharlieBrown>This is interesting. Someone I know wrote something similar, but in another language. I might link them to this. <CharlieBrown>daviid: I understand the International Phonetic Alphabet and the Wikipedia respelling system for English. <daviid>note that this is a time keeping, as opposed to time tracking system, it is simple, on purpose <daviid>CharlieBrown: cool! right now it needs 2.0.14, be aware, it does not work yet with 2.2, i really have to work on it!! <daviid>CharlieBrown: its conficure.ac says its ok with 2.2, but that's a mistake, so feel free to change the configure.ac to only have <daviid>CharlieBrown: ok, but i mean if you package it for guix, right now, you need to change the configure.ac (or what ever it is that substitute that in guix) so it indeed depends on guile 2.0.14 (and not accept if it finds 2.2) <daviid>CharlieBrown: and it depends on grip, which also is missing from guix <daviid>CharlieBrown: so it is going to represent some work, it's not going to be 'that easy', but if you do, well thanks! <daviid>CharlieBrown: ny url of this similar app you were refering to earlier compared to foliot? <spauldo>I've got me a list of records, each representing a vertex. I'd like to put edge information in there as well. I tried putting a list of vertices this vertex is connected to into the record, but that blew up with recursion. <spauldo>In C, I'd just use pointers. Is there any good alternative to just keeping edge info in a separate list? <CharlieBrown>Someone asks, "Can you compile things normally on GuixSD?" <amz3>spauldo, is gone, but for the record, I implemented a graph datastructure using two list one for the vertices and one for the edges <amz3>I stumbled upon a strange behavior yesterday <amz3>I want to create a 'call-with-context' procedure which takes an "env" and a 'proc' as argument and has the same semantic as othe 'call-with-...' procs but the issue here is that 'proc' returns two values <amz3>it seems like 'call-with-foo' can not return multiple values <amz3>ok! call-with-values is again the solution <amz3>that's how is implemented call-with-input-file <lloda>amz3: which call-with-foo cannot return multiple values? <lloda>scheme@(guile-user)> (call-with-input-string "x" (lambda (i) (values i i))) <lloda>$3 = #<input: string 19f38c0> <lloda>$4 = #<input: string 19f38c0> <amz3>lloda: indeed it's me the issue <amz3>I don't have performance issues yet <amz3>but i don't do much either <manumanumanu>Chaos`Eternal: probably not at all. The discussion now is that it might be included as an import to make jit per-module. <amz3>I guess the maintainers expect us to test guile-git before further work <manumanumanu>Chaos`Eternal: how often is guile not fast enough for you? Sure, it isn't chez, but it is a couple of times faster than python. <ArneBab>manumanumanu: except for things like string handling where Python defers to C-based implementations. <manumanumanu>ArneBab: is the difference really that big though? I haven't benchmarked things like that, but I have found guile's string speed to be pretty good... <manumanumanu>I haven't been doing a lot of string searching though... mostly substrings and such. <ArneBab>I have an extreme case here where my optimized Guile script takes roughly 15 times as long as a simple Python script. It’s extreme because it just checks for null-bytes in files. <ArneBab>(and that’s a single statement in Python which instantly defers to C) <ArneBab>(while with Guile I implemented the logic in Scheme) <ArneBab>manumanumanu: for the wordcount challenge I currently need 30% longer than Python3, but I might be able to reduce this. <manumanumanu>ArneBab: do you have any source up? I'm probably not much help, but I'd like to see how you would do it. <manumanumanu>The wordcount seems to depend a bit on whatever sorting algorithm you use. The java one uses either timsort (java stdlib) or what seems to be a specialized fast radix sort <ArneBab>it’s the simplest approach: just use rdelim :) <ArneBab>yes, I still need to check which sorting algorithm sort uses <ArneBab>and whether another one would be better <ArneBab>but before that, the code will want profiling to see where it actually needs time <ArneBab>I had thought about looking into the datastructure for entering the new words: If we could have the words mostly sorted because the most recent ones will come first, then a simple insertion sort would beat anything else. <manumanumanu>I wonder what is the best approach. To read "until" three characters or read while-not-whitespace. <manumanumanu>Is char-whitespace? just a range check over unicode codepoints it is probably faster, no? <ArneBab>feel free to build on my code for that <manumanumanu>ArneBab: I don't speak shell. How do you measure the time of that? <amz3>nalaginrut: when is the next artanis expected <amz3>ACTION uploading a new episode for the screencast where I add database support <amz3>I am wondering how to do uploads of files <nalaginrut>amz3: I'm wring the manual, gosh, so many new things <nalaginrut>amz3: the code has been freezed, so if you can't wait, you may try the git repo now <amz3>I will try to have a look <nalaginrut>amz3: oh cool! are you ready for a geeky Internet celebrity? nowadays there're many people trying to make live casting in technics field <nikita1>nalaginrut: I have a question concerning throw-catch of different errors in artanis <nikita1>there is a catach of 'artanis-err in work-with-request. <manumanumanu>ArneBab: Can't find the implementation of %read-delimited. Is guile also offloading to a c procedure? <nikita1>How to handle 'artanis-err to not stop the service? <nalaginrut>nikita1: do you mean the skill to avoid stop service but can still throw exceptions? <nalaginrut>nikita1: the trick is to set you own specific exception key 'nikita-err in the handler which is running in a event-loop, each time it's caught, don't throw it to the next level, then the exception was limited to the level you want <nalaginrut>if you let the exception be caught by higher level, then it may cause the whole program quit <nikita1>nalaginrut: yes, I undestand it. Let me re-formulate the question: there are several 'formats' of artanis-err that can be catched properly to continue the loop, otherwise, the error is passed uplevel. F.e. I want to add some exception on db-level when the pool of db connections should be increased, but new connection to db cannot be established. Which kind of exception it should be? artanis-err ? or something <amz3>fwiw I don't handle the error of spawning more database connection than available, this should be set when running the fiber server. The number of database connection should be equal to the max number of fibers running <amz3>the database is embedded in the guile process <amz3>that's a choice, prolly a badd choice, but I don't have the time and energy to invest in a standalone database server <nalaginrut>nikita1: note artanis-err is for the exception shouldn't make the server quit, if you see I use error or not artanis-err somewhere, then it means such kind of exception should break the program at once <nalaginrut>nikita1: so if you want to increase db pool on the fly, you should use artanis-err <nalaginrut>amz3: in Ragnarok (artanis' new server core), there could be N connections (limited to the size of connection pool) held by the scheduler, maybe different from your server <manumanumanu>ArneBab: I am not very successful. I am going to try copying how read-delimited does it, but using char-whitespace?. I did bring the runtime down around 5%... <nikita1>nalaginrut: create-new-DB-conn is used for db connection pool expansion, but it uses (error ...) for exceptions. so, for on-the-fly increase, either use create-new-DB-conn or a new separate procedure? <nikita1>in the first case create-new-DB-conn should know about current state -- initialization/running <nikita1>+ the error is used in DB connections recycling. You mean it is so critical to stop server? <nikita1>(I just want to understand design principles) <nikita1>May be it will be stupid, but IMHO it will be better to separate initialization-stage exceptions (which are always critical) and run-stage exceptions (which can be further separated on critical and non-critical). <nalaginrut>nikita1: there's a question, if the connection pool expanding failed because it's failed to connect DB, do you think it should keep working the requests? <nikita1>for that web frameworks i know -- yes. <nikita1>that db connection problem may be temporal <nikita1>but if to fail the web server -- you have to restart it <nalaginrut>nikita1: then there should be an elegant way to wait the temporal connecting issue. Please remember that Artanis has no any threads <nalaginrut>we can't spawn a new thread for the DB expanding <nikita1>may be throw a special exception for db <nalaginrut>but it is possible to take advantage of (break-task) <nalaginrut>we can't use threads, but we can create green thread for that purpose if it's necessary <nalaginrut>and it'll be scheduled if it throw exception, and wait for the next scheduling till it succeed <nalaginrut>this will introduce complexity, we have to think about it carefully, and if we do, there must be mechanism to do it easier. It's bad experience to let user face delimited continuations directly <nikita1>Yes, it may be hidden on the (:conn rc) level <nalaginrut>nikita1: hmm...good idea, then we don't need new type green thread, we can let the task handing the DB query schedule <nikita1>so getting connections from pool may wait a proper connection in cycle with break-task inside it <nalaginrut>nikita1: I haven't implemented a timer based scheduler, the current scheduler is a simple one <nalaginrut>I want to provide customizable scheduler for users <nalaginrut>I have to support protobuf and zeromq in 0.2.1, our product needs them soon <nikita1>nalaginrut: for the first time, i'd propose the simpler thing: introduce a 'db-err for db module. Catch it during initialization and quit with it. And in runtime handle it as a simple 500 error without stopping. <nikita1>and use db-err for all db-related error (respectively) <nikita1>so any db - related problems in runtime -- will cause simple status 500 error <nalaginrut>nikita1: it's possible, then we need to modify create-new-DB-conn to loop till succeed, each time it throw db-err will be scheduled <nikita1>nalaginrut: I think, for the initial implementation even looping is not needed -- just throw-catch->500 db error. The first step is just make db module abstract from (throw 'artanis-err 500 ...) and (error ...) errors. <nikita1>If the codebase is quite stabilized, i'll try to prepare a pull-request with my vision (If you don't object). <nalaginrut>nikita1: if we just throw 500, the user will get it and lost connection, but schedule it may let user thought it's a slow connection. I don't know which one is better <nikita1>Let's see what other frameworks do in case of lost db connections.. <ArneBab>manumanumanu: with 5% reduction we’re getting closer to Python’s C-dominated runtime <ArneBab>manumanumanu: Did you find how to get the runtime? <nikita1>nalaginrut: small offtopic: linux-compare>=? uses string string comparison, so 3.9 becomes higher than 3.10 ;) is there any simple version-compare or dictionary-compare implenentation? <manumanumanu>ArneBab: I really did nothing much. Made it rely on read-delimited only so we can avoid checking for char's and using only string<? and made it a regular cons pair instead of a list so that it doesn't have to chase pointers <nikita1>nalaginrut: for the start, I've a *tiny-tiny* patch for postgresql connection fixes... <nikita1>could you take a look on the pull-request.. <ArneBab>manumanumanu: git clone git://github.com/juditacs/wordcount; cd wordcount; mkdir guile; wget -O guile/wordcount.scm https://pastebin.com/raw/prmkrYYb; scripts/create_input.sh; time cat data/huwikisource-latest-pages-meta-current.xml | head -10000 | guile guile/wordcount.scm > /dev/null; time cat data/huwikisource-latest-pages-meta-current.xml | head -10000 | python3 python/wordcount_py3.py > /dev/null <ArneBab>manumanumanu: on the box I’m using right now, your changes halve the runtime. <manumanumanu>I just downloaded some stupid file from project runeberg <ArneBab>(but on this machine, the Python version is significantly faster — a factor 2) <ArneBab>manumanumanu: the command above should give you the real inputs <manumanumanu>on my machine, the python version is 0.05s faster (compared to guile 2.2) <ArneBab>,profile (for-each (lambda (x) (format #t "~a\\t~a\\n" (cdr x) (car x))) (count-words f)) <ArneBab>with (define f (open-input-file "data/huwikisource-latest-pages-meta-current.xml")) <ArneBab>well, this would have been better: ,profile (for-each (lambda (x) #t) (count-words f)) ☺ <ArneBab>(but not the sort function itself, the comparison) <ArneBab>expensive: %read-delimited (17s), count-words (15s), string<? (10s), read-delimited (8s), count-or-unicode>? (6s), string-concatenate-reverse (5s), … <nalaginrut>nikita1: merged, thanks! I have to go, could you report the version comparison issue as a memo? <nikita1>nalaginrut: already have a fix -- preparing a new pull request <ArneBab>manumanumanu: according to ,profile, half the time is spent in garbage collection <amz3>does artanis use guile fibers? <manumanumanu>ArneBab: We could turn garbage collection off for such a short-running script <ArneBab>manumanumanu: I tried, but then it uses tons of memory <ArneBab>the unicode comparison is less than 20% of the runtime <amz3>another episode is the pipe! <ArneBab>manumanumanu: I so no that feeling … ☺ <amz3>I am missing one feature <ArneBab>manumanumanu: using vlists did not make it faster — it got slower by factor 10. Another idea to die :) <amz3>the ability to upload files <amz3>paroneayea: how do you upload files using guile and guile fibers? <ArneBab>manumanumanu: if you compare to python2: that implementation isn’t actually correct (does not do string comparison) <paroneayea>amz3: well, first you need a multipart library... ;) <ArneBab>can I tune the garbage collector to be less aggressive? <paroneayea>amz3: but basically once you have that you should be able to do an http POST with the multipart message as the body using guile's (web client) library and it should Just Work (TM) I think? <amz3>paroneayea: once I have the multipart thing. yes it should work ok <amz3>but I think I made a mistake, I mean I think that I can... re-invent the wheel to do post using my frontend code <manumanumanu>ArneBab: got it down to 33s from 41s on my computer by using %read-delimited! <amz3>one thing is important in my frontend code, compared to regular http+html thing, is that forms are not handled by the browser and are handled by the frontend code manuallly <amz3>anwyay, long story short I might not need the multipart parser for some reason. <amz3>ArneBab: manumanumanu: IMO one should PR a solution even if the numbers are not as good as you expected <amz3>not everything is about speed of execution in a language... <ArneBab>amz3: I plan to (@manumanumanu: or rather: we plan to?) <ArneBab>manumanumanu: I’d start with speed and then try for clarity without sacrificing too much speed <ArneBab>your version (read-word) is 10% faster here (0.233s instead of 0.254s for the reduced data) <ArneBab>that’s 45s here — compared to 34s with python3 <ArneBab>which likely is already faster than julia <ArneBab>manumanumanu: let’s make one wordcount_reference.scm and one wordcount.scm — like it’s done for python <rekado>amz3: this works with the JavaScript “fine uploader” <manumanumanu>My version is ugly, since it breaks on words larger than 1000 chars <ArneBab>you can increase the buffer size without significant cost <amz3>rekado: you have awesome music in your soundcloud account <ArneBab>manumanumanu: got down another 10% by replacing hash-fold with hash-map->list <rekado>amz3: oh, thanks. It’s all old and abandoned stuff. I no longer use the account, but I haven’t yet removed everything from there. <rekado>amz3: new music is being recorded these weeks, actually. More interesting stuff than what I dumped there. <civodul>rekado: you'll have to let us know when it's ready :-) <rekado>Don’t hold your breath. We’re really slow, and the longer we wait with recordings the more rewrites will happen… :) <rekado>But once it’s done I’m gonna let you all know. <paroneayea>(case) is still linear to the number of branches, right? <paroneayea>I thought I heard somewhere that it wasn't but that wouldn't make sense to me <ArneBab>(I’m defining everything mutable within count-words in the hope that this will make it thread-safe) <ArneBab>it avoids having to resize it — doesn’t save a huge amount of space and relies on pre-existing knowledge, but it’s useful :) <ArneBab>here this is 30% slower than Python for the larger dataset, and I wonder whether someone can golf it lower still :) <manumanumanu>Hmhhh.. not very it seems. it still accumulates and reverses a result. <ArneBab>it’s easy to make something slow — the wisp parser got faster by factor 20 with help from people here <manumanumanu>We are getting faster by the minute with this as well :) <ArneBab>checking for " " sadly makes it slower again <manumanumanu>ArneBab: Got it a bit faster again. it is no single space, it is a null string. instead of the string-every, just check string-null? <manumanumanu>now it is only about 50% slower than the python3 version <amz3>again ipfs on the breaking news <amz3>manumanumanu: ArneBab : wow that still very readable <amz3>you do a full hashmap->list that's a n big o complexity <amz3>I think one can use 'bisect' like thing to improve on that... maybe not... <amz3>what I am thinking is that the result of hashmap->list could be constructed while reading the file <manumanumanu>Then the question is, does guile optimize code in modules? In chez scheme, code gets a lot faster jusst by being in a module <OrangeShark>you mentioned about renaming the project. What was the reason? <amz3>OrangeShark: renaming to libgit2 to avoid the confusion with guile git repository <OrangeShark>amz3: okay, I thought that was the reason. We could either name it libgit2 or come up with a new unique name for the project but that might make it too confusing <OrangeShark>amz3: like the libgit2 bindings for ruby is called "rugged" <amz3>we can call it orange or guile-orange? or guile-apple? idk. <OrangeShark>maybe just guile-libgit2 is fine :) We can still use git as the module name though <OrangeShark>amz3: I think we just need to change the repo name and configure.ac <manumanumanu>ArneBab: I'm pretty happy with how we have been working on this! I am pretty happy with my own contributions as well :) <manumanumanu>The person doing the tests is probably running 2.0 though :( :( :( <ArneBab>manumanumanu: we can avoid that by calling guile-2.2 <manumanumanu>Instead of reading whitespace and filtering it later, I remove it at the port level <ArneBab>amz3: I also thought about constructing on the fly, but since the elements must contain the number of times a word appears and that word can come at the end, this might be problematic. Would definitely need some bisecting (like writing into a tree structure), but that’s a larger effort (making the right datastructure, and making it fast). <ArneBab>manumanumanu: nice! I’ll have a deeper look later <manumanumanu>but I see it uses fx-operations. Those are darn slow in guile. Will have to fix that <amz3>I got an idea, I will rename my game "build yourself a civilization" <amz3>you will start with civilization pack which can turn dirt and ore into various tools like small computer <amz3>and then one must write algorithm to build bigger tools <amz3>... this sound like life. <amz3>it's better if that was a game written in scheme, so that if you fail, you civilization you can call/cc to the point where you made a mistake <amz3>talking of that, I must make wiredtiger persistent somehow like datomic does <manumanumanu>ArneBab: oh, the pairing heap is functional. it is sloooow <ArneBab>manumanumanu: on the machine I’m at now, guile wins! <ArneBab>manumanumanu: If I read it right, mutable-heap uses pairing-heap <ArneBab>manumanumanu: ah, yes, you’re right. <ArneBab>manumanumanu: I used your latest code, but this machine has fewer cores. For multithreaded code (like the gc) this could actually be an advantage <ArneBab>(multi with the number of threads < number of cores) <ArneBab> (or (eof-object? ch) (not (char-whitespace? ch)) <ArneBab>it’s correct as far as I can tell: it only skips, if ch is not eof and whitespace. <ArneBab>it’s correct as far as I can tell: it only skips, if ch is (not eof) and whitespace. <ArneBab>manumanumanu: shall we add an LGPL header and file a pull-request? If I should do it: how can I call you? <manumanumanu>I am not too good with git, so it's probably better if you do it <manumanumanu>no idea... impossible to say. tiny tiny difference maybe <ArneBab>manumanumanu: I think the unless wasn’t correct — skipped too much <ArneBab>manumanumanu: I usually use Mercurial to interface with git repos :) <CharlieBrown>ArneBab: I installed guile-wisp. How do I convert my Lisp to Wisp?