<davexunit>people on reddit and #emacs insisting that guile-emacs is dead :( <amz3>with guile 2.2 my batch import script doesn't leak <amz3>my day started well, guile 2.2 solves the leak issue I have with a small import script <amz3>problem is it crashed with a stacktrace <amz3>and obvioulsy I lost the stacktrace. I closed the screen terminal by mistake <amz3>also it's faster, definitly faster, I did not timing, but guile 2.2 will be fast <amz3>(disclaimer: my script is not a benchmark) <nalaginrut>yes, 2.2 is faster, I've been using it for almost a year, but there's an unknown bug was introduced later so that I can't run Artanis with 2.2 <nalaginrut>I was planing to develop Artanis all in 2.2, then I think it's better to use 2.0 for stable <amz3>do you use bytevectors ? <amz3>because IIRC the stacktrace mention some iconv bv->string error <amz3>to give you an idea of my leak bug, I have a file of 1G with 16M tuples. With guile 2.0 after parsing one of this file, there 16G of RAM+swap occupied. On the disk the database is only 4.5G. <amz3>with guile 2.2 there is no extra memory occupied, only the database cache <amz3>which is handled in C not guiel <mark_weaver>I'd like to understand where that leak is coming from <amz3>I tried to eliminate the source of leak, it only appears when I work with pairs <amz3>right now the leak is slower because I don't to alist-delete <mark_weaver>if you're able to distill it down to a minimal self-contained example that demonstrates the problem, I could investigate further <nalaginrut>amz3: I use bytevectors a lot, but the problem is something wrong in macros <amz3>mark_weaver: yes I will try to do something more simple <nalaginrut>I won't be surprised to see anything strange ;-D <mark_weaver>nalaginrut: did you file a bug report about that problem with macros? <nalaginrut>mark_weaver: not yet, I still don't know how to reproduce it, except let people run Artanis, which is not good report <nalaginrut>mark_weaver: I think some bug was introduced into macro expander, it works fine before <mark_weaver>if you can't reproduce it, what makes you think it's a problem with macros? <nalaginrut>mark_weaver: I mean I can't use a simple case to reproduce it, it occurs every time when I run Artanis <mark_weaver>I don't think I've ever seen a macro problem that failed non-deterministically. <nalaginrut>well, I can see different while expand it manually <mark_weaver>there are actually some bugs in the 2.0 macro expander that you might have become dependent on, which have been fixed in 2.2. <nalaginrut>that's what I fear for, I don't want my feature based on a bug, or we have to call it a feature ;-O <amz3>I found the bug in guile-2 related to bv->string <amz3>it's chinese word (!) that is not parsed correctly in 2.0, 2.2 and python 2.7 (I think) <amz3>guile 2.2 throw an error <mark_weaver>the problem is that last code point 0xFFFD, which is not a normal character. it is the so-called unicode "replacement character". it is meant to indicate that an error occurred. <mark_weaver>it's the last three bytes of your example bytevector <amz3>this is not a serious bug ? <amz3>I mean is it a serious bug ? :) <mark_weaver>I found the relevant code, in scm_from_utf8_stringn in strings.c, line 1678: if (c == 0xfffd) <mark_weaver>basically, the code detects an error by looking for a replacement character in the result of u8_mbtouc <mark_weaver>which makes sense in a way, because the replacement character is itself used as a placeholder to show that there was an invalid encoding at that point. <mark_weaver>I wonder if u8_mbtouc provides a way to signal an encoding error other than putting one of those 0xfffd code points in the result. <mark_weaver>if I'm interpreting the docs correctly (section 2 [Conventions] of the libunistring manual), we should be able to detect an error there in a better way: the return value should be -1 in case of an error. <nalaginrut>IIRC, there's an option to avoid throw error directly <amz3>nice, will study a bit a guile code <amz3>I'm trying to reproduce the leak with a single file, I let you know if I reproduce <ArneBab_>re guile-emacs on reddit: “performance terrible” ← only tested *startup* performance. <ArneBab_>but then the perf-tester says that that’s the case <ArneBab_>regarding donations: I would join a patreon to give monthly funding or such. <amz3>I can't run guile-emacs on debian using guix. I think that's what started the conversation, I hope it's not poisonous. <amz3>I'm working on planet Guile right now, but looking at the best way to proceed <amz3>I don't feel like hacking on guile-emacs, which looks like the case of a lot of people <amz3>Regarding planet guile, I'm wondering how to proceed <amz3>I think I'll use davexunit's haunt static blog engine <amz3>(At least it teach me a lesson #guile-emacs) <amz3>what do you think about the layout: <amz3>1) classic planet with summary/full article <amz3>2) HN style, only title, blog, author and date of publication <amz3>nalaginrut: do you have by any means, a `unescape' procedure? <nalaginrut>yeah, so the question become 'what's escape?' ;-) <amz3>inside atom files, html is escaped it looks like this "<p>GNU Guile provides" <amz3>this is regex to write i guess, but I though maybe you already wrote it <nalaginrut>amz3: but you have to take care of it, since there could be embedded script, even you don't use database <nalaginrut>encode it each time fetch the data maybe better, but depends on your design <amz3>I need to strip `<script>' tags ? <amz3>URL encoding and html escaping is different, I will get my hand regex ;) <ArneBab_>amz3: I prefer the classic layout: it lends itself better to a moderate rate of articles <nalaginrut>amz3: and for html escape, I have a helper function in utils <amz3>ArneBab_: That's what I am thinking <nalaginrut>since Artanis will decode the content before send html to client, this function also replaced some chars from url encoding, to make sure it's safe <nalaginrut>of course, just do it for the contents, not html code... <amz3>I don't understand how chars form url encoding can appear in html <amz3>what do you mean by html code? <nalaginrut>amz3: if you call this function with html code, it won't be interpreted anymore <nalaginrut>I just use it for the data gotten from DB (any kind of DB as you may thought) <nalaginrut>amz3: url appears in html is possible for a RESTful framework <amz3>I will read the code later, maybe the wil more clear, right now i'm busy with something else.. <nalaginrut>it could be passed from URL, and stored into DB, and if crackers do some tricky things, it could send some script to users <nalaginrut>amz3: maybe you don't need to care about it too much <amz3>i think it's an important matter <nalaginrut>but you don't use RESTful in a static page generator, anyway, the replace function is useful for you <amz3>I've started to work on my database, i'd like to finish before pushing further planet work <nalaginrut>amz3: you use your own DB for that project? cool ~ <amz3>nalaginrut: that the thing, I wondering if should use my own db, or not... <nalaginrut>amz3: well, nowadays people love static pages because they want take advantage of github <nalaginrut>and I have to say, savannah provides the similar feature too, the only problem is that documents have to use CVS, you can't use git/hg/svn... <nalaginrut>interesting, maybe decades later, I'll force young men use git as well... <amz3>people use git because it convenient <amz3>in terms of software my database or haunt is the same. <amz3>btw, it's better to describe wiredtiger and specifically guile-wiredtiger as framework to build database <amz3>The reason I want to use wiredtiger are: <amz3>0) doing more work with to learn more patterns <nalaginrut>amz3: I guess decades ago they said "people use CVS because it convenient" ;-P <amz3>1) spreading the joy: because even if it's a framework for newbies it's much easier to get to know wiredtiger well, than learning from scratch SQL. There is only scheme to know and few procedures and patterns <amz3>I find it easier to wiredtiger and especially guile-wiredtiger and what it will be <amz3>than SQL and the ORM and the database configuration stuff that you need to do <amz3>nalaginrut: the problem with git for weblogs, is that the raw schema provided by the fs is very primitive and there is no library to make it better. <amz3>I think one of the reason less people share on the internet is because of static weblog <amz3>almost nobody (me included) maintains bookmark list online. It's all about writring (which takes time (and self promotion)) <amz3>anyway, my point is the schema that we can implement using wiredtiger is more powerful <amz3>BUT it's not useful for the planet <nalaginrut>you don't have to be constrained by the planet ;-) <amz3>I have some time, I just need to focus, anyway to be honest i started a blog using wiredtiger, it's a matter of reusing the procedures <ArneBab_>I like static pages, because they are really, really hard to compromise <nalaginrut>although I would recommend you build planet with the simplest approach as possible ***dje is now known as dje42
<amz3>how does `lift' compare to `compose' <amz3>2) where is lift in guile? <amz3>3) what do you think about using haunt for the planet? <davexunit>1) 'lift' is the name for an operation when working with monads, it's entirely different from compose. <davexunit>3) I don't think haunt makes sense for that. <paroneayea>ACTION debates again switching activitystuff back to GOOPS <paroneayea>namely because I'm kind of reimplementing the inheritance model myself <amz3>why are you re-implementing the inheritance model? <amz3>I'm curuious because I add this same idea <daviid>paroneayea: i was going to tell you exactly this when you started to talk about a while ago ... :) <daviid>it's like writing a complex app in C, oyou end rewriting a poor lisp ... <davexunit>everyone already knows my stance: just say no to OOP. :) <daviid>haha .. except if generic function multimethod polmorphic .. but say no to message-passing <amz3>daviid: is it possible with GOOPS to do url dispatch? <daviid>do use goops yes! be aware of its bugs though and way to use it until it's fixed <daviid>amz3: goops, in its current stage, doesn't dispatch on value, unfortunately, that could be fixed to, but right now it does not <daviid>paroneayea: I'm happy to help anyway, let me know <daviid>there is nothng in goops, based on stklos, which implements almost the full clos protocol by the way, that prevents of favour more then scheme itself ... <davexunit>anyway, I know I'm not going to sway anyone. <amz3>I'd like to know how to implement multiple "ContentTypes" ie. Photo, Slideshare, Diaporama, Article, Live blog, etc... using functional programming <amz3>I do guile to learn functional programming so I hope I'll learn some things ;) <paroneayea>I wanted to experiment with redoing it in a purely functional style <daviid>but anyway, not willing to spend time on this/these talks now, but yes, do use goops if you need inheritance <paroneayea>I'm not interested in goops as in terms of object mutation <paroneayea>daviid: re: "wait on bugs", I did think maybe I should wait till guile 2.2 is out with wingo's reworkings, if at all :) <daviid>paroneayea: i woudn't wait, neither use guile 2.2 [2.2 it has a teriible bug wrt merge-generics, which stable does not have], and since you won't mutate, you're fine [setters are not inherited, a serious bug, but since you won't use them]. the bugs I'm refering too can be circumvented, easily. All depends also on your stage of 'advanced/notadvanced' use of it... <mark_weaver>fwiw, I strongly discourage use of merge-generics, but I don't have time for that conversation now either :) <daviid>do use merge-generics, or do not use goops, but also don't have time now <mark_weaver>briefly, I believe that each generic should be defined and exported from a single module, and every other module that wants to use that generic, or add methods to it, should first import the module that exported the generic. <jusss`>does parallel function will destory threads after threads have done ? ***jusss` is now known as jusss
<paroneayea>mark_weaver: daviid: thanks... I do doubt I would use merge-generics anyhow :) <daviid>mark_weaver: agreed, but there is a bug in the module system, as it is, it will create a new generic function even when a module imports, therefore, i suggest, to paliate this bug, to merge ... <daviid>paroneayea: it is impossible not to, actually <daviid>if the module system would first import then see if a generic fucntioned exists and add, or create and add it would work, but a maintainer must fix the module system, wrt its nteraction with goops for that to work as expected <daviid>so, fairenough, in the mean time, i recommend, and until you really relly know what you're doing/taling about, to merge-generic as an app method [in a eval-when (expand ...) then to always use a specific macro to export acc, getters setters and methods, then you will be able to play without facing extremelly heisenbugs <paroneayea>I'll admit this has made me a bit more nervous about GOOPS usage again ;) <daviid>it is not goops, these are implementation mistakes, since 1.6 ... <daviid>and only 2 suggstions/things to be happy, no big deal <daviid>and I can help in case ... be happy to <daviid>with these two, you're good to play. ther are 2 other bugs, the setters are not inheritied and slot can't be redefined, so don't inherit slots and change the default #:init or be aware [i can paste the bug reports if you want] <daviid>A Grip of Really Important Procedures [thanks dsmith-work, he found the name, I did find the acronym] <paroneayea>humorously, I set up readline so I can test readline <davexunit>but maybe I should have it add load paths for all my projects for easier REPLing <paroneayea>I want to build a nice little REPL for the activitystreams stuff so a user can have a MUD-like interface <paroneayea>and so I wanted to try the completion with readline stuff <davexunit>so, I understand that readline is disabled by default for licensing compliance reasons <davexunit>but a lot of users don't seem to even know that it's an option <davexunit>I saw an old tweet yesterday saying that guile sucks because the repl doesn't have readline <mark_weaver>daviid: regarding "there is a bug in the module system, as it is, it will create a new generic function even when a module imports" did you file a bug report for that? I believe you are mischaracterizing the situation. if you follow my advice about always explicitly defining each generic in one module and exporting it, and importing it from all other modules that use it or add to it, I'd be surprised to hear of any such bug as you <mark_weaver>I suspect that the problem you are seeing is due to your way of doing things, which I do not recommend. <mark_weaver>paroneayea: fwiw, I don't use readline because I run my shells and guiles within emacs instead <paroneayea>mark_weaver: same here, though I'm building an experimental REPL for the federation stuff I'm doing <paroneayea>mark_weaver: mainly, I'm trying to build a simple interface to the federation tasks I'm workin on <daviid>mark_weaver: no one should never define a generic function, this is a big mistake, and hence a very bad advice imo, because the system will do it, if needed and it is a bug to redefine a generic function, by clos spec, which we should strictly implement/follow, imo too. therefore, this I really really do _not_ recommand, as opposed to you. <daviid> I did fill a bug report about the bad interaction between the module system and goops, as I mentionned to paroneayea earlier, bug#19459 <mark_weaver>paroneayea: as you can see, daviid and I strongly disagree with each other on this matter :) ***michel_mno is now known as michel_mno_afk
<daviid>:) paroneayea if you know deeply, really deeply guile's module system implementation, do as you wish, otherwise, I recommend you play with my settings first, grab and learn some clos tutorial, stklos as reasonibly good intro [our goops nmanual, is a 'copy' of it] http://www.stklos.net/Doc/html/stklos-ref-8.html#STklos-Object-System [goops implementation started with stklos source code fyi] , et voilà, post and ask, i'll be more then ***michel_mno_afk is now known as michel_mno
<paroneayea>I normally use bcrypt in python applications to do password hashing <paroneayea>I wonder if in guile I should use something from libgcrypt or something ***michel_mno_afk is now known as michel_mno
<paroneayea>guess GOOPS was excluded as an option anyway now that I found out that activities can be of multiple types in activitystreams :P <daviid>paroneayea: why would multiple types of/in activitystreams a problem? it seems at the contrary that this is exactly why goops would be a good option :) <paroneayea>daviid: multiple types per object, without subclassing! <daviid>that does not sound possible :) how could be a single object of different type ? <daviid>any pointer to what you're tyring to model? my quizz is id clos can't do it, no model can <mark_weaver>paroneayea: take a look at the r6rs conditions system, used for describing errors. sounds similar <mark_weaver>paroneayea: SRFI-35 is very similar to r6rs conditions <daviid>this, SRFI-35, and any similar, could perfectly be implemented using goops, but not all schemers are as lucky as guilers :) <mark_weaver>daviid: the fact that every method call in goops goes through globally-mutable method tables is a bad property, for some use cases. <mark_weaver>goops is very powerful, but it's a mistake to think that it's the best approach in all cases. <paroneayea>I have friendly little procedures for pulling data into and out of here now... <paroneayea>the multiple type stuff has lead me to decide to stick to my simple record stuff for now at least <paroneayea>if I have to implement *weird* type stuff, I guess I can do that on my own <daviid>mark_weaver: the way you express things does not suite/respect the spec, imo, and as such, participate to a false bad reputation of goops/clos: of course applicable methods evolves as you add method to generic functions, it is why it is powerfull <daviid>I don't say either it is perfect ofr everything :) but this condition system yes :) it is that SRFI has no goops <daviid>anyway, have to concentrate a bt on a not so nice task for now <daviid>there is no mutable table, unless a implementation detail, there is a function call applcable-methods and friends ... we should stick to the clos vocubalury imo <mark_weaver>daviid: sorry I don't remember the clos terminology, but every generic function has an associated set of methods that is mutable. it is indeed a mutable table. <daviid>no, unless you implement this way, the spec does not force implementors to have a mutable table <daviid>these are performance options for implementors <mark_weaver>sometimes that mutability is exactly what you need and want, e.g. for things like user interfaces or computer algebra systems along the lines of maxima. <daviid>but have to do other things now, sorry <daviid>you won't find even once the word table, mutable, immutable in the clos spec, mast last word for todat on this subject :) <mark_weaver>whether they chose to use those particular words does not imply that my statements are false. <mark_weaver>if you have some code in module A that calls a generic function 'foo', and then later someone adds a new class and some more methods to 'foo' to handle that new class, that *requires* mutation, whether they use that word or not. <mark_weaver>the behavior of 'foo' is being mutated, in the sense that its behavior depends on *when* it is called, i.e. whether it is called before or after some new method is added. <daviid>no, you call applicable-methods, there is no storage neither mutation of anything, also remeber <daviid>behavior of foo is never mutated by clos spec <daviid>you can call a million times the result is deterministcic <daviid>sorry, love to talk but I can't :( <paroneayea>just realizing that my usage of vhashes in activitystuff might not matter or make sense and maybe I might as well just go with hash tables I ask the user not to mutate... <paroneayea>when using guile-json I would convert things to vhashes, but of course I just realized I didn't do so recursively anyway, so! <mark_weaver>well, I had a long conversation about this once with Eli Barzilay, who is quite knowledgeable, and he explained to me that this mutation in CLOS is the main reason why they didn't use such a model for Racket's default OOP system. <davexunit>ACTION should've bought that book when I saw it for $5 at the MIT Press loading dock sale <mark_weaver>paroneayea: if the lists are not large, then plain association lists are probably superior. <mark_weaver>it just depends on whether you need the scalability. <paroneayea>mark_weaver: since I'd have to deserialize from json twice <paroneayea>once through guile-json, another time through my own code <davexunit>basically: one of these days I'm going to write guile-json2 <mark_weaver>it's really terrible that guile-json chose to use hash tables :-( <paroneayea>davexunit: guix doesn't process json structures all day every day as its main job ;p <davexunit>long before guix used it (which is my fault) <paroneayea>really I think the best solution in the long term: <davexunit>writing the reader/writer is trivial because thankfully JSON is a rather sane format <davexunit>but the question is what structure to use to distinguish "objects" from "arrays" <paroneayea>guile should have an immutable hashtable stuff we're "happy" with, and then a replacement for guile-json should be shipped with guile. <paroneayea>it is true that sometimes I process some pretty big json files, and not having O(n) performance when looking up a key <davexunit>paroneayea: using an immutable hash table wouldn't be great for this because we want to be able to write stuff to serialize to JSON in plain s-exps <davexunit>I'm more than happy to submit a (json) module for guile core if we can all agree upon the format <daviid>mark_weaver: racket did not implement clos, r-clos, raclos, racklos ... :) is a terrible mistake, and a totally different discussion <davexunit>paroneayea: that's exactly the thing I was talking about above. <davexunit>racket uses the vector type to distinguish arrays <mark_weaver>daviid: well, my point is just that Eli Barzilay told me that the Racket community decided against CLOS because of the mutation. <daviid>we have goops, not good yet but pretty close <mark_weaver>which seems to contradict your understanding of CLOS. <davexunit>paroneayea: because most of the time we have lists, if we used vectors our serialization code would have to call list->vector <mark_weaver>if you're right that CLOS doesn't require mutation of the behavior of generics, then both the racket community and me are apparently confused, and I'd like to hear about it some time. <paroneayea>davexunit: it might be nice to have a reader macro for that kind of cast to be used in the sexp-like builder <davexunit>so, one other option is to tag objects (or arrays) with a special symbol <davexunit>(@ (foo 1) (bar (2 3 4))) => { "foo": 1, "bar": [2, 3, 4] } <davexunit>the problem there is that you should be able to do (sexp->json (json->sexp foo)) and get something that is equal? to foo back <paroneayea>davexunit: shouldn't vhashes work then still? :) <davexunit>you can add a macro, and that's what guile-json does <mark_weaver>paroneayea: have you tried comparing the performance of vhashes with alists for small lists? <paroneayea>mark_weaver: no, and I'm sure for small lists it's fine ;p <paroneayea>maybe I just need to make a special deserializer that's in a contrib module to guile-json2 <davexunit>besides, part of the deserialization process is usually to instantiate proper objects <paroneayea>which can deserialize to something O(log(n)) or something :) <mark_weaver>anyway, I think that guile's json interface should work with alists, but of course users can then convert to a different structure internally where needed. <paroneayea>davexunit: I meant as part of the deserialization process :) <davexunit>serialization is a multi-step process. in guix-web I take package objects, convert them to a form suitable for guile-json, and then guile-json converts that to text <paroneayea>davexunit: glad I could help you (hash-set!) it in your mind <paroneayea>ACTION gets booed off the stage with rotten tomatoes! <paroneayea>davexunit: the nice thing about using @ I guess is it looks similar to the way @ works in sxml too <paroneayea>since the xml attributes are also a kind of mapping <davexunit>I wanted to write a YAML parser to use with guix for ruby stuff, but I looked at the spec and said NOPE <davexunit>it's literally the worst markup language ever <davexunit>it's hard to find sanity in anything around web development these days, but JSON is alright. <daviid>mark_weaver: these racket people should move to clojure :) and schemer should never ever use set! and any friends ... :)