IRC channel logs
2017-10-30.log
back to list of logs
<xelxebar>Has anyone played around with the support for other languages in guile? <xelxebar>I was surprised to read that it has a (partial) ecmascript/javascript implementation and stuff <davexunit>sneek: later tell amz3 I don't have much built-in for 3D yet but there is no limitation to using only 2D. it's OpenGL after all, where 2D is just a special case of 3D. <xelxebar>Is there a way to search for/list available modules from the repl? <brendyn>I had geiser repl running a process when I tried to run geiser-eval-definition in a buffer, which caused Emacs to freeze up. I canceled it with C-c C-k, but now Emacs is freezing up for several seconds whenever i so much as try to enter text into the repl and then tells me it is compiling my scheme file <brendyn>I closed Emacs and started fresh and it's fine. I could try reproduce it but I wouldn't know how to debug such things <brendyn>Yay, I'm on to chapter two of SICP now. <brendyn>No I'm a beginner but I use the command line a lot and a have packaged a couple things for Guix <brendyn>It really hurts my brain trying to solve the problems in SICP and I often come up with overly complicated code when Bill's solutions are only 4 lines but I'm better <happy_gnu[m]>I found how to design programs and the little schemer very helpful <brendyn>When I first thought about how to write `repeated', I thought it would end up repeating exponentionally, so I started writing all that which ended accidently being an iterative solution <brendyn>and then a looked and the answer and realised how simple it should have been <brendyn>You think I should postpone studying SICP? <civodul>ACTION has a fix for the overly-weak hash tables \\o/ <civodul>it's all a matter of finding the right level of weakness <wingo>ACTION landed instruction explosion for conditionals to master <civodul>rekado_: just sent the updated patch set <civodul>wingo: what's your take on going back to 2.0-style weak hash tables? <wingo>civodul: i am not sure. the issue about the total count seems to be one you have solved so that is good. but i would not want to reintroduce weak pairs in any way to the api <sneek>wingo, amz3` says: people keep asking that question <civodul>wingo: right, weak pairs are only used internally, not visible from the outside <wingo>have you been testing heavily with multi-threaded workloads? <civodul>not specifically, i'll give it a try <civodul>but it's essentially the same code as in 2.0 <civodul>i'll try that, and if there are no red flags, i'd like to go ahead and push it <civodul>we can always go back to weak-tables if/when we have proper fixes for that <civodul>though i'm not confident this can happen <wingo>the 2.0 code did poorly in that case; that was the source of my attempts to refactor it <wingo>additionally there are many uses of weak hash tables now in guile that assume they can be accessed in a thread-safe manner <wingo>i think that's the only sane way to use weak hash tables: with some kind of atomicity guarantee <civodul>do you have specific problems in mind? <wingo>so if we switch back, we should make sure that two threads can add/remove to the weak hash table at the same time <civodul>wait, weak tables and hash tables are not thread-safe per se <wingo>something that's not the case for guile's core hash tables <wingo>weak tables and sets are thread-safe in that regard <wingo>there is a mutex when adding/removing/accessing <wingo>so you will find a number of areas in guile scheme that no longer take fat mutexes when accessing weak hash tables <wingo>because the weak table ensures that concurrent access is safe <wingo>finally i am not sure *why* the current weak table implementation doesn't provide the characteristics that you want <wingo>i totally get that it's not doing the right thing atm :) <civodul>it's weird to have thread-safe weak tables, and non-thread-safe hash tables, given they share the same Scheme-level API <wingo>honestly i think they shouldn't share the same api but that is another story <civodul>i think i've explored the problem at length by now :-) <wingo>yes i saw all that and was a bit disappointed ivan hasn't looked at it, but then i haven't looked seriously either :) <wingo>i have tried to follow your work but it takes a lot of time to be sure about anything in this area :/ i have been sure so many times in the past and been wrong :/ <wingo>left for holiday about that time, back just recently <wingo>so if we revert in 2.2 what does that buy us? <civodul>and reduces memory consumption noticeably on weak-table-heavy workloads, like compilation <wingo>is that the source of the guix issues? <jlicht>civodul: will this do something for the guix issues? <civodul>the other source is data structures in the compiler <wingo>that's the weird thing, compilation *shouldn't* be so weak-table-heavy... <civodul>there are lots of them when compiling <wingo>most time is after the front-end, and it's only the front-end that needs weak associations <wingo>the weak data gets into tree-il fairly directly and after that it's all strong references <civodul>right, but just when reading gnu/packages/python.scm, you create a huge source property table <civodul>then that big table has to be marked, etc. <wingo>and that's really the issue? i guess i am just really surprised in that regard <wingo>doesn't mean you are wrong of course :) <civodul>yeah i think i posted more details about all this <civodul>one profile i had was to just read the CPS-as-sexp representation, then look at the wall-clock time and heap size <civodul>of course that's not exactly what happens when compiling python.scm <wingo>but the size of the open-adressed table should be similar to that of the buckets-and-chains table <civodul>that's also unclear to me, as i wrote in a message some time ago <civodul>that doesn't matter much anyway, given the other issues <civodul>so if we go for the revert, how strong do you feel about adding locking to (weak) hash tables? <civodul>obviously i'd like to avoid doing more work ;-), but also i'm not sure what this buys us <wingo>so regarding 28590, from what i can see your fixes should fix that bug <wingo>but that bug mixes the memory leak and incremental mark issue <civodul>essentially the fixes there are not as good as reverting <wingo>i feel pretty strongly about the thread-safety of weak hash tables <wingo>civodul: you mean for memory leak or for the overall performance evaluation? <civodul>so the problem to me is that, at a hashq-ref call site, you don't know if you're manipulating a thread-safe or a non-thread-safe table <wingo>weak hash tables are an implementation detail. you rarely want to expose them in api <wingo>instead you want to expose some accessors <wingo>so you know at your hashq-ref call site when you are working on a weak table <wingo>ime anyway, yours may differ <civodul>where does locking matters? do you have examples? <wingo>yes, 633f3a18b7c6804b75ecd8ae94cf6cf82c9bcbed for example; also note that entries will be removed concurrently by GC so there are thread-safety concerns there; or 203a92b67b6a6c64c9e9f33d99c48f4699ed30e2; <wingo>basically anywhere in guile there was a weak table, it needed to be protected by a mutex <wingo>civodul: is the essential problem with the new hash tables that they are too big for a mark function to handle? <wingo>besides the fixes you have in 28590 which are excellent fixes and important but not essential bugs with the data structure <wingo>if i understand correctly anyway... <wingo>so besides a revert (which is a real possibility) <wingo>other possibilities: replace weak-table/weak-set impl with a buckets-and-chains impl <wingo>behind same api, behind same struct weak_table with the same locking characteristics <wingo>we could shard the weak table / weak set internally, if that's useful... <wingo>i wish that there were some more standard weak hash table + libgc solution <wingo>it seems like we are really struggling to do something that libgc doesn't want to do <wingo>we could possibly implement a concurrent hash table <wingo>perhaps one that locks each bucket instead of the table as a whole <wingo>i can give a try to implementing that <wingo>i think in the future we should use the deprecation mechanism to move people away from using the normal hash table API on weak tables, to use a specific api <wingo>like weak-vector-ref vs vector-ref <wingo>they don't have to be all that different in form / structure but i think they should be separate <wingo>in that regard i think it makes sense to keep weak and strong hash table implementations separate, and i think we should tolerate a bit of difference between the two (e.g. threadsafety) <wingo>so for preference i would rather replace the existing weak table implementation but keep things in weak-table.c (out of hashtab.c) <wingo>civodul: you back? just wanted to close this loop <civodul>ACTION is back to keyboard, sorry for the delay <civodul>and in the other bug report about unbounded growth <civodul>unbounded growth in itself is enough of a problem, esp. for long-running processes <civodul>started with a new implementation again doesn't sound reasonable <civodul>so going back to the bucket approach, with code that's already well used, sounds reasonable to me <wingo>the unbounded growth is of course bad but just a bug afaiu (which you tracked down and fixed) <civodul>and overall i'm less confident with the resulting code than with the good'ol buckets <civodul>notably because marking the whole thing remains a bottleneck <wingo>what if we put the old implementation in weak-table.c ? <wingo>that's what i meant by "replace weak-table/weak-set impl with a buckets-and-chains impl" <civodul>the revert i sent preserve the subset of the weak-table C API that's used internally <civodul>so i think it's essentially equivalent <wingo>but it put the implementation back in hashtab.c, right? <wingo>it didn't preserve the threadsafety aspect <wingo>i feel pretty strongly about it fwiw <civodul>i feel it's a property that cannot be relied upon, and that's not documented <civodul>so, the bottom line is i'm not going to spend more days on this <wingo>do you want my help? i am getting negative signals here <wingo>it seems to me that what i was suggesting meets all your needs <wingo>i.e. old implementation in weak-table.c <wingo>and i offered to do it, but you didn't want that? i am a bit confused there <civodul>oh if you offer to do it, that's fine with me! <wingo>sure, let me give it a go directly <civodul>and i'll be happy to run my "benchmarks" with it <civodul>there's little Scheme, but lots of excellent stuff <ays>hi everyone! I wanted to know if there was a way to configure guile with a startup script. Something like .guilerc? <ays>Where do I install guile-sdl2 from? <jlicht>OrangeShark: nice post, I wish I had had this half a year ago <OrangeShark>jlicht: thanks, I've been meaning to write this a year ago but kept putting it off. <jlicht>OrangeShark: Looking forward to more posts about tests and documentation indeed :-) <jlicht>OrangeShark: Incidentally, what is the license for your blogpost? <OrangeShark>jlicht: yes :) I want to also add guix into the mix as well. <OrangeShark>jlicht: I should probably put an appropriate license, shouldn't I? <jlicht>OrangeShark: You should do whatever you feel like, of course. But something which allows others in the community to adapt (translate, embed etc etc) your work could be worthwhile indeed :) <jlicht>OrangeShark: IANAL, and not really a license buff myself, so I can't really help your there. <davexunit>use a creative commons license for blog posts <davexunit>the code snippets can use a software license, say MIT Expat <davexunit>(copylefting code snippets seems counterproductive) <jlicht>thanks for the tip davexunit. Incidentally, which license applies to the code snippets on your blog? <davexunit>jlicht: I don't think I ever explicitly state it anywhere, but let's just say MIT Expat :) <jlicht>the bigger pieces have their own GPL header, so that is somehow clear <jlicht>thanks for the clarification :-) <davexunit>yeah when I copy code from projects I use the original license <davexunit>but consider a freestanding example snippet with no license to use a lax free license <bavier>any way to prevent ldconfig from printing this: "/sbin/ldconfig: /usr/local/lib/libguile-2.0.so.22.8.1-gdb.scm is not an ELF file - it has the wrong magic bytes at the start."? <dsmith-work>bavier: Nope. It's a bogus message from ldconfig (IMHO) <dsmith-work>The various lib dirs were not just for shared object files. Originally, all kinds of "support" files where in there. <bavier>I wonder if the *.so prefix is throwing it off <dsmith-work>I just sing "la la la" with fingers in ears when I see that. <dsmith-work> /* The file is neither ELF nor aout. Check if it's a linker <dsmith-work> script, like libc.so - otherwise complain. Only search the <dsmith-work> if (memmem (file_contents, len, "GROUP", 5) == NULL <dsmith-work> && memmem (file_contents, len, "GNU ld script", 13) == NULL <dsmith-work> error (0, 0, _("%s is not an ELF file - it has the wrong magic bytes at the start.\\n"),