IRC channel logs

<xelxebar>Has anyone played around with the support for other languages in guile?

<xelxebar>I was surprised to read that it has a (partial) ecmascript/javascript implementation and stuff

<davexunit>sneek: later tell amz3 I don't have much built-in for 3D yet but there is no limitation to using only 2D. it's OpenGL after all, where 2D is just a special case of 3D.

<sneek>Will do.

<xelxebar>Is there a way to search for/list available modules from the repl?

<brendyn>I had geiser repl running a process when I tried to run geiser-eval-definition in a buffer, which caused Emacs to freeze up. I canceled it with C-c C-k, but now Emacs is freezing up for several seconds whenever i so much as try to enter text into the repl and then tells me it is compiling my scheme file

<happy_gnu[m]>brendyn: is it still giving you trouble

<brendyn>I closed Emacs and started fresh and it's fine. I could try reproduce it but I wouldn't know how to debug such things

<brendyn>Yay, I'm on to chapter two of SICP now.

<happy_gnu[m]>brendyn: pretty cool!

<happy_gnu[m]>Are you experienced in programming

<brendyn>No I'm a beginner but I use the command line a lot and a have packaged a couple things for Guix

<happy_gnu[m]>brendyn: oh cool!

<happy_gnu[m]>Have you read how to design programs?

<brendyn>It really hurts my brain trying to solve the problems in SICP and I often come up with overly complicated code when Bill's solutions are only 4 lines but I'm better

<brendyn>Nope I just started with SICP

<happy_gnu[m]> http://www.ccs.neu.edu/home/matthias/HtDP2e/

<happy_gnu[m]>brendyn: what problem are you solving right now?

<happy_gnu[m]>I found how to design programs and the little schemer very helpful

<happy_gnu[m]>Specially The Little Schemer

<happy_gnu[m]> https://7chan.org/pr/src/The_Little_Schemer_4th_2.pdf

<brendyn>Nothing yet. I'm reading the text at the start of chapter 2, but yesterday I came up with this for 1.43, compared with Bill's solution http://paste.lisp.org/display/359820

<happy_gnu[m]>I think you are using things you learned from guix

<happy_gnu[m]>Why don't you give the little schemer a chance

<brendyn>When I first thought about how to write `repeated', I thought it would end up repeating exponentionally, so I started writing all that which ended accidently being an iterative solution

<happy_gnu[m]>You will advance a lot in a cfew days

<brendyn>and then a looked and the answer and realised how simple it should have been

<happy_gnu[m]>Is a very good book

<brendyn>You think I should postpone studying SICP?

<happy_gnu[m]>No

<happy_gnu[m]>You will do The little schemer very quickly

<happy_gnu[m]>It will take you a few days

<happy_gnu[m]>Until you hit chapter 8

<happy_gnu[m]>Then it becomes confusing but by then SICP is easier

<happy_gnu[m]>It iwill be just a few days

<brendyn>Hmm ok

<happy_gnu[m]>You don't have too though

<happy_gnu[m]>I just say because it helped me

<happy_gnu[m]>I am not advanced

<happy_gnu[m]>I started to learn in may

<happy_gnu[m]>More or less

<civodul>ACTION has a fix for the overly-weak hash tables \\o/

<civodul>it's all a matter of finding the right level of weakness

<wingo>ACTION landed instruction explosion for conditionals to master

<civodul>rekado_: just sent the updated patch set

<civodul>wingo: what's your take on going back to 2.0-style weak hash tables?

<wingo>civodul: i am not sure. the issue about the total count seems to be one you have solved so that is good. but i would not want to reintroduce weak pairs in any way to the api

<sneek>wingo, you have 2 messages.

<sneek>wingo, amz3` says: we need to push forward this bug https://debbugs.gnu.org/cgi/bugreport.cgi?bug=15228

<sneek>wingo, amz3` says: people keep asking that question

<civodul>wingo: right, weak pairs are only used internally, not visible from the outside

<civodul>or did you mean something else?

<wingo>no, that is what i meant

<wingo>have you been testing heavily with multi-threaded workloads?

<civodul>not specifically, i'll give it a try

<civodul>but it's essentially the same code as in 2.0

<civodul>i'll try that, and if there are no red flags, i'd like to go ahead and push it

<civodul>we can always go back to weak-tables if/when we have proper fixes for that

<civodul>though i'm not confident this can happen

<wingo>the 2.0 code did poorly in that case; that was the source of my attempts to refactor it

<wingo>additionally there are many uses of weak hash tables now in guile that assume they can be accessed in a thread-safe manner

<civodul>in which case, multi-threading?

<wingo>i think that's the only sane way to use weak hash tables: with some kind of atomicity guarantee

<civodul>sure

<civodul>do you have specific problems in mind?

<wingo>so if we switch back, we should make sure that two threads can add/remove to the weak hash table at the same time

<civodul>wait, weak tables and hash tables are not thread-safe per se

<wingo>something that's not the case for guile's core hash tables

<wingo>no

<wingo>weak tables and sets are thread-safe in that regard

<wingo>there is a mutex when adding/removing/accessing

<wingo>so you will find a number of areas in guile scheme that no longer take fat mutexes when accessing weak hash tables

<wingo>because the weak table ensures that concurrent access is safe

<wingo>i was sad that this didn't work out: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=10836#8

<civodul>riight

<wingo>but oh well :/

<wingo>finally i am not sure *why* the current weak table implementation doesn't provide the characteristics that you want

<wingo>i totally get that it's not doing the right thing atm :)

<civodul>it's weird to have thread-safe weak tables, and non-thread-safe hash tables, given they share the same Scheme-level API

<wingo>honestly i think they shouldn't share the same api but that is another story

<civodul>yes

<civodul>did you see my messages, notably https://debbugs.gnu.org/cgi/bugreport.cgi?bug=28590 ?

<civodul>i think i've explored the problem at length by now :-)

<civodul>reverting is really a last resort

<wingo>yes i saw all that and was a bit disappointed ivan hasn't looked at it, but then i haven't looked seriously either :)

<wingo>i have tried to follow your work but it takes a lot of time to be sure about anything in this area :/ i have been sure so many times in the past and been wrong :/

<civodul>yeah

<wingo>ah actually i hadn't seen https://github.com/ivmai/bdwgc/issues/183

<wingo>left for holiday about that time, back just recently

<wingo>so if we revert in 2.2 what does that buy us?

<wingo>just to know

<civodul>it fixes https://debbugs.gnu.org/cgi/bugreport.cgi?bug=28590

<wingo>that's a bad one indeed :)

<civodul>and reduces memory consumption noticeably on weak-table-heavy workloads, like compilation

<wingo>is that the source of the guix issues?

<jlicht>civodul: will this do something for the guix issues?

<civodul>it's one of the sources

<jlicht>oh, ninja'd

<civodul>the other source is data structures in the compiler

<wingo>that's the weird thing, compilation *shouldn't* be so weak-table-heavy...

<civodul>symbols, procedure properties, etc.

<civodul>there are lots of them when compiling

<wingo>most time is after the front-end, and it's only the front-end that needs weak associations

<wingo>the weak data gets into tree-il fairly directly and after that it's all strong references

<civodul>right, but just when reading gnu/packages/python.scm, you create a huge source property table

<civodul>then that big table has to be marked, etc.

<civodul>right, but still

<wingo>and that's really the issue? i guess i am just really surprised in that regard

<wingo>doesn't mean you are wrong of course :)

<civodul>yeah i think i posted more details about all this

<civodul>one profile i had was to just read the CPS-as-sexp representation, then look at the wall-clock time and heap size

<civodul>of course that's not exactly what happens when compiling python.scm

<civodul>but it does shed some light

<wingo>but the size of the open-adressed table should be similar to that of the buckets-and-chains table

<civodul>that's also unclear to me, as i wrote in a message some time ago

<civodul>that doesn't matter much anyway, given the other issues

<civodul>so if we go for the revert, how strong do you feel about adding locking to (weak) hash tables?

<civodul>obviously i'd like to avoid doing more work ;-), but also i'm not sure what this buys us

<wingo>so regarding 28590, from what i can see your fixes should fix that bug

<wingo>but that bug mixes the memory leak and incremental mark issue

<civodul>essentially the fixes there are not as good as reverting

<wingo>i feel pretty strongly about the thread-safety of weak hash tables

<wingo>civodul: you mean for memory leak or for the overall performance evaluation?

<civodul>so the problem to me is that, at a hashq-ref call site, you don't know if you're manipulating a thread-safe or a non-thread-safe table

<civodul>which to makes makes locking moot

<civodul>wingo: both

<wingo>weak hash tables are an implementation detail. you rarely want to expose them in api

<wingo>instead you want to expose some accessors

<wingo>so you know at your hashq-ref call site when you are working on a weak table

<civodul>well, sure

<wingo>ime anyway, yours may differ

<civodul>where does locking matters? do you have examples?

<wingo>yes, 633f3a18b7c6804b75ecd8ae94cf6cf82c9bcbed for example; also note that entries will be removed concurrently by GC so there are thread-safety concerns there; or 203a92b67b6a6c64c9e9f33d99c48f4699ed30e2;

<wingo>basically anywhere in guile there was a weak table, it needed to be protected by a mutex

<wingo>civodul: is the essential problem with the new hash tables that they are too big for a mark function to handle?

<wingo>besides the fixes you have in 28590 which are excellent fixes and important but not essential bugs with the data structure

<wingo>if i understand correctly anyway...

<wingo>so besides a revert (which is a real possibility)

<wingo>other possibilities: replace weak-table/weak-set impl with a buckets-and-chains impl

<wingo>behind same api, behind same struct weak_table with the same locking characteristics

<wingo>we could shard the weak table / weak set internally, if that's useful...

<wingo>i wish that there were some more standard weak hash table + libgc solution

<wingo>it seems like we are really struggling to do something that libgc doesn't want to do

<wingo>dunno tho

<wingo>we could possibly implement a concurrent hash table

<wingo>perhaps one that locks each bucket instead of the table as a whole

<wingo>wdyt civodul

<wingo>i can give a try to implementing that

<wingo>i think in the future we should use the deprecation mechanism to move people away from using the normal hash table API on weak tables, to use a specific api

<wingo>like weak-vector-ref vs vector-ref

<wingo>they don't have to be all that different in form / structure but i think they should be separate

<wingo>in that regard i think it makes sense to keep weak and strong hash table implementations separate, and i think we should tolerate a bit of difference between the two (e.g. threadsafety)

<wingo>so for preference i would rather replace the existing weak table implementation but keep things in weak-table.c (out of hashtab.c)

<wingo>civodul: you back? just wanted to close this loop

<civodul>ACTION is back to keyboard, sorry for the delay

<civodul>yes, so, i explained the problem in the thread rooted at https://lists.gnu.org/archive/html/guile-devel/2017-09/msg00031.html

<civodul>and in the other bug report about unbounded growth

<civodul>unbounded growth in itself is enough of a problem, esp. for long-running processes

<civodul>so i think the status quo is bad

<civodul>started with a new implementation again doesn't sound reasonable

<civodul>so going back to the bucket approach, with code that's already well used, sounds reasonable to me

<wingo>the unbounded growth is of course bad but just a bug afaiu (which you tracked down and fixed)

<civodul>in https://debbugs.gnu.org/cgi/bugreport.cgi?bug=28590 i fixed the unbounded growth of the disappearing-link table

<civodul>the unbounded growth of the table itself isn't really fixed: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=28590#22

<civodul>and overall i'm less confident with the resulting code than with the good'ol buckets

<civodul>notably because marking the whole thing remains a bottleneck

<civodul>as discussed at https://lists.gnu.org/archive/html/guile-devel/2017-09/msg00031.html

<wingo>sure

<wingo>what if we put the old implementation in weak-table.c ?

<civodul>in what sense?

<wingo>that's what i meant by "replace weak-table/weak-set impl with a buckets-and-chains impl"

<civodul>the revert i sent preserve the subset of the weak-table C API that's used internally

<civodul>so i think it's essentially equivalent

<wingo>but it put the implementation back in hashtab.c, right?

<civodul>right

<wingo>it didn't preserve the threadsafety aspect

<wingo>right?

<civodul>indeed

<civodul>i'm not convinced it matters though

<wingo>i feel pretty strongly about it fwiw

<civodul>i feel it's a property that cannot be relied upon, and that's not documented

<civodul>so, the bottom line is i'm not going to spend more days on this

<civodul>so we need a resolution

<wingo>do you want my help? i am getting negative signals here

<wingo>it seems to me that what i was suggesting meets all your needs

<wingo>i.e. old implementation in weak-table.c

<wingo>and i offered to do it, but you didn't want that? i am a bit confused there

<civodul>oh if you offer to do it, that's fine with me!

<civodul>sorry i didn't get that part

<civodul>so yes, definitely

<wingo>np :)

<wingo>sure, let me give it a go directly

<civodul>and i'll be happy to run my "benchmarks" with it

<civodul>awesome

<wingo>oh that would be great

<dsmith-work>Morning Greetings, Guilers

<civodul>talks from the SPLASH conference: https://www.youtube.com/channel/UCgNOrQWOr3rKR9XPB1Km66Q/videos?sort=dd&view=0&shelf_id=0

<civodul>there's little Scheme, but lots of excellent stuff

<ays>hi everyone! I wanted to know if there was a way to configure guile with a startup script. Something like .guilerc?

<daviid>ays: it is .guile actually

<ays>daviid: Oh thanks!

<ays>Where do I install guile-sdl2 from?

<daviid> https://dthompson.us/projects/guile-sdl2.html

<stis>hello guilers!

<OrangeShark>hello stis

<OrangeShark>hooverville: https://erikedrosa.com/2017/10/29/guile-projects-with-autotools.html

<OrangeShark>sneek: later tell jeko the blog post is up https://erikedrosa.com/2017/10/29/guile-projects-with-autotools.html

<sneek>Okay.

<jlicht>OrangeShark: nice post, I wish I had had this half a year ago

<OrangeShark>jlicht: thanks, I've been meaning to write this a year ago but kept putting it off.

<jlicht>OrangeShark: Looking forward to more posts about tests and documentation indeed :-)

<jlicht>OrangeShark: Incidentally, what is the license for your blogpost?

<OrangeShark>jlicht: yes :) I want to also add guix into the mix as well.

<OrangeShark>jlicht: I should probably put an appropriate license, shouldn't I?

<jlicht>OrangeShark: You should do whatever you feel like, of course. But something which allows others in the community to adapt (translate, embed etc etc) your work could be worthwhile indeed :)

<OrangeShark>jlicht: do you recommend any license for that?

<jlicht>OrangeShark: IANAL, and not really a license buff myself, so I can't really help your there.

<jlicht>You could try the GFDL (https://www.gnu.org/licenses/recommended-copylefts.html), but I am not sure if there are any drawbacks to using that. Maybe some of the guile devs can help?

<davexunit>use a creative commons license for blog posts

<OrangeShark>jlicht: that seems more for documentation

<davexunit>the code snippets can use a software license, say MIT Expat

<davexunit>(copylefting code snippets seems counterproductive)

<OrangeShark>davexunit: thanks, I will look into those licenses

<jlicht>thanks for the tip davexunit. Incidentally, which license applies to the code snippets on your blog?

<davexunit>jlicht: I don't think I ever explicitly state it anywhere, but let's just say MIT Expat :)

<jlicht>the bigger pieces have their own GPL header, so that is somehow clear

<jlicht>thanks for the clarification :-)

<davexunit>yeah when I copy code from projects I use the original license

<davexunit>but consider a freestanding example snippet with no license to use a lax free license

<dustyweb> https://lists.gnu.org/archive/html/guile-devel/2017-10/msg00045.html was an exciting email to read today

<bavier>any way to prevent ldconfig from printing this: "/sbin/ldconfig: /usr/local/lib/libguile-2.0.so.22.8.1-gdb.scm is not an ELF file - it has the wrong magic bytes at the start."?

<dsmith-work>bavier: Nope. It's a bogus message from ldconfig (IMHO)

<dsmith-work>The various lib dirs were not just for shared object files. Originally, all kinds of "support" files where in there.

<bavier>makes sense

<bavier>I wonder if the *.so prefix is throwing it off

<dsmith-work>Maybe. But it should see it's a .scm file.

<dsmith-work>My opinion

<dsmith-work>I just sing "la la la" with fingers in ears when I see that.

<janneke>o/

<dsmith-work>bavier: Ya, they could fix it.

<dsmith-work> /* The file is neither ELF nor aout. Check if it's a linker

<dsmith-work> script, like libc.so - otherwise complain. Only search the

<dsmith-work> beginning of the file. */

<dsmith-work> size_t len = MIN (statbuf.st_size, 512);

<dsmith-work> if (memmem (file_contents, len, "GROUP", 5) == NULL

<dsmith-work> && memmem (file_contents, len, "GNU ld script", 13) == NULL

<dsmith-work> && !is_gdb_python_file (file_name))

<dsmith-work> error (0, 0, _("%s is not an ELF file - it has the wrong magic bytes at the start.\\n"),

<dsmith-work> file_name);

<dsmith-work>They have a special case for python!

<dsmith-work>Why not for guile too?

<dsmith-work>from glibc/elf/readlib.c

IRC channel logs

2017-10-30.log