IRC channel logs

2020-07-18.log

back to list of logs

<manumanumanu>Good night everyone!
***catonano_ is now known as catonano
<a_v_p>Hello Guilers! Guile-SSH author is here.
<manumanumanu>Hi ho!
<a_v_p>How do you debug Guile C code in GDB? Namely I need to see what's inside an SCM object in the debugger.
<a_v_p>"print obj" gives me only "$2 = (SCM) 0x55c52d5705a0"
<a_v_p>That's not very helpful.
***nikita_ is now known as nikita`
<manumanumanu>a_v_p: sorry, went afk. I have very little experience with that. The manual says this: https://www.gnu.org/software/guile/manual/html_node/GDB-Support.html
<a_v_p>manumanumanu: Thanks! I've read this part of the manual indeed, but it seems that I have no Guile extension for GDB in Ubuntu GNU/Linux 20.04.
<manumanumanu>oh...
<manumanumanu>a_v_p: seems to be a debian thing
<manumanumanu>or a gdb thing? It seems disabled by default.
<R1ck77>Hi! If I create a SCM object in C, like 'SCM thing = scm_from_utf8_string("something");' how long does it remain valid? Is it eligible to be garbage collected? If not, how do I get rid of it?
<RhodiumToad>it remains valid as long as something in memory points to it
<R1ck77>ok, so I guess I have to do a "free" on it, in C
<R1ck77>reasonable. Thank you!
<RhodiumToad>no
<R1ck77>no?
<R1ck77>if I create a SCM object in C (the aforementioned guile string) nothing is "pointing" to it apparently. It seemed a logical conclusion that I owned the object in some way
<R1ck77>(although my "free" remark was plain dumb. I got confused with plain strings converted FROM scheme)
<RhodiumToad>as long as the value is in a machine register, local variable on the stack, or pointed to by another scheme object or a global or static variable, or pointed to from a C data structure that Scheme has been told about,
<RhodiumToad>then the garbage collector will leave it alone
<RhodiumToad>(see "Garbage collection" under Programming in C -> General Libguile Concepts in the docs
<RhodiumToad>once it's no longer pointed to by anything the garbage collector knows about, then it is eligible for collection and will be freed at the next opportunity
*R1ck77 did just read the GC section
<R1ck77>I was completely wrong. To my defense, I didn't expect the GC could be so smart to actually find the SCM objects on the stack and on the registers. I'm positively impressed.
<R1ck77>my (wrong) assumption was -> GC cannot find my C objects -> I own them -> I need to do "something" with them
<R1ck77>wow.
<R1ck77>thank you, RhodiumToad
<RhodiumToad>that said, reading the gc docs does induce some slight skepticism
<RhodiumToad>the way that compilers are becoming increasingly aggressive about weird optimizations is cause for concern
<RhodiumToad>(the GC is not actually part of Guile, it's a separate library)
<RhodiumToad>"Some C optimizers may lose the last undisguised pointer to a memory object as a consequence of clever optimizations. This has almost never been observed in practice."
<R1ck77>yes, I know, libgc. Good choice I can say
<R1ck77>well, about your last remark, a 64 bit memory space is a very very large search space so it's not that surprising
<R1ck77>I admit that having always worked with non-gc embeddings (lua and python specifically) I feel definitely out of my confort zone
<R1ck77>still, I hope to manage to get guile to run on Android (not an easy task up to now): I would love to use it for app development
<RhodiumToad>lua is garbage collected...
<R1ck77>yes, but in the same thread, it's all very predictive\
<RhodiumToad>the main thing in lua is that the actual object internals are all hidden inside the API
<RhodiumToad>so as long as you make sure you have a value (string or userdata) on the lua stack while you poke at its innards, you're ok
<RhodiumToad>the only real rough edge is the number to string mutation in lua_tostring
<R1ck77>completely correct. Let's say that what I'm trying - very confusedly - to say, is that it's so simple "it feels" like a reference counting thing when you use it
<R1ck77>now it's been a while since I did serious work on it, but after reading the lua programming guide the general impression was of control and predictability
<R1ck77>why did you said that lua_tostring is a rough edge?
<R1ck77>(again, it's been a while, but I don't recall any special issue with it)
<RhodiumToad>if stack index n contains a number, and you do char *s = lua_tostring(L, n); then that converts the number to a string _and replaces the stack entry with the string value_
<RhodiumToad>it does the replacement because otherwise, nothing would be anchoring the returned string
<R1ck77>strange I never stumbled into this thing (or that I did and I don't remember about it)
<R1ck77>maybe I never held on lua values but always kept them inside of the lua state, so I never noticed/cared? :/
<RhodiumToad>the case where it causes the most trouble is iterating a table with lua_next
<R1ck77>yep, I just read the lua_tolstring and it's mentioned
<RhodiumToad>if you call lua_tostring on the stack index of the returned key without copying it first, you mess up the iteration if the key was actually a number
<R1ck77>I guess you are pulling the carpet from it
<R1ck77>in retrospect I think I probably duplicated each and every number on the stack before converting them
<RhodiumToad>that works :-)
<R1ck77>play it safe (and dumb :-) )
<R1ck77>back to work. I got a Guile library and an embedding that compiles, now I "just" need to understand why guile segfaults (on termux) or hangs the app (when used as a library) :)
<R1ck77>native debugging on an Android machine. How hard could it be?
<R1ck77>RhodiumToad, since you seem very informed - all around, not just from my n00b perspective - I'm working on 2.2 right now, do you think moving to guile 3.0 is worth the time? Do you know if anybody did work on Android compatibility or portability between 2.2 and 3.)?
<RhodiumToad>I know little of android
<RhodiumToad>3.0.x jit is broken when using clang (not gcc) on 32-bit arm
<R1ck77>oooook....
<RhodiumToad>otherwise, 3.0 seems good
<R1ck77>well, the native development kit is based on clang and 32 bit arm is half of my architectures, so I think I have my answer (2.2 or bust...)
<RhodiumToad>ah.
<RhodiumToad>the problem is known, it just needs to be sorted out what the best way to fix it is
<RhodiumToad>but the fact that it doesn't manifest with gcc has reduced interest in it, I think
<RhodiumToad> https://gitlab.com/wingo/lightening/-/issues/15
<R1ck77>the problem strikes me more for what it implies, rather by itself: the take away is that probably Android as a platform is hardly considered
<RhodiumToad>yes, that may be true
<RhodiumToad>I found that issue using freebsd on an RPI
<RhodiumToad>I'm a relative newcomer to guile, but I don't recall android having been mentioned here much
<R1ck77>well, It's seems a nail more in the coffin of my "let's see how Guile would be as a development tool on Android" thing :(
<RhodiumToad>personally I'm happy to help out with getting it fixed
<R1ck77>I mean, I think the language is great, the idea behind it is great, but up to now when I try to use it, Guile seems to fight back
<R1ck77>are you one of the main contributors?
<RhodiumToad>no
<RhodiumToad>wingo and civodul both show up in this channel from time to time
<R1ck77>If I may ask, what do you use Guile for, RhodiumToad ?
<RhodiumToad>so far, mostly for learning scheme and some functional programming tricks
<R1ck77>you have an assembly background? I did read your bug report, you seem to be confortable in getting your hands dirty in ASM more than I would :)
<RhodiumToad>yeah, assembly was the second language I learned after basic, and lisp was the 3rd or 4th
<RhodiumToad>(we're talking 40-odd years ago)
<R1ck77>WOW
<R1ck77>I feel a bit better now, at least
<R1ck77>I mean, with "only" 20 years of coding under my belt now it's clear why I felt so overwhelmingly amateurish xD
<RhodiumToad>finding this bug: https://gitlab.com/wingo/lightening/-/commit/1bb909a44d2303f88bb05125fc6742e97f80cd1d
<RhodiumToad>was probably harder than the other one
<R1ck77>jeezz..
<R1ck77>I have very little experience with assembly and low level programming in general (I learned a bit of x86 and ARM assembly in my spare time and had fun with it, but that's it...), but now I'm working with a lot of (competent but young) people who have no clue about anything under a JVM
<R1ck77>I guess being the only "senior" around who knows what an opcode is sort of inflated my ego
<R1ck77>thank you for pulling me back on earth and returning me some sense of perspective :D
<R1ck77>anyway going back to Guile, I can't escape the feeling that no matter how much I seem to like it and how good it may be, it's practically dead :(((
<R1ck77>I mean, there has been this new lisp/scheme wave with Clojure, Racket and other languages on the forefront, with people creating new lisp languages all around, but Guile doesn't seem to have been touched by it
<R1ck77>RhodiumToad, thank you for the help (and congratulations about your journey as a programmer: I look forward from being able to say, one day "I did that 40-odd years ago" too)
<dsmith>a_v_p: The HACKING file mentions using the gdbinit file in the guile sources.
<dsmith>Has things in it like
<dsmith>define gdisplay
<dsmith> call (void)scm_display ($arg0, scm_current_error_port ())
<dsmith> newline
<dsmith>end
<dsmith>Which will help seeing SCM value
<dsmith>Wow. Looks like lots of cool stuff in there..
<ArneBab>sneek: later tell R1ck77 please tell me if you get Guile working for app development. I‘d love to use it to *start* into app dev
<sneek>Okay.
<ArneBab>sneek: later tell R1ck77 regarding being dead: Guile has been taken up by Guix and has started flying, not so much in publishing but rather in the OS work
<sneek>Okay.
<a_v_p>dsmith: Thanks, I'll give it a try.
<tohoyn>sneek, botsnack
<sneek>:)
<a_v_p>Woohoo, it seems that I fixed most of the random segfaults in Guile-SSH on the 'master'!
<a_v_p>The tests would sometimes fail mostly due to concurrency and GC'ing issues.
<mwette>a_v_p: sweet!
<dsmith>a_v_p: Awesome. Did that gdbinit help?
<a_v_p>A new Guile-SSH release is here: https://github.com/artyom-poptsov/guile-ssh/releases/tag/v0.13.0
<a_v_p>dsmith: No, I haven't checked this GDB feature yet, I'll do it later.
<dsmith>a_v_p: To be clear, this is an init file that adds some functions to gdb the help debug Guile C code.
<dsmith>It is NOT adding Guile to gdb so you can use Guile to script gdb for debugging other things.
<a_v_p>dsmith: Yes, I know. And I was looking exactly for something like that, thanks.
<dsmith>Cool.
<dsmith>I totally forgot about it when I was debugging something a few weeks ago.
<a_v_p>I was able to fix segfaults by eliminating some bad multi-threading from Scheme code.
<a_v_p>When one is using the FFI in Guile with some tricky library code below the surface, it's easy to mess with concurrent GC'ing, especially with multi-threading.
<a_v_p>Some Guile-SSH procedures would SEGFAULT as I was trying to use an libssh channel when the parent libssh session was freed.
<a_v_p>The worst part is that such errors are hard to track down. That's where GDB with Guile extensions would be very helpful.
<dsmith>multi-thread and gc issues are the worst! Because cause and manifestaion are so separated by both code space and time. And usage patters. Ugh.
<ArneBab>a_v_p: how did you fix them?
<ArneBab>I had some very nasty segfaults when trying to do multi-threading with futures and atomic variables, but I did not report it since I found a workaround :-/
<a_v_p>ArneBab: The most obvious fix was to remove threading in 'call-with-ssh-forward' procedure.
<a_v_p>Also I added additional checks on SMOBS GC callbacks.
<a_v_p>And added 'scm_set_port_needs_close_on_gc (channel_tag, 1)' call to Guile-SSH channel type initialization code.
<ArneBab>a_v_p: so you now only thread outside?
<a_v_p>ArneBab: It's better to use separate processes. Here an example: https://github.com/artyom-poptsov/guile-ssh/blob/master/examples/rpc/client.scm.in#L86
<a_v_p>The code starts SSH port forwarding in a spawned process and then connects to the forwarded socket from the main process.
<ArneBab>that sounds like a hard hack to fix something which should have worked …
<ArneBab>maybe we should get together and gather problems
<ArneBab>to get trheading to just work
<ArneBab>my solution was to move the future inside the procedure instead of running procedures in futures.
<a_v_p>FWIW, I was looking for something like a "POSIX socket forwarder" that forwards all I/O on an "virtual" socket to a specified Guile port (better yet -- using the caller's thread), but found no such thing. So I proposed to use a separate process that forwards the data.
<rlb>Hmm, not sure if it would be relevant, but a bit back I noticed that you can forward file descriptors over local sockets to another process.
<rlb>(worked quite well for the case I had in mind)
<ArneBab>I had a similar problem: I‘m communicating with TCP process and want to multiplex messages I send over different threads.
<ArneBab>this is my socked-handling procedure: https://notabug.org/ArneBab/guile-freenet/src/master/fetchpull.w#L709
<ArneBab>looking at the code again now, I see that I basically removed the multithreading from anything that writes to the port
<ArneBab>I now have a read thread and a write thread and they synchronize over an atomic box: https://notabug.org/ArneBab/guile-freenet/src/master/fetchpull.w#L311
<ArneBab>I‘m now blocking everything on the network, because it didn‘t work otherwise
<ArneBab>What I would have needed: A threadsafe queue where I can put messages so the sender thread can take them out and one where the reader thread pushes messages that the handlers can take out; with limited size so reading/putting blocks when too many messages are in queue
<rlb>Yeah: https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/BlockingQueue.html I may hack something like that up at some point if we don't already have it.
<rlb>While I may have some issues with the jvm, java.util.concurrent is very useful.
<ArneBab>though concurrency in Java is still ugly
<ArneBab>(but more reliable)
<rlb>fwiw, clojure has a good bit of precedent here too.
<rlb>e.g. https://clojure.github.io/clojure/clojure.core-api.html#clojure.core/seque
<rlb>though somwhat different model from what you describe, perhaps
<ArneBab>I’d actually need a threadsafe deque
<ArneBab>as a fifo
<ArneBab>I wonder whether I might already have one from another project I hacked together …
<ArneBab> https://github.com/scheme-requests-for-implementation/srfi-134/blob/master/contrib/arne-babenhauserheide/deque.scm
<ArneBab>since this is immutable, thread-safety with a global value should only depend on atomic blocking of a variable
<roelj>I can't seem to find an example of how to call a Guile function from C. I thought I'd seen an example in the manual about two years ago.. Does anyone know where it is?