<roelj>I have a core dump of a program that mostly uses libguile and guile scheme code. The process I dumped seems to leak memory, so I'd like to analyze the memory. Does anyone have tips to do this? I loaded the dump in gdb, but am quite lost :) <civodul>roelj: hi! you might want to start by profiling heap consumption at the Scheme level, using gcprof <roelj>civodul: Hi! Good to see you again :) Can I do that "offline" -- as in, after the process has ended? <roelj>civodul: It would be really cool if I can "simply" read the contents of heap memory, so I can see if anything sticks out. <civodul>roelj: no, you cannot do that offline <civodul>it'd be great to be able to scan the heap and get a usage profile <civodul>what you could do is periodically dump gc-stats so you can plot heap usage over time, if it's a long-running process <roelj>civodul: Can I also run gcprof without running the entire process in the thunk passes to gcprof? So more like I would invoke (gc)? I guess that's just not how 'gcprof' works, eh? <roelj>I kind of suspect it's the interaction between gnutls and guile. I do 'scm_with_guile' from C, but before I do that, I use 'gnutls_rnd' to seed the RNG in Guile with 'scm_seed_to_random_state'. I read the changes regarding GMP. I don't fully grasp how this works, but could it be that since I initialize gnutls before 'booting' guile in the C program, I'm making things more complex? <roelj>Also a funny observation; If I don't deploy it with Guix, but use Fedora's packages and then compile my program with that, the memory usage stays low. Obviously, almost all dependencies have slightly different versions, so it's hard to compare or pinpoint the problem. <civodul>roelj: re GnuTLS/Nettle/GMP, Guile 3.0.7 has a knob <civodul>GUILE_INSTALL_GMP_MEMORY_FUNCTIONS=0 <roelj>That's a runtime option, no? <civodul>but note that if you were doing it wrong, GnuTLS would behave badly or crash <roelj>I also set "scm_install_gmp_memory_functions = 0;" right before the call to "scm_with_guile (initialize_guile, &config);". Would that have the same effect as setting the environment variable "GUILE_INSTALL_GMP_MEMORY_FUNCTIONS=0"? <lloda>or it should be, afair there was only one patch in the queue which i couldn't evaluate <tohoyn>lloda: is there going to be a new upstream release? <lloda>there haven't been any changes since 1.11.2 <lloda>if i missed any patches lmk and i'll try to have a look <tohoyn>lloda: does guile-cairo work with Guile 3.0? <tohoyn>lloda: it seems to build for Guile 3.0 <tohoyn>lloda: it would be good to build the debian packages for guile-cairo to use Guile 3.0 but that would break guile-gnome2 <tohoyn>lloda: if we published g-golf in Debian we could forget guile-gnome2 <tohoyn>daviid: do you have anything to say to this? <tohoyn>I just looked rdepends for guile-gnome2-gtk in Ubuntu. There is some package "gwave" depending on it. <lloda>i use guile 3.0 with guile-cairo daily <lloda>there isn't any issue that i'm aware of <leoprikler>could you theoretically have both guile2.2-cairo and guile3.0-cairo in debian as we had in Guix? <tohoyn>yes, but AFAIK the debian/rules is difficult to code that way <lloda>i don't use 2.2 but afaik guile-cairo doesn't depend on anything specific to 3.0 so it should work just as well with either <tohoyn>lloda: the debian packages with dependencies to guile-2.2 seem to work fine with guile 3.0 <tohoyn>is it ok for everyone if I just change the dependencies in debian/control to "guile-3.0-dev | guile-2.2-dev" <tohoyn>wingo: could you say something to this? <wingo>tohoyn: wrt debian packaging, rlb is the person to talk to :) <tohoyn>rlb: what's your opinion on the change I proposed to guile-cairo debian packaging? <rlb>civodul: I let debian's 3.0.7 rely on the defaults wrt gmp. I thought that was appropriate now (with respect to allowing other things like gnutls to link against it)? <rlb>tohoyn: I don't know offhand - both because I know very little about guile-cairo, and because I don't know how packages that currently depend on guile arrange themselves right now. So I'd have to investigate. I can do that, and try to help, but likely not until at least the weekend. <rlb>I don't think we have anything like debian-emacs-policy for guile, but we could if we needed it. Though it could be that we might really want a debian-scheme-policy. There's also a debian-scheme lists, but I've (personally) been remiss on that front. I see there's at least one message there (about slib wrt guile) I never responded to. <rlb>I also suspect that getting compilation right might be difficult (not sure what other platforms do), since we might want to have one set of compiled files across the system, rather than on-demand, per-user, but I'd guess that really would require something more like a guile equivalent of emacsen-common and debian-emacs-policy. <rlb>i.e. given macros, we need to recompile the whole dependency tree at package install/upgrade time whenever a parent changes. (That's what the debian emacs tooling/policy does.) <rlb>(And by implication, when guile itself changes, we need to recompile *everything*, since it might have changed macros that affect any given dependent.) <rlb>Unless we promise "ABI comppatibility" with respect to macro expansions -- maybe we do? <rlb>(at least in guile proper) <tohoyn>rlb: I'll try to make the change and check that guile-gnome2 works correctly after it <civodul>rlb: 3.0.7 is fine as long as you --enable-mini-gmp <rlb>Oh, I thought we defaulted to *not* installing boehm allocators in gmp in 3.0.7. Guess I did get it wrong. <rlb>I'll need to upload a 3.0.7-2 "soon". <rlb>civodul: and thanks. <rlb>civodul: and what was the summary (if there is one) of what we'd need to do to just use libgmp? Was it something like switching to lower-level gmp "in place" operations or something? <civodul>rlb: the root problem is that both Nettle (used by GnuTLS) and libguile use GMP; libguile wants to install its allocators, but when it does, Nettle's mpz can be reclaimed too early <rlb>Oh, right, I understand the problem (I think), but I thought I recalled you saying that we could go back to using libgmp if we made adjustments to how (or maybe that was wingo). i.e. by not letting it allocate the structures? <rlb>(And clearly I also mistakenly thought we'd changed the default in 3.0.7, i.e. to *not* install our allocators.) <civodul>rlb: right; if you do that, you have to set scm_install_gmp_memory_functions = 0 by default <civodul>the drawback is that it significantly slows down Scheme payloads that are bignum-heavy <civodul>so it's better to use mini-gmp instead <rlb>civodul: no I meant wrt future code changes, i.e. rewrite numbers.c or whatever to use in-place api calls, or something? <rlb>i.e. I thought I'd heard that being discussed as a possibility. <civodul>ah yes, the better solution would be to use the mpn API i think <rlb>Ahh *that's* what I was asking about :) <civodul>and rewrite numbers.c to use that lower-level API <rlb>OK, and so just to make sure I understood -- the default in 3.0.7 still installs our allocators, so I need to switch debian to mini-gmp, since that's preferable to doing what we were doing in the previous 3.0.5 debs, which was setting "scm_install_gmp_memory_functions = 0". <rlb>(I thought 3.0.7 did that by default -- though if it's runtime configurable, then I see -- we'd still want mini-gmp.) <rlb>i.e. so you can't choose to shoot yourself in the foot :) <stis>WDYT aboyt super vectors? <stis>oh it was private try again ***ec_ is now known as ec