IRC channel logs

<meaty>Hi guile, how can I get guile/geiser to locate the library I'm working on? when I try to import I get "no code in module", when I try to evaluate the .sld (r7rs library definition) the import directive doesn't work, I get "attempt to include relative file name but could not determine base dir"

<rlb>meaty: I'm not using geiser, but roughly speaking, the module has to be in the GUILE_LOAD_PATH (e.g. %load-path -- see the info pages for additional info, *and* guile has to be set to recognize the extension. Not sure if .sld is by default, but it may be.

<rlb>See also -x (perhaps).

<meaty>hmm... my project's directory is getting correctly added to %load-path

<rlb>Looks like .sld isn't in %load-extensions (see the info pages) by default.

<rlb>That may be your issue.

<rlb>If so, -x .sld might help.

<apteryx>I think you may need --r7rs to have it honored

<rlb>Oh, right, imagine that might imply -x .sld.

<apteryx>but it'd be nice to have .sld understood as r7rs by default

<meaty>ok, that is one thing... another is that guile seems to only be aware of 'define-library' and its syntax on the first eval

<meaty>I kludged the "could not determine base directory" issue for now by just using the full path in the library def

<apteryx>meaty: it's a macro

<apteryx>it lowers down to rnrs 6 forms

<meaty>However, if I try to re-eval the library def, suddenly guile has forgotten that 'define' can exist within 'define-library'

<meaty>'definition in expression context'

<apteryx>meaty: you may need of of the r7rs define-library fixes from that series: https://lists.gnu.org/r/guile-devel/2023-12/msg00050.html

<apteryx>in this series context, I think instead of https://lists.gnu.org/r/guile-devel/2023-12/msg00053.html I'd try adding --r7rs to the arguments of guild compile.

<meaty>ok... so `guile -L . -x .sld` or `guile -L . --r7rs` allows everything to work perfectly

<meaty>however, eval'ing the library definition with geiser still gives the 'could not determin base dir' error...

<apteryx>is there some 'include' in your define-library form?

<apteryx>Guile messes up the underlying file name info in the port through its canonicalization process, which breaks them.

<apteryx>I nearly fixed it with https://lists.gnu.org/r/guile-devel/2023-12/msg00047.html, but it doesn't work in the case where you've never byte compiled the project, IIRC, plus it's a gross hack.

<apteryx>can I access any Guile internals with the FFI? e.g. the 'thread_count' variable.

<apteryx>nope

<apteryx>I'm trying to profile my mmap + elf parser experiment; how can I do it?

<apteryx>,profile (call-with-mmap-bytevector "/gnu/store/x00bkpkikw60m9mknl3y1shsj482pxzc-llvm-21.1.1/lib/libLLVM.so.21.1" (compose elf-dynamic-info-soname elf-dynamic-info parse-elf)) -> No samples recorded.

<apteryx>I'm interested in the peak amount of memory used.

<rlb>apteryx: one place to start might be /usr/bin/time -v ...

<rlb>...also depending on what you're trying to test "echo 2 > /proc/sys/vm/drop_caches will dump the kernel caches.

<rlb>(more relevant if you're trying to test elapsed times with a cold cache, etc.)

<apteryx>rlb: I cannot use the Guile profiler to do this?

<apteryx>otherwise thanks for the time command suggestion

<apteryx>I already have it aliased as time+, but tend to forget about it ;-)

<rlb>Oh, when I hear mmap, I tend to immediately think of wanting to track rss, etc., but that's just due to things I've been hacking on -- not sure what guile currently has wrt memory stats.

<rlb>I see gc-stats, and gc-live-object-stats in the info pages, but also notice that the latter appears to be gone.

<apteryx>yeah, so far I was doing ,gc, then ,stat, then experiment, then ,stat

<apteryx>it'd be nicer if I could do ,profile exp and it'd tell me everything about it (cpu time, memory usage, etc.)

<apteryx>pum

<apteryx>old: would you know why, at https://codeberg.org/lapislazuli/blue/src/commit/10782d4e0fdc60a3e9078b7cc1193b11175cdee6/blue/fibers/events.scm#L166, pump-epoll-guardian calls itself recursively?

<apteryx>old: rough blueprint (untested): https://paste.guixotic.coop/io.scm_guix_.html

<apteryx>trying out the guardian approach (with a weak key hash table to keep the ptr + size associated data

<apteryx>is it normal that doing (length bv) appears to read all of bv?

<apteryx>I would have expected it was some property, cheap to return

<sneek>wb ekaitz

<ekaitz>hi

<ekaitz>apteryx: re guile-lib under guile, i like the idea

<lloda>apteryx: that's what happens when you define a list as a chain of pairs

<lloda>there's no where to put that property

<kestrelwx>rgherdt: Hi! Would you rather I submit patches to your repository or just

<kestrelwx>Didn't mean to send; or just publish a fork?

<kestrelwx>For the WebIDL parser.

<apteryx>ekaitz: how do we do it?

<apteryx>probably need the empty repo with write access for myself

<apteryx>lloda: I see :-/

<apteryx>(re (length bv))

<apteryx>so even if by bv is mmap'd, if the code using it applies it to length somewhere, the gain vanishes.

<apteryx>is it fine to use a bytevector object as the key in a hash table?

<apteryx>seems so, as long as I use 'hashq-ref' to rea it

<apteryx>read*

<apteryx>hm. Are my assumptions that the bv keyed entry in my weak hash table should stay alive until bv itself is GC'd, which should not happen until it's pop'd from the guardian wrong? https://paste.guixotic.coop/io.scm_guix_.html

<apteryx>see my repl session at the bottom

<ekaitz>apteryx: i sent you an email with some info about it. You just need to transfer the repo, civodul could accept the transfer and then add you as the contributor or something

<apteryx>great, thanks!

<apteryx>looks like as soon as I (set! bv #f), the entry gets cleared from (hashq-ref %mmap-bytevector-map bv)

<dariqq>Hi, if I want to report a new issue should I do it on codeberg or debbugs?

<apteryx>I think the bug tracker is not yet enabled on codeberg?

<apteryx>oh, it is! I'd go with codeberg.

<ekaitz>civodul: i don't know how to build the guile website locally

<apteryx>hm, I'm silly, if I set bv to #f of course the (hashq-ref %my-map #f) returns #f ^^'

<ekaitz>i just changed the config, but i don't know... i'm trying to `haunt serve` but it doesn't do anythingg

<ekaitz>and of course it doesn't have any docs on how to hack on it

<dthompson>haunt build

<ekaitz>dthompson: it doesn't work :S

<rgherdt>kestrelwx: many thanks for the PR. I'll give it a look later after the work, and perhaps add a test regarding this problem.

<ekaitz>dthompson: it gets stuck in "Producing Web pages for local tests *only*!"

<rgherdt>kestrelwx: PRs are always welcome :)

<ekaitz>dthompson: yesterday i worked in the bootstrappable.org website and it simply worked

<apteryx>re guardian, bv -> mmap ptr... I'm adding 'bv', a bytevector to the guardian. When the guardian cleanup routine runs, bv is already gc'd, so I can't use it to access the associated ptr / size in a hash table? What am I missing?

<kestrelwx>rgherdt: I also had a question if `enum?` is meant to be exported, I was meaning to clean up the exports anyway.

<dthompson>ekaitz: I'm not familiar with guile's particular config so idk what that message means. it's not a haunt message.

<ekaitz>dthompson: the config is doing something weird: https://codeberg.org/guile/website/src/branch/master/haunt.scm

<ekaitz>yeah it's written in the config

<ekaitz>i'm not familiar enough with haunt to know what should this config actually do

<rgherdt>kestrelwx: definitely, I forgot to export it

<dthompson>the config itself looks fairly straightforward

<ekaitz>haunt build command never finishes

<kestrelwx>Alright, as I'm working with it I'll send patches to you then.

<dthompson>does it print anything else? haunt will log all the files it produces

<ekaitz>dthompson: it just hangs with nothing in the output

<ekaitz>it happens with both the build and the serve command, but bootstrappable.org simply works!

<dthompson>serve runs a web server so hanging around and not exiting is expected

<dthompson>haunt serve will log all http requests as they come in

<ekaitz>yeah yeah but it doesn't serve anything, and build also hangs

<ekaitz>so it's not a matter of the command itself, something hangs before

<ekaitz>OOOH I understand now!

<ekaitz>i need to guile -L . haunt.scm

<ekaitz>my godness

<kestrelwx>Oh, I'm getting leisurely 15kBps from Codeberg.

<kestrelwx>Well, it's 50 now I spoke too soon.

<ekaitz>but i don't know where it's serving it wtf

<ekaitz>this has to be a very bad joke

<apteryx>info haunt

<apteryx>"By default, the web server listens on port 8080, so the URL to visit would be <http://localhost:8080>." :-)

<ekaitz>apteryx: i'd say the problem is how this thing is built, not haunt itself...

<ekaitz>no no, it's not that, it's this website does some crazy stuff

<kestrelwx>Yeah, I'm also struggling to figure it out.

<dthompson>wish I had the time to debug it but alas

<dthompson>I can try to offer advice as I am able

<ekaitz>there's an http server here and stuff... haunt is only being used tangentially... I have to read it better

<ekaitz>thanks for the help anyway...

<kestrelwx>ekaitz: `haunt build` hangs on "resolve-interface '((website resources))".

<kestrelwx>I put `pk` before and inside the let and only get the one before.

<kestrelwx>Well it's just one pair of brackets.

<kestrelwx>Maybe it just computes for a real long time?

<ekaitz>no, i think this has a website behind that does some work

<apteryx>refining my question about guardians: it seems adding the key object of a weak-key hash table should cause that entry in the hash table to be available at the time the guardian return the key?

<ekaitz>someone is having some fun with the gc hehe

<ekaitz>apteryx: i'd say so, yes, but I'm not sure

<dthompson>no I think it will already be gone

<dthompson>iirc...

<apteryx>it's weird, the guardian is designed to guard the object from being gc'd, and I can retreive it from the guardian,, yet the weak-key hash table has already dropped it. Seems like a bug to me?

<euouae>Hello

<apteryx>o/

<apteryx>chatgpt says "the guardian’s reference does not count as a strong reference.", and this is why my weak-key hash table would drop it.

<apteryx>then how to tack some extra information to a guarded object, that can be retrieved at finalization time?

<apteryx>e.g. a pointer size when finalizing a ptr (for munmap, say?)

<euouae>Still suffering the woes of mmap I see

<apteryx>ACTION tries a regular hash table again

<dthompson>apteryx: I recommend not asking chatgpt for help

<apteryx>it's often garbage but it's like a rubber duck on steroid

<euouae>ChatGPT is quite nice compared to the snobbish behavior you can find online

<euouae>Try asking for help for the postfix mail server versus using ChatGPT e.g.

<dthompson>we seriously gonna sing the praises of chatgpt in here? wow

<euouae>I don't use it for the code I write due to copyright and license issues, and I generally don't use it for programming

<dthompson>read docs, read source code, talk to humans!

<euouae>But it does supplement nicely the reading of documentation

<dthompson>at great cost to our world.

<euouae>In terms of energy or something else?

<dthompson>energy is but one aspect

<euouae>Yeah I mean it is definitely bad, I'm only commenting on the aspects I've found useful

<identity>it has all of its aspects either way

<dthompson>for the purposes of this channel, if chatgpt tells you something about guile you should assume that is incorrect

<identity>if the lies machine tells you lies then it is lies unless proven otherwise

<identity>replace «lies machine» with «a random person on the street» and it stays true

<civodul>uh, i did not expect to see LLMs used in these circles

<euouae>I mean that's fair, I'll give you an example of where it was useful earlier, I couldn't figure out in ELF if e_shnum == 0 is ambiguous but I had overlooked in elf(5) the additional info on e_shoff == 0 disambiguating, ChatGPT pointed it out

<euouae>I checked back on elf(5) and it was right. Would it be nicer if I could chat with people who know about the ELF format? Perhaps, but intellectually I live in a desert ;-)

<ekaitz>civodul: i sent a pr to the guile website, but didn't manage to see the website locally

<euouae>apteryx: I was looking at <https://codeberg.org/guix/guix/issues/1262> and <https://issues.guix.gnu.org/59365>. The second link requires also fixing gremlin.scm. Does the first also involve gremlin.scm?

<apteryx>yes, through file-needed/recursive

<euouae>Okay so all the bugs involve gremlin.scm?

<apteryx>I think so, which relies on the elf parser

<euouae>Yeah I'm just trying to make sense of it

<euouae>I am altering the ELF parser API so I'd have to alter gremlin.scm too

<civodul>ekaitz: yup, i’ll take a look later, thanks!

<ekaitz>civodul: thank you for all the support, I know you are a busy man

<civodul>everyone’s busy :-)

<old>apteryx: re pump-epoll-guardian: The idea is that on GC hooks, we drain the guardian

<apteryx>does it sleep when it has nothing to return?

<old>when calling the guardian (without argument), it will return #f if there are no objects left that's it's guarding

<apteryx>ah yes, we loop it only while it's returning objects ready for finalization

<old>no it's a no sleep operation so really we really on the GC to wake ourself by calling the hook and we call the guardian in a loop to drain all epoll fds that were GC

<apteryx>if it's empty, nothing happens until the hook calls it again

<apteryx>(gc hook))

<old>s/we really/we rely

<old>exactly

<apteryx>old: can you see what I'm doing wrong here? I have one level of indirection, trying to access some extra data keyed by the guarded object, and this fails in the pump/finalizer procedure: https://paste.guixotic.coop/io.scm_guix_.html

<apteryx>(see my experiement at the bottom )

<kestrelwx>ekaitz: It is just really slow.

<old>apteryx: well first you would need to loop in your hook but I guess that's not the problem

<old>hmf

<old>hold on I will the whole thing

<apteryx>I loop inside the match

<old>ahh

<old>yes

<ekaitz>kestrelwx: what? really?

<kestrelwx>Well I'm not sure where the pages are, though.

<ekaitz>that was my issue too

<ekaitz>i managed to trigger some builds

<ekaitz>but i don't think this is serving the website... :S

<kestrelwx>Resource not found: /website/index.html

<kestrelwx>I also get that for / and other stuff.

<ekaitz>i'm unable to connect to localhost:8080

<kestrelwx>I've run `haunt serve`.

<apteryx>old: ah, also s/unless/when/ in my hook

<ekaitz>kestrelwx: and does it serve you anything?

<kestrelwx>I can connect to 8080 but I can't find anything yet.

<ekaitz>If I run `ss` i see my haunt connected to some IP

<ekaitz>it's connected to some GNU website lol

<old>apteryx: I think the problem is the weak table

<ekaitz>kestrelwx: anyway, i'll leave it as is

<ekaitz>ACTION has to go

<apteryx>old: I changed it to make-hash-table in my last attempt

<apteryx>still same issue

<old>you can do without any table

<apteryx>actually, different issue... I think the finalizer then doesn't trigger

<old>just use:

<apteryx>because it's live in the hash table

<kestrelwx>Bye!

<old>(bytevector->pointer bv) and (bytevector-length bv)

<apteryx>won't (bytevector-length bv) read the whole bv in memory?

<old>and yes if you put the bv in the hashtable, you are leaking the bv for ever

<old>no it access a single field IIRC

<apteryx>earlier I was using (length bv) and it was causing that

<apteryx>old: thanks for the ideas, I'm trying it now

<old>there's an easy way to test

<old>create a mapping with PROT_NONE

<old>see if any operation touch the mapping you will get a segfault

<apteryx>old: brilliant, that seems to work, and it doesn't read the whole bytevector: https://paste.guixotic.coop/io.scm_guix_.html

<apteryx>ACTION checks if they can push the idea lower level, eliminating munmap altogether

<apteryx>in (guix build syscalls)

<old>great :-D

<old>why would you eliminate munmap?

<old>you have to give the memory back to the kernel otherwise you are possibly taking physical memory on the system if the pages were touched

<apteryx>old: I mean, abstract it away from the Guile API by using guardians to finalize things

<apteryx>ACTION ponders whether mmap can be useful in the case there is decompression returning a port, e.g. 'call-with-zstd-input-port'

<apteryx>now I have the same quetion mwette had when impleminting their C version; how to finalize with munmap when it not only wants the pointer, but its size?

<old>what do you mean?

<apteryx>old: https://paste.guixotic.coop/magit_guix-383-1339.html

<apteryx>I want to add an implicit finalization via the same guardian idiom to the underlying mmap wrapper

<apteryx>so that if someone forgets to call (munmap ptr size) it gets done automatically at some point.

<old>apteryx: you should not expose the pointer returned but only a bytevector IMO

<old>pointer without a length is a mess in C that should have get fixed 40 years ago

<apteryx>that's what I do for the file->bytevector we were looking at earlier, but this uses the lower level mmap, and I'd like this mmap to have a finalizer too

<apteryx>but maybe that's a bad idea since the ptr returned by mmap would not be carried around when using file->bytevector (or does the bytevector keep a reference to it, which is what it returns on bytevector->pointer ?)

<apteryx>if the later, than if we could wire the guardian at that lowest point (mmap), it'd make the whole API including mmap more convenient/safer.

<apteryx>the only thing to do so is the ability to carry two pieces of information instead of one when using the guardian (ptr *and* its size), to be able to call munmap with that.

<old>ya that's what I meant, do not expose the mmap syscall

<old>make wrappers around it that return bytevector and use your finalizer

<old>the bytevector is the owner of the memory not the pointer object that way

<old>I'm not jusre if pointer->bytevector will keep a reference to the pointe

<apteryx>I'll experiment some, thanks

<old>even if it is the case

<old>how will you get the length from the pointer in the GC hook

<old>that's the exact problem you had with weak hash table

<old>the pointer object do not encode any length, but you need a length

<apteryx>that the question. maybe a list of objects can be added to the guardian, with the first object used for the reference test?

<apteryx>(my-guardian (list ptr size)) ?

<apteryx>I'm reading the old '93 guardians paper

<old>but then you have a bytevector just there

<old>bytevector are literraly a cons cell I think with the pointer address and it's size

<old>+ a bit for type

<apteryx>oh, so bytevector is the perfect representation for a pointer + size info

<old>yes

<apteryx>that's a good point

<apteryx>if the lower level (mmap) returns a bytevector directly, then it solves the issue of having distinct ptr and size objects

<old>well there's is a header to bytevector

<old>when creating a fresh one

<old>I'll check pointer->bv

<old>see scm_c_take_typed_bytevector

<old>so a bytevector is a little more than a cons cell

<apteryx>civodul seems to think that mmap in scheme is a bad idea though, as "it will not be able to prevent user code from segfaulting." I don't yet see why.

<old>you have some flags, a parent attribute (the pointer or another bv if a slice I think)

<old>then the contents and the length

<old>I don't see any difference with other allocated memory if the protection bits for the allocation are correctly set

<old>and a bytevector is used

<old>all of this can be wrap in a single module and the user just receive bytevector which are safe

<euouae>scheme doesn't understand bytevector access becomes IO I think that's the trap

<old>I'm not following

<apteryx>I've asked civodul for clarification here: https://codeberg.org/guix/guix/pulls/3737#issuecomment-7889177

<old>apteryx: I've repleid a suggestion for manual memory management

<apteryx>thanks!

<apteryx>I was under the impression that guardians were transparent: if the user manually clear an object, the guardian will loose its reference to it, no?

<rlb>apteryx: sounds like you've found it, but bytevectors.[hc] has the relevant size/layout info -- and they're currently 4 "words" with the second being the length, the third being a pointer to the raw signed char* byte array, and the 4th being a pointer to a "parent" if any.

<rlb>apteryx: and I believe that bytevectors have no hashing yet.

<rlb>but might want to double-check if it's critical (see hash.c)

<rlb>(I was poking at that a week or two ago.)

<rlb>i.e. I think they might all have the same hash...

<rlb>ACTION tests it

<old>apteryx: no it's fine. The manual memory management is an advise to the OS for reclaiming physical memory

<old>and you let the finalizer reclaim the virtual memory

<old>wich will reclaim the physical memory also if needed

<apteryx>old: I've made the change to return bytevectors from mmap directly, and accept them from munmap, and also used the guardian directly at mmap, so I don't need to use it on file->bytevector anymore.

<apteryx>You can see it in https://codeberg.org/guix/guix/pulls/3737/files. Thanks for all the help so far!

<rlb>apteryx: also not sure if this is relevant to what you were seeing, but of course mmapped regions may do prefetching, so when you access the length, the kernel might decide to read a lot more. (cf. (posix_)madvise -- though whether or not any given kernel respects those calls (and how) "depends", I think).

<apteryx>rlb: I was using 'length', I should have been using bytevector-length it seems :-)

<apteryx>ACTION -> zzz

<rlb>Hmm, I'd have thought those would be roughly the same.

<rlb>i.e. offhand would have thought that length for a bytevector would end up calling bytevector-length after checking the type...

<rlb>(there's nothing to iterate to get the length for a bytevector)

<rlb>ACTION guesses you might have been seeing the effect of prefetching, which went away once the cache was warm.

<ane>mra: did you try meson and guile?

<mra>ane: sorry, been a bit busy the last few days. i'm gonna try to get some more work done on it over the weekend

IRC channel logs

2025-10-24.log