IRC channel logs

2021-05-12.log

back to list of logs

<dsmith-work>Hmm. recent updates anyway. Looks like mostly for chibi
<civodul>dsmith-work: i think the original snow was started ca. 2006
<civodul>it didn't really take off
<flatwhatson>not sure about snow usage, but it says there's a client built into chibi so maybe some folks use that?
<flatwhatson>there's also https://akkuscm.org/ which is more recent and mirrors the snow packages
<flatwhatson>i love guix but it's pretty heavyweight as "just" a language package manager on non-guix-system machines
<tohoyn>sneek: botsnack
<sneek>:)
<wingo>good morning :)
<lampilelo>hi
<manumanumanu>Great morning, everyone!
<taylan>bit of a weird question but does Guile offer a way of accessing bytes at arbitrary memory locations without having to allocate a bytevector object for it first? think bytevector-u8-ref but taking a memory address as an argument instead of a bytevector and index.
<leoprikler>taylan you can do a few things with just pointers, but IIUC bytevectors should not be much of an overhead
<leoprikler>in particular, bytevectors are "backed" by whatever memory you have there, so it's about as much overhead as a pointer
<taylan>leoprikler: there's the overhead of allocating a Scheme object, which later has to be garbage collected. let me shortly explain the problem...
<taylan>with the bytestructures library, you can represent a pointer, including the pointed-to type, then use it to access the pointed-to values...
<taylan>pointers often point to the beginning of an array, whose size may not be known until some time during run-time.
<taylan>to allow fully dynamic access to the elements, what the library currently does is create a bytevector every time you want to access a value, only to have it thrown away again
<taylan>if you want to loop over 1000 elements, that's 1000 bytevectors being allocated and thrown away, which is not OK
<leoprikler>hmm, that does sound like an issue
<taylan>so I need either a way to access bytes without allocating a bytevector, or the user has to tell the library to explicitly instantiate the bytevector at some point with the appropriate size.
<leoprikler>taylan I personally think it's fine to ask the user to tell you the size of the bytevector
<taylan>added an example on how to avoid the allocations, search for "example with guile" on the page: https://github.com/TaylanUB/scheme-bytestructures
<leoprikler>I don't quite understand how the third version does its magic, but otherwise LGTM
<flatwhatson>it's re-interpreting the chars buffer as a bs:vector, which can get bytestructure-ref'd efficiently
<bjoli>taylan: that define-inlinable does nothing in that case, btw. Define inlinable defines the procedure and replaces any direct application with the code expansion, but any reference like (for-each ref million-times) with a pointer to an otherwise hidden definition of ref with a gensymmed name
***bjoli is now known as manumanumanu
<manumanumanu>so: even though it is not immediately obvious, you are still subject to the same run-time lookup if you are referring to a top-level procedure.
<flatwhatson>taylan: would you like a routine like (pointer-ref pointer type [offset]) where type is one of guile's "foreign types"?
<flatwhatson>i guess it would make sense to have a corresponding pointer-set!
<flatwhatson>basically peek and poke
<taylan>flatwhatson: (system foreign) pointers are Scheme objects so that'd also require allocation. what I need would be like (memory-ref address type), or a number of memory-type-ref procedures like bytevector-u8-ref and such
<leoprikler>numbers are also scheme objects, so you gain nothing
<taylan>leoprikler: numbers up to a given size reside on the stack in Guile. actually I'm not sure now whether pointers are really allocated, or use the same kind of optimization, lemme check...
<taylan>does seem like it. (pointer-ref pointer type) would be OK too then I guess.
<manumanumanu>taylan: great work on srfi-64, btw :) I have been linking to your implementation about 600 times.
<taylan>manumanumanu: thank you! great to get positive feedback :)
<manumanumanu>I have been bitten by the reference implementation 2 times myself
<flatwhatson>taylan: curious, what did you check to determine that pointers are optimized?
<taylan>flatwhatson: I found make-pointer in libguile/foreign.c, which uses scm_from_pointer, which creates the pointer as follows: ret = scm_cell (scm_tc7_pointer, (scm_t_bits) ptr);
<taylan>that scm_tc7_pointer thing is a type flag (they're all called ...tc7...) and the value of the pointer is saved directly in the cell as well.
<taylan>contrast for instance to what make_bytevector_from_buffer in libguile/bytevectors.c does: ret = SCM_PACK_POINTER (scm_gc_malloc (SCM_BYTEVECTOR_HEADER_BYTES,
<taylan> SCM_GC_BYTEVECTOR));
<taylan>libguile/scm.h is also a useful place to look at to understand some of the magic. I don't have a very firm understanding of it all but from what I can tell, make-pointer doesn't allocate, whereas pointer->bytevector indirectly does
<flatwhatson>makes sense, thanks!
<taylan>flatwhatson: correction on what I said before: comes out scm_cell actually allocates a pair. the TC7 type codes identify special kinds of objects that fit into "fake" pairs somehow.
<taylan>it's the TC3 codes that identify objects that reside directly on the stack
<mwette>taylan: (pointer->bytevector (make-pointer #x12345678) 128)
<mwette>where #x12345678 is the address and 128 is the size; from (system foreign)
<mwette>but doen't help if you don't know the size
<taylan>yeah, that's the problem. dereferencing a bs:pointer already does exactly that, but with the size only as big as the referenced type, so e.g. size = 4 for (bs:pointer uint64)
<taylan>in other words, bs:pointer was originally meant really just for pointers to a single value, not as pointers to the beginning of an array.
<taylan>I now added the ability to dereference it with an integer like you suggested, but it has to do the (pointer->bytevector (make-pointer x) y) for every such reference
*taylan goes AFK for now
<flatwhatson>rough draft pointer-ref/set: https://paste.gnome.org/p1ow9dbyn
*wingo not really a fan, fwiw
<flatwhatson>the idea in general or this approach?
<wingo>a few things. in general, pointers to raw scalars aren't so common -- you have pointers to structs, etc. where it's important, we shouldn't be allocating heap pointers at all. where it's not, the bytevector api is more expressive and ties together better with the rest of the system
<wingo>we do need a more efficient ffi but i think the path there is through the compiler and not through run-time routines
<flatwhatson>is it feasible for the compiler to eliminate intermediate bytevectors?
<flatwhatson>the chez FFI has (foreign-ref type address offset) and (foreign-set! type address offset value)
<wingo>yes i think it's feasible for the compiler to eliminate those. the chez low-level ffi looks nice tho, i assume the "address" is just a fixnum
<wingo>or usually a fixnum anyway; anyway the compiler could lower such a foreign-ref / foreign-set very nicely
<wingo>quite the cowboy primitive of course
<civodul>i think pointer objects should be immutable
<civodul>and i think we should have "immediate pointers" when possible
<amirouche>flatwhatson: IIUC what you want is http://www.r6rs.org/final/html/r6rs-lib/r6rs-lib-Z-H-1.html#node_toc_node_sec_2.4
<amirouche>except instead of bytevector, you have a pointer?
<amirouche>Chez can not expose a bytevector from a C pointer, see also https://github.com/cisco/ChezScheme/issues/486#issuecomment-808829706
<wingo>civodul: i think when you have access to make-pointer, then you might as well have the equivalent of foreign-ref / foreign-set
<amirouche>(pointer->bytevector pointer length) that works without copy will be very handy
<wingo>amirouche: the compiler could eliminate that fwiw
<amirouche>eliminate what part ? the copy ?
<wingo>it doesn't right now, but it could do so relatively easily
<wingo>making the bytevector
<wingo>iirc pointer->bytevector aliases memory without copying, right?
*wingo looks
<wingo>yes, there is no copy there
<amirouche>that is good then :)
<wingo>just an allocation of a bytevector
<dsmith-work>Wednesday Greetings, Guilers
<davexunit>allocation of bytevectors in pointer->bytevector can be problematic sometimes.
<davexunit>like in ffi-using code in hot loops that needs to keep allocation low
<davexunit>bytevector->pointer also
<davexunit>depending on the situation I usually cache these when performance is a concern
<taylan>wingo: FWIW how this topic first came up was: how to support the "flexible array member" special feature of C structs in the bytestructure library.
<taylan>since the bytevector API doesn't allow accessing beyond the declared size of a bytevector, one effectively has to create a second bytevector after getting the length information (that would typically be in the struct somewhere) via the first bytevector.
<taylan>wingo: do you think it would be useful to have a bytevector sub-type of sorts that doesn't have size info / doesn't do bounds-checking?
<civodul>wingo: actually i'm sure what foreign-ref is; i thought it was Chez's equivalent of pointer->address, but maybe not?
<leoprikler>pointer->address just gives a number idk how that would be usable to e.g. reference some int inside a struct
<pkill9>has anyone created a GUI application with guile?
<taylan>leoprikler: re. your bug report on elisp keywords: while using Scheme you can call (read-set! keywords 'prefix) to allow :foo to be read as #:foo. have you tried activating that in elisp mode somehow?
<leoprikler>taylan whatever I do in Scheme has no effect on the Elisp reader, that seems to be a specific bug within it
<taylan>leoprikler: I think it's normal, since every language defines its own reader. I'm not sure if elisp reuses the regular Scheme 'read' somehow so one just has to set a parameter when starting up the language, or if actual coding is needed... I think it's the latter.
<taylan>yeah it seems one has to fix it in module/language/elisp/lexer.scm
<taylan>hmm, or one could just let them parse as symbols like currently, then special-handle them later. compile-symbol in module/language/elisp/compile-tree-il.scm looks promising
<leoprikler>IIRC Elisp knows the difference between symbols or keywords, so keywords as self-quoting symbols won't fly
<taylan>leoprikler: hmm. I was planning to compile them into keywords, but I guess that's not enough, as (read ":foo") will then still return a symbol.
<taylan>what I find puzzling is, there was (is?) a whole Guile-Emacs that actually starts up, and how can that be if the Elisp parser doesn't even know keywords. are they not used anywhere in Emacs's standard libs? or maybe bpt's branch had a fix for this.
<taylan>does anyone happen to know what happened to bpt aka Robin Templeton? their work was very promising, it's sad that they didn't continue working on it.
<leoprikler>they use a known-good commit, that perhaps has a different parser
<leoprikler>I'm using Guile 3.0.2+
<RhodiumToad>as far as I recall, elisp has no "keyword" type at all
<leoprikler>(keywordp :really?)
<RhodiumToad>that is, :foo is a symbol whose initial value is itself (rather than being unbound)
<leoprikler>(setting-constant :really)
<leoprikler> (setq :really nil)
<RhodiumToad>(symbolp :really?)
<leoprikler>yeah, elisp symbolp is (or (symbol? this) (keyword? this)) when translated to guile
<RhodiumToad>elisp's keywordp is like (and (symbolp x) (symbol-name-of-x-starts-with-:))
<RhodiumToad>so elisp keywords are just a subset of symbols, not a separate type
<leoprikler>still, I'd argue that for Guile interop it might be more useful to have them as keywords
<leoprikler>(define (my-function #:keyword a b c) ...)
<leoprikler>,L elisp
<leoprikler>(my-function :a 5)
<leoprikler>please don't mind the typos in the scheme part
<civodul>wingo: i'm preparing the 3.0.7 upgrade in Guix and it's a tiny bit slimmer: 129 MiB vs. 132 MiB for 3.0.5
<wingo>nice
<wingo>probably was the assembler optimizations
<civodul>er sorry, that's for its whole closure
<wingo>sure
<civodul>guile itself is unchanged actually
<civodul>(libltdl and gmp were removed from the closure)
<wingo>ah
<wingo>ah! neat
<wingo>well, we got away with rewriting "read" in scheme and still made the thing smaller. good
<civodul>yes, that's nice
<pkill9>anyone know of any GUI applications written in guile?\
<RhodiumToad>the state of play with GUI bindings for guile is not entirely satisfactory
<RhodiumToad>I wrote some stuff for myself using guile-gnome, it could be made to work but got crashy as soon as you tried anything fancy with events
<RhodiumToad>guile-gi and g-golf are the more recent attempts at providing access to gtk
***rekado_ is now known as rekado
<rekado> https://gitlab.com/kavalogic-inc/inspekt3d.git has disappeared
<rekado>I liked it as a libfive viewer that can be used from geiser.
<pkill9>guess i might just have to use python
<RhodiumToad>tcl/tk is still orders of magnitude easier than any other scripting language for doing simple gui stuff
<zzappie>rekado: :( libfive is cool. now looking back I think playing with libfive was my first exposure to guile.
<taylan>oh man, I'm building the wip-elisp branch and Guile used to take AGES to build :D
<civodul>rekado: it's 503, perhaps that's a transient issue?
<civodul>it's apparently on Software Heritage too
<zzappie>civodul: ha, what do you type into the search field? I checked swh and there wrere no results
<pkill9>RhodiumToad: what programming language do you use for it? python?
<civodul>zzappie: i run "guix lint -c archival inspekt3d", which does it fo me :-)
<RhodiumToad>pkill9: for what?
<RhodiumToad>pkill9: tcl is a language
<pkill9>ah
<pkill9>I meant for tk
<dsmith-work>Deep in the dim dark past, Guile actually had Tk in it.
<rekado>taylan: I rebased wip-elisp: https://git.elephly.net/?p=software/guile.git;a=shortlog;h=refs/heads/wip-elisp
<zzappie>civodul: wow! didn't know its an optioin :)
<taylan>rekado: OMG neat!
<rekado>taylan: with that I was able to build guile-emacs but it crashed ¯\_(ツ)_/¯
<pkill9>dsmith-work: why was it removed?
*zzappie -> zzz
<dsmith-work>pkill9: No idea. Maybe because it also needed Tcl?
<taylan>rekado: well it wasn't all that stable to begin with... I'd love to work on it but my knowledge of low-level stuff sucks and my 40 hr/week job is so taxing...
<RhodiumToad>tk is basically unusable except with tcl. there are tk bindings for other languages, but as far as I know all the still-maintained ones work by embedding tcl into the other language too.
<RhodiumToad>(there used to be a couple that worked by ripping the tcl parts out and filling them in from the host language, but I believe they all died due to the heavy maintenance effort in keeping up with Tk changes)
<dsmith-work>Looks like the Tk stuff was removed before the first CVS commit.
<dsmith-work>RhodiumToad: !!