<muffindrake>Looks like I'll have to look into that, then. And yes, guile is obviously already installed <apteryx>I guess map is what I want rather than xpath here; xpath seems to be designed to seldom select content, not modify it ***apteryx_1 is now known as apteryx
<leoprikler>apteryx: I think when it comes to active rewriting, you'd probably prefer sxml-match/tree-fold ***rekado_ is now known as rekado
<rekado>we even have an example in (gnu packages java) <pkill9>does anyone know why guile is saying string-match is an unbound variable? <pkill9>if i run guile repl, i can run it without importing anything <pkill9>ah it's in ice-9 regex according to help command <pkill9>i must've put an import somewhere or something <leoprikler>IIRC the REPL comes with more bells and whistles than you'd typically have inside code. <leoprikler>(which is good – you want to be explicit when you're writing such stuff to file) <apteryx>rekado: oh, that pre-post-order looks interesting, thanks! <apteryx>I'm now trying to match nodes which are both paragraphs ('p') and *text* at the same time; or is this always the case by definition? <rekado>nodes cannot be *text* and p at the same time. <rekado>*text* is the text content of a node; it’s not a tag. <rekado>the *text* of <div>this is text</div> is the string “this is text”. <rekado>you can, however drill deeper and get at the *text* inside a “p” and then jump out and return the matching paragraph. <rekado>you can’t use a short-hand like (p 1) then, though <rekado>you could use a filter procedure to check on the contents of the node <wingo>spk121: what in the world could #b#b0 mean ? :) <wingo>i am thinking it is a bug in the old reader, yes? <wingo>spk121: regardinng #\11# that's quite curious <wingo>weird that # is a synonym for 0 somehow??? <wingo>seems to be in r5rs syntax but not r6rs <weinholt>i believe "#" in place of a digit was used in inexact numbers to show that there's no significant digit there <wingo>yeah and could only be in trailing digits <wingo>i suppose the r6rs way is "we're using ieee floats anyway" <sh4rm4^bnc>wingo, i'm interested in lightening, especially how it differs from lightning, but documentation is sparse. especially it's been hard to figure out what actually changed between lightning 1.x and 2.x <wingo>sh4rm4^bnc: briefly, afaiu -- lightning 1 was a very direct codegen library. you write jit_movi (JIT_R0, 42), it emits mov rax, $42 <wingo>lightning 2 didn't do that. instead you write jit_movi, that adds a node to a graph <wingo>then when you finish, lightning runs optimizations over that graph, including a kind of register allocation, and then emits code <wingo>lightening (what a confusing name) goes back to a more lightning 1 kind of approach. <wingo>not sure tbh. i don't think it's very extensive. but for guile it conflicted with how we wanted to use a codegen library <sh4rm4^bnc>ok, and are there any other differences ? like same api? (also, why does e.g. the mips backend need work to become compatible) <wingo>similar but not same api. see lightening.h for the api <wingo>need to update the manual, but not a high priority atm <sh4rm4^bnc>my main interest is in using light(e)ning as a universal backend for a lightweight C compiler-i.e. write backend once, let lightning do the rest <wingo>takes work to add more backends. probably 40-60h or so. lightning 2 has lots of backends, if it suits your purposes and that's your most important factor, use it over lightenig <wingo>otherwise a port of a backend from lightning to lightening takes time <sh4rm4^bnc>i guess one could also just add an option to turn optimization off to lightning <wingo>spk121: civodul: anybody else: how to describe the motivation for mini-gmp? <wingo>"more convenient in some circumstances" ? <wingo>allows Guile GMP uses to use a different allocator than other GMP users? <rlb>I mean it's really "because we have to"? <rlb>(or perhaps I misunderstand) <rlb>i.e. we're designed to be embedded, so we can't mess with gmp's allocation globally <wingo>if it's a must, then shouldn't we default to having it on ? <wingo>i still want to rewrite guile's bignums to use the mpn API rather than mpz <rlb>I think it might always need to be on? Unless you're never going to link something else aginst guile. <wingo>which would allow us to use external GMP again <rlb>(something that might load anything else that uses gmp I mean) <sneek>manumanumanu, you have 1 message! <sneek>manumanumanu, leoprikler says: yeah, there are a lot of instances, when Guile's supposedly immutable things are actually mutable. I sadly can't recall a way of debugging this. <wingo>i fixed many of these instances btw, could be that in 3.0 i messed something up tho <rlb>And one of the more significant concerns afaict is whether or not this causes ABI breakage that means we'll need to bump the soname, and affect what kind of release we can put it in. <rlb>i.e. is it ok for a Z release, etc. <wingo>a mini-gmp option is always possible if off by default <wingo>but you are wondering for debian i guess <wingo>whether it's possible to ship a 3.0.6 with mini-gmp <rlb>it's not a fix if you can't use it, and debian, for example, can never change that option if it breaks the abi (during a stable series) <wingo>and does that break ABI relative to 3.0.5 <rlb>but I guess we might just have to leave things broken for the lifetime of the release... <manumanumanu>wingo: was that a reply to me and leoprikler's conversation? If so, I was replying to a question on reddit on (set-car! '(0) (list 1 2 3)) actually producing a (weird) result and not erroring <rlb>not sure, first I'd just like to know what the constraints are, i.e. does it actually break the abi, etc. <manumanumanu>I went in and said "oh no, that is a literal list. it is an error". It turns out guile did not raise an error <wingo>so what is the argument that it breaks ABI? would a user be able to link to Guile and then use GMP without explicitly linking to GMP, is that the issue? <wingo>manumanumanu: i think 2.2 raised an error in that case, and that i messed up during the switch to 3.0 <manumanumanu>and maybe try some older builds and see if it works there <rlb>I think it may depend on what kind of "breakage" it is, and what fixing it entails. i.e. a bunch of stuff in debian (as an example) has been built against the current arrangement, and if a new 3.0 were to come out that say changed the size of a function argument, then we couldn't make that change because as soon as the new version was uploaded, everything else would of course start crashing until it was recompiled and distributed. <leoprikler>w.r.t. light(e?)ning, would there be a way to make the switch from the guile fork to lightning 2.x (perhaps disabling some optimizations for better throughput or whatever)? <wingo>rlb: we certainly do not change the size of a function argument <rlb>Oh, sure. I'm just describing a clear case where it'd be a big problem, and the question is, what does this case look like exactly... <spk121>wingo: #b#b0 and others in that family are a recursion bug in the old reader. The 2nd #b0 is processed to mean zero, and then simplifies to just #b0, which simplifies to 0 *rlb is remembering the difficult time when we broke the threading abi more like that back in the 1.6 era? Was awkward... <leoprikler>I think it's probably going to be a mismatch in symbol tables. <rlb>(accidentally did iirc) <wingo>leoprikler: i don't see the way to do that, fwiw. i started with lightning 2 but ran into a lot of bugs <rlb>(I also wonder if we might have any additional flexibility wrt versioned/named symbols if that's common enough...) <wingo>i did not have the feeling that lightning 2 could be a good fit for guile's needs <wingo>ok so now i see that with the mini-gmp work, we expose mini-gmp.h to users <wingo>will see if i can change to include it only when BUILDING_LIBGUILE or so <rlb>In any case, I'm just speculating -- would need to know exactly what the differences are, i.e. do we expose gmp data structures, and if so, are mini-gmp's an exact match, and can we guarantee that throughout 3.0, etc. <spk121>wingo: doh! didn't think about problem w/ exposing header <wingo>yeah good thing we have rlb here to ask the good questions :) <wingo>rlb: we expose mpz and mpq data structures via numbers.h <wingo>mini-gmp's data structures are an exact match <spk121>perhaps we should remove gmp-using api completely w/ mini-gmp in use <wingo>but, i think that if we build with mini-gmp.h, perhaps we should not export the MPZ / MPQ functions <rlb>Hmm, can we remove functions in a Z? <rlb>(public functions -- I'd guess not?) <wingo>if you explicitly enable a nonstandard config option -- yeah sure <wingo>dunno, maybe we could keep those functions... tricky <rlb>also have to think about what happens if libgmp changes and we don't. <wingo>like if they would effectively do mpz_copy() or whatever on their arguments, then we would be good <rlb>i.e. wrt the data structures <spk121>you could make the return (void *) and push struct-matching responsibility to the user <wingo>i really doubt gmp changing mpz_t / mpq_t <wingo>they have a lot of users, it would be a mess <rlb>since we'll be in a situation where we're producing gmp datastructures, and so will whatever libgmp the app is linked against. <wingo>and those are simple data structures <rlb>sure, but they could go to libgmp N+1 during 3.0 and the app could rebuild against the newer release of gmp <rlb>so now it has two sources of incompatible gmp_foo <rlb>"two incompatible sources" <rlb>or maybe that wouldn't compile (still leaves runtime dynamic linkage I guess, unless we have ersioned symbols?) <rlb>Anyway, not sure what the questions are yet, just suspect there are some :) <wingo>so concretely the question is about mpz_t and mpq_t. with mini-gmp, these data structures have the same size as they did but may have a guile-specific allocator. it would not be safe to call mpz_clear on an mpz value that you got from Guile. <spk121>OK. Maybe the thing is that w/ minigmp enabled, we define our own scm_mpz_t which is identical to the included version mini-gmp, and if stuct changes in future, we can rev or rename our scm_mpz_t et al <wingo>let us discard the possibility for now of a change in upstream GMP mpz_t size <rlb>we can't change those without changing our soname? <rlb>i.e. we'll change any code compiled against our changed spec, so our abi changed <rlb>i.e. we couldn't be 3.0.x anymore? <wingo>i think what spk121 was getting at was a different name for a type but which would have the same layout <wingo>so API would change but ABI not. not sure if it solves the problem but soname version would not be the problem afaiu <rlb>What I'm getting at (if I'm thinking straight) is that we can't change to track upstream changes without changing our own soname? <rlb>wrt gmp upstream (if that's what's being contemplated) <wingo>honestly rlb i would really discard the idea of gmp changing their struct sizes for mpz or mpq <rlb>we *could* if we made minigmp a separately versioned shared lib? <ArneBab>manumanumanu: that sounds great! You can just truncate the csv <wingo>what i want to avoid is a user calling mpz_clear on a value produced by Guile. <rlb>basically all I'm thinking is that this may put us at risk wrt potential upstream soname changes, but that might be a risk we're willing to take through the lifetime of 3.0 <rlb>s/thinking/wondering/ <wingo>which to me says, if mini-gmp is enabled, then we should not export scm_to_mpz / scm_from_mpz <manumanumanu>ArneBab: cool! I really do think I can cut the time in half without much fuss. <wingo>which is an ABI change. so --enable-mini-gmp should be off by default. *rlb wonders what we're going to do (and will be allowed to do) wrt debian bullseye... <wingo>if debian thinks it's more important to enable mini-gmp than to keep ABI, it will have to bump library version, because it will be changing the build config in an incompatible way <rlb>we can't change the soname ourselves, or at least that's a huge mess iirc. <rlb>the core problem right now of course is that gnutls is broken <spk121>Couldn't you bump lib version while still calling it 3.0.x? <rlb>but to fix it, it's going to be a hard sell wrt the release managers that we need to upload a new guile and rebuild *everything*, particularly if it'll also break anything people have built themselves against guile testing during the past year or two. <spk121>do there exist any programs that use Guile's mpz api in debian? <rlb>spk121: that might be fine as long as we do it upstream, more difficult iirc otherwise, but it might also mean I'd have to rework the guile packaging, since we only have guile-3.0-libs. <rlb>And I'm not sure how you'd really do it sensibly, i.e. if debian picks some new soname, now it's not compatible with *any* other distribution or build? <rlb>and we'd have to pick one that was guaranteed not to be one that we're going to move to upstream eventually? <rlb>Ideally perhaps, build options maybe shouldn't affect the abi, or that should be rare and clearly called out (plus or minus things like enabling support for extra functions, etc.). <spk121>rlb: for ncurses, there are tons of build options that change the ABI, and ncurses is in debian <rlb>I suppose as long as the build system picks the same defauts "everywhere it matters", then perhaps that's fine. But I think the general expectation is that a binary linked against ncurses 5 on debian should also work against the ncurses 5 on redhat... <spk121>really? I wonder if that is actually true <rlb>I thought that had historically been true, but maybe not. <wingo>rlb: alternate fix for debian, lower risk: set scm_install_gmp_memory_functions to 0 <wingo>keep using external GMP, but guile doesn't use libgc functions to allocate gmp memory <rlb>I guess the main thing for us is just that ./configure have stable behavior on that front during a Y series... <wingo>i mean, if ABI is the paramount thing <rlb>Hmm, ok, what's the disadvantage to that? <wingo>or maybe Guile should change to default that value to 0 if mini-gmp isn't enabled <wingo>rlb: hard to say. broad strokes, Guile can be faster if it allocates mpz values with libgc. but are there external users that depend on not having to mpz_clear their memory? i think unlikely. but if so, they might have a memory leak, if this changes <rlb>(Oh, and of course, this isn't really debian specific, we'll need to make sure that the other distributions know about the situation if we end up requiring accommodations.) <wingo>spk121: summary proposal: if --enable-gmp, we don't expose scm_to_mpz etc. if not --enable-gmp, we keep same API / ABI but don't install custom allocators <wingo>then people can choose their constraints as they see fit <civodul>wingo: re mini-gmp, the two motivations can be: one less dependency, no interference with other GMP users (the alloc thing) <spk121>wingo: sounds fair. And don't expose mini-gmp.h in libguile.h <rlb>...you mean external users that were *relying* our messing around with the allocators? <civodul>and i agree we shouldn't expose mini-gmp.h <wingo>civodul: wdyt about my summary proposal (last thing i said) <rlb>(And if so, would it even have been sensible or likely for them to be relying on that?) <wingo>users do the darndest things :) <rlb>Heh, true, but if we didn't document that we were doing that, and that people could/should take advantage of it, then I think I'm not too worried on that front either... <wingo>s/--enable-gmp/--enable-mini-gmp/ <civodul>wingo: yes, proposal sounds good to me! <wingo>followup question would be, i guess we default to --disable-mini-gmp <wingo>i.e. require explicit --enable-mini-gmp <civodul>so that by default things are like before <rlb>OK, so if we do that, then other than the corner case wrt leakage, we'd need a new debian version, and things might be a touch slower, but nothing else would break, and gnutls, et. al. should be fixed? <rlb>If so, I'll have to talk to the release managers (and the sooner the better) -- not sure whether they'd accept all of 3.0.6, or if I'd need to try to backport the change. <wingo>rlb: you can make this change already with a patch <wingo>numbers.c:162, change 1 to 0 <rlb>And do we know what the *actual* risk is wrt enabling mini-gmp, i.e. if I were to do that, then do we expect anything to break, other than the previously public mpz functions disappearing? <rlb>So maybe that's what we need to do for debian bullseye. <rlb>for "right now" at least. <wingo>probably your least-risky bet considering that 3.0.6 has things that should be fine but may introduce bugs (dlopen rewrite, reader rewrite) <spk121>Just out of curiosity, looking at the big code search sites, the only client using scm_to_mpz is Sorts Mill Tools and the Rust Guile bindings <rlb>do we have any idea if the performance impact is more likely to be minor or major? <wingo>ok i think i am evaporating for the evening -- tx all. spk121 if you feel like poking this tonight feel very free, otherwise i can fix tomorrow evening <wingo>rlb: usually no impact. sometimes causes 80% slowdowns, when compiling specific large files <civodul>rlb: re the Guile/GMP/Nettle/GnuTLS issue, i recommended in the Debian report that Debian built Guile with scm_install_gmp_memory_functions = 0 <civodul>the only case where performance degrades is for bignum-heavy applications <civodul>such as when compiling large files as -O2 <civodul>it's a use case that matters to Guix, but not so much for regular people :-) <civodul>(actually Guix compiles most of its large files with the baseline compiler nowadays) <rlb>Not sure exactly what our Y release policy requirements are, but is this the kind of thing we'd be allowed to change in 3.2? <rlb>i.e. make mini-gmp the default <civodul>but really, the default doesn't matter that much IMO <rlb>...sounds like we'd be removing functions, but maybe that's fine for *our* Y releases (i.e. doesn't require an X) <civodul>in the sense that we have a solution for Debian (& co.) and one for Guix <apteryx>rekado: that's how I understood *text* too, but info '(guile)SXPath' has this procedure: 'node-typeof?', which can supposedly "tests if the node is text via the '*text*' symbol (criterion). <rlb>Sure, though I'd guess if nothing else, if perf is better, we'd rather switch the default as soon as it's reasonable... <rlb>(say 3.2 rather than 4.0, if that's acceptable) <civodul>mini-GMP is supposed to be slower than GMP, for arithmetic operations <civodul>it doesn't have all the fine-tuned asm <civodul>but again, it's "good enough" for most applications <rlb>and I suppose maybe we could eventually borrow some asm for common cases if we liked... <manumanumanu>Forking things like mini-gmp seems like a lot of not fun. <rlb>yeah, I didn't really know what'd be involved -- might well be ill-advised <spk121>aside from avoiding the gmp allocation problem, the use case for mini-gmp is a more portable build or a more embed-friendly build. The real GMP is better computationally, and will likely always be so <rlb>Out of curiosity, do we happen to know if replacing the allocator increases performance because libgc's approach is faster than malloc/free here, or...?