IRC channel logs
2025-10-23.log
back to list of logs
<rlb>What's the issue? (If I can help.) <ekaitz>rlb: a merge request was merged, but it was changing code in lightening <ekaitz>instead of in the lightening project, the changes were applied in guile <ekaitz>so I picked those changes to lightening, and I believe we should revert those from guile <rlb>And we don't want that in guile itself for 3.0.11? <rlb>If so (if we don't want it), then yeah, just need to push a revert I suppose. <ekaitz>rlb: maybe but lightening is merged in guile from its own repop <ekaitz>so the change has to be in lightening first, then pulled to guile's lightening branch, and then merged in the subdirectory <rlb>That's fine I think, i.e. it's fine (if everything else is fine with having it in guile now) to have it both places -- might just cause a bit of noise on the next merge. <rlb>But if there's no reason we want it in guile for now (on its own) and/or if it's not suitable for a Z release, then right, probably need to push a revert. <ekaitz>i'd say the easiest is to revert it and update lightening in guile <ArneBab>but revert + update lightening in guile might be cleaner. <ekaitz>ArneBab: do you know how to do the merge? <rlb>I assume current lightening is OK for a Z? <rlb>(no backward compat changes) <ekaitz>let's check which one is the latest change <ArneBab>can I somehow see the permissions I have for a repo in codeberg? <rlb>You should be able to see in settings if you have "sufficient perms" I think? <rlb>Can you push directly to the repo? <rlb>If not, could also raise a pr with the revert. <ArneBab>can you do the update from lightening directly after? <rlb>me? (If so, I've never done that, and don't know what's required, and/or how to review the lightening side wrt ABI, etc. So I'd at least want some guidance from Andy or Ludovic.) <ekaitz>we can start merging lightening from the separate repo <ekaitz>and then we can open a pr to merge it in the proper directory <rlb>Also don't know if there are any docs in guile's tree about that -- could grep if you haven't (and/or look at the last time someone did merge lightening in the log). <ArneBab>ekaitz: ok, then I’ll create a PR to revert the lightening changes. <ekaitz>rlb: the lightening branch is merged from lightning itself and then it's merged directly in guile with the subdirectory strategy or whatever that is called <ArneBab>(please only merge this once updating lightening is ready: undoing a fix that finally got in is a bit sensitive) <ekaitz>ArneBab: well maybe not that much, we just need to merge yours first <ekaitz>rlb: lightening didn't change since months ago <rlb>ekaitz: hmm, I'd assume the main question is about the lightening diff with respect to the last commit that was merged into guile, whenever that was. <rlb>i.e. "git diff LAST_MERGED_HASH" in the lightening repo on the relevant branch. <rlb>If that CI config isn't in the lightening repo, then I'd incline toward putting that in a separate pr <rlb>i.e. keep the lightening "merge to update" changes separate, and maybe also keep the merge commit (i.e. no ff). <rlb>So it's obvious in the graph that the relevant collection of commits is all "the lightening update". <rlb>(e.g. like I did for srfi-207) <ekaitz>the CI thing and such are all part of lightening <rlb>In truth, I might just merge the revert as a ff, then the lightening changes with a merge commit (and whatever documentation in the merge commit we typically want for a lightening update -- guessing that might be obvious from the last time it was done) <rlb>Then the CI update separately. <rlb>I'm assuming Andy or Ludovic wanted to proceed with the CI changes? <rlb>ACTION is in favor of pr tests, as long as we get any security issues right (gh is a mess on that front, wrt prs, I believe). <rlb>(Though we probably also don't have many/any secrets to worry about.) <ekaitz>rlb: so we should merge ArneBab 's PR first and then mine <rlb>Sounds plausible -- though then, of course yours would want an update to remove the revert (I'd do that to be sure the merge is clean). <rlb>i.e. I'd add the revert, then remove the revert (and CI) from yours, then merge yours with a merge commit, and then deal with CI. <ekaitz>are you sure? codeberg should just do <ekaitz>if ArneBab 's is merged as ff then mine can be merged with a merge commit <ekaitz>like other lightening changes are included <rlb>yes, but I'd force-push yours to remove the revert so I don't have to worry whether it'd automatically disappear cleanly -- it might, but if it doesn't... <ekaitz>if it doesn't I'll fix it, don't worry <rlb>you mean it'll automatically change your pr? <rlb>Because otherwise "if it doesn't" would be too late, the history's already made. <rlb>But none of this is critical -- I'm just suggesting what I might do if I were the one handling it. <ekaitz>rlb: prs are live, mine won't be changed, but it will compare mine against the new guile <ekaitz>and the new guile will have the other's pr changes included, so they won't appear in the diff <rlb>It depends on what it does when you hit "merge", and since I've had experiences with gh I ended up with weird extra "trangular" merge commits (not the same situation) I just tend to fix things the way I want them ahead fo time. <ekaitz>i'd say you are thinking on this like the prs were patches in email <ekaitz>anyway, if it's merged with ff mine will just work, we have the same commit <ekaitz>if we merge it in any other way and it doesn't work I'll force-push and fix my pr <rlb>Hope you feel better soon. <rlb>ekaitz: your earlier comments were right -- I thought this was a merge of some commits to update guile wrt lightening changes, not a merge of the full lightening branch from the last time, from within our repo, but if so, then I wondered if we might want to update our lightening branch (head) too? <rlb>(Your subtree merge comment didn't fully sink in earlier...) <rlb>(will tell ekaitz later...) <ekaitz>ArneBab: i saw you merged everything, pretty clean! good job <apteryx>euouae: do you sometimes review guile stuff? <apteryx>the truth source guile repo is now the one on codeberg, right? <apteryx>I can't seem to find an announcement on guile-devel <euouae>I think there's some lag, gnu.org does not mention anything about codeberg but you can see that the savannah repo is far behind <civodul>apteryx: i’m not sure this was announced… <apteryx>with more interesting things to come if I get this one in ;-) <apteryx>I seem to recall some conversation somewhere, but can't find it <euouae>apteryx: I can take a look after I'm done with the elf library but keep in mind that I'm not a maintainer <euouae>so my reviews do not necessarily accelerate the acceptance <old>what's that elf library stuff? <euouae>even if only specific segments are needed to be examined <euouae>I took a look at (guix elf) and decided to rewrite it in r7rs-simple <euouae>fixing in the process those issues <euouae>It's also part of guile as (system vm elf) <old>why guix does not use Guile elf parser? <old>I guess this is a problem only for huge elf file <old>possibly on 32-bit systems also <old>other just mmaping the whole file in virtual memory should not be a problem <euouae>personally I dont know how to use mmap <euouae>in a manner that I can say "this is secure" but also mmap on windows available? <euouae>of course they have their own thing, but it's not called mmap <euouae>guix doesnt run on windows but guile does and if its part of (system vm elf) ... it's nice to provide a platform indep. elf library <old>Windows has virtualloc <apteryx>but so far it seems to not help, I just loaded a ~300 MiB .a.debug ELF and it used a vast amount of memory and CPU <apteryx>maybe the parser had to scan the whole file, though looking at the parser I don't see why this should happen. <euouae>you need to modify it, but anyway ;P I'm fixing this <old>apteryx: you don't need to parse the whole file for inspect a ELF file <old>but maybe the parser in Guile is not very smart and does parse everything <apteryx>another thing with mmap, pointed by civodul, is the use after free risk from Scheme, which is not great. I think their proposal was to use a C implementation that can use a finalizer to conveniently abstract away munmap when the mmap-backed array is no longer referenced <old>that can't be that difficult to manage if we can manage file-descriptor no? <apteryx>ACTION wonders why we can't just have some finalizer API exposed to Scheme <old>we can just use guardian for that? <apteryx>that's ugly, you need a background thread <old>or just do manual memory management when using mmap <old>guardian can be installed on gc hook <apteryx>oh? any examples I could look at? maybe that could be a way then <apteryx>manual memory management means introducing C problems at the Scheme level... not great. <civodul>finalizers are exposed via set-pointer-finalizer! for instance <old>apteryx: in fibers there's one for epoll file-descriptor <apteryx>civodul: ah! So we don't need C for an mmap wrapper, do we? <civodul>apteryx: we do, because i think we need to change bytevectors to do it right <old>that's the adaptation of fibers I made for Guile <civodul>also, note that read-elf in Guile would have to be changed to use mmap <old>I think the guardian is also present from the original fibers if you want to look at it <euouae>civodul: I'm writing an elf parser that uses seek instead and let's you filter which program header segments (etc) you want <euouae>It's also r7rs compliant (despite r7rs not having seek; I just used a higher-order proc for that) <apteryx>civodul: yeah, I was testing the mmap binding I have with: (call-with-mmap-bytevector "/gnu/store/sn7gb8wbw5q1f1r0daqm5imdwahx7b1g-qtdeclarative-6.9.2-debug/lib/debug/gnu/store/x6cipsvawmsm33pb9ryzx2kqxjv8d5wa-qtdeclarative-6.9.2/lib/libQt6QmlLS.a.debug" (compose elf-dynamic-info-soname elf-dynamic-info parse-elf)) <old>euouae: the problem with that is now you have global state <old>you can not share your elf object in multiple-thread, the seek cursor is global to the file-descriptor <civodul>apteryx: oh right, it actually takes a bytevector, not a port <euouae>old: no it's fine because it seeks to read the things it needs <euouae>As long as you lock for the read it's fine (You're talking about using a port between multiple threads for some reason?) <old>well I might have misunderstood but let say I open a elf file and pass the result to two threads <old>one thread want to read section A, the other section B <civodul>for things like ELF, mmap remains the best option, but having a seekable variant that reads stuff lazily would be an improvement <old>the seek cursor is shared, so it won't work <euouae>old: as long as you use a mutex for the reads you're fine, otherwise open it twice <civodul>old: it would work provided the caller pays attention :-) <euouae>any file operation needs a lock in that sense <civodul>that is, if you open a port for just this purpose *and* you make sure to access it from a single thread <old>euouae: well not true. some file operation are atomic by nature <old>but seeking + reading is never atomic <old>well not true .. you could use readv <euouae>I don't think I follow your point honestly <euouae>you're viewing it from the system call perspective, but what about the port interface? <old>I haven't touch the elf api for a long time, you provide a port to it? <euouae>Anyway IMHO what I'm doing is fine and I don't agree with the criticism because it applies for all ports <civodul>but i think the most important part here would be to have a declarative fully ELF interface, à la Poke <civodul>which could work equally well on a port or a bytevector <euouae>like a general scheme macro that allows you to define formats by declaring the bytes/orders/sizes? <euouae>and provides parsers/validators? <old>euouae: there are no criticisms. I'm just trying to understand the problem at hand here and why mmap is not used in the first place <old>if the reason is: 32-bit systems need to be support at all coast then fine by me <civodul>euouae: yes, sorta like bytestructures <old>otherwise, I don't see why mmap can not be just used <euouae>civodul: it just gets complicated because of the strange logic that can occur between bytes (for general format) <euouae>and I don't think that's the most important thing, actually ELF is not so complicated, I am close to finishing the elf library <rlb>ekaitz: mentioned last night, but before I noticed you'd left -- I had indeed misunderstood more broadly; I'd thought that was a merge of some commits to update guile wrt lightening changes, rather than a (re)merge of the full lightening branch, from within our repo. Sorry for the distraction. <ArneBab>ekaitz: thank you for your good job, too! <ArneBab>shouldn’t we automatically synchronize codeberg to savannah? <ArneBab>(⇒ push changes from codeberg to savannah) <ekaitz>rlb: don't worry mate! it was late yesterday <ekaitz>civodul: we should put fibers in the same org and make an announcement, shouldn't we? if it makes some trouble for you to update the links I can help <dpk>is there any way in Guile to hook into the compilation pipeline? specifically i want to grab the Tree-IL that comes out of the expander and modify it before it goes to peval etc. <dpk>ACTION considers a monkey-patch of (language scheme compile-tree-il) … <rlb>dpk: hmm, did I do that for lokke? I did have to make tree-il level adjustments, so if that might help, I can track it down. <rlb>(but might not be what you mean) <mwette>dpk: it might be easier/cleaner to create a new language by copying language/scheme/spec.scm and hooking into those routines <rlb>(I think that's how I did it.) <rlb>yeah - thikn I added a #:compilers `((tree-il . ,tree->tree-il)) to the define-language spec. <mwette>scheme's compile-tree-il just calls the macroexpander <dpk>okay, then how do i tell Guile to use the dpk-scheme language to load modules instead of the regular scheme one? <rlb>dpk: hmm, not at all sure -- could changing current-language do it? <dpk>rlb: well, it breaks it in the right way :D <dpk>when i set current-language and then import something, it complains that there is no such language, although i already loaded it … <rlb>...hmm, I recall hitting some "complexities" around defining/"activating" a language, but unfortunately don't remember the details offhand. <dpk>ah, it was a silly error. i had copied and pasted the (language scheme spec) definition to get started, and remembered to change the name of the library but not the name given in the define-language syntax <dpk>so, i wrapped compile-tree-il with something that just prints the result of compile-tree-il, and it works <dpk>so yes, that’s how to do it. thanks!!