<anadon>I sent this to John Cowen, but maybe a few people here could also comment. I'm trying to consider a general use case of a Guile file describing a datastructure which spans something obscenely large -- say 100's on TB. So big that is actually can only exist across multiple systems each of which only have some chunks of it. Then also say that there are multiple readers/writers acting on this said file which need to be able to access inde <anadon>xed or referenced data efficiently and not with linear parsing of the entire thing. How might I think about or solve this problem? <anadon>My best thought is to put in some reference in the place of a subtree when it gets too big. Then when a given subsubtree becomes too large to parse or index efficiently to apply a similar reference and put that data into a new further down. Then for normal lists which become too large, have a seperate data structure to have ranges of indecies put into sub-trees for that portion of the now split list, and have that tree structure handle <anadon>what were flat datastructures. But this has to have been solved by someone before. <anadon>RhodiumToad: I'm assuming in 100TB of a file that there would be 1D parts which are large enough to need to be split. <anadon>A general case "how do I handle stupid large files" <RhodiumToad>I swear I've seen something describing the use of just such a sparse object, but I'm drawing a blank on what it was <anadon>I figure I can't have been the only one to run face first into this. Just, I have no idea how to find their work. <daviid>if it hellp ... not guile related obviously <rlb>wingo: make-regex isn't exported from (ice-9 regex) even though it's documented? *rlb is surprised no one's noticed if he's not missing something. <rlb>i.e. with 3.0.2 (use-modules ((ice-9 regex) #:select (make-regexp))) crashes here. <rlb>s/make-regex/make-regexp/ <anadon>daviid: That's a side effort, and one that I find knee-capped. <dsmith-work>rlb: I've been using it (without #:select) with no problem. <rlb>Does it work with the #:select for you (if you have time to try)? <dsmith-work>In procedure resolve-interface: no binding `make-regexp' in module (ice-9 regex) <rlb>I'm guessing it's missing an export somewhere... <rlb>Though don't know offhand why it works in the non-select case, if so. <rlb>hah, oh, well that might explain it <rlb>I forgot to check that. <rlb>Since make-regexp is available by default, and regexp-substitute/global isn't, maybe it *is* a bug. Not sure, perhaps I didn't re-read the docs carefully enough. <dsmith-work>rlb: Yeah, it's not clear from the manual that some regex things are not in the module (regexp? make-regexp regexp-exec and the regexp/{thing} flags) ***Aurora_iz_kosmos is now known as Aurora_v_kosmose
***wxie1 is now known as wxie
<sneek>Welcome back tohoyn, you have 2 messages! <sneek>tohoyn, wingo says: definitely a big <sneek>tohoyn, wingo says: definitely a bug <str1ngs>daviid: thanks daviid, will check it out. <wingo>RhodiumToad: depends on how the rest of the system is compiled. there is a bug tho in the jit; some patches not yet merged <RhodiumToad>ok. building it for armv7 on freebsd (under qemu for speed) crashed for me <janneke>do we have a convention for writing changelog entries for goops methods, especially if they have several specializations? <janneke>(foo): Add specialization for <bar>. <janneke>(foo): In <bar> specializer, do blah. <dsmith>RhodiumToad: That lightening MR *definitely* fixes the "Illegal instruction" failures I was having on armv7, both with -mthumb and -marm. However, <dsmith>I would still get some segfaults. A different problem I think. <dsmith>RhodiumToad: You can build without errors if you set GUILE_JIT_THRESHOLD=-1 (disables the jit) <manumanumanu>I am getting some errors that are interfering with building from master under macos/homebrew <manumanumanu>I can probabl correct them all, but I have a question about language: Does guile documentation use what I think is called the oxford comma? <dsmith-work>manumanumanu: I think ", and" from some simples grepping. <manumanumanu>anyway, I am trying something else. I think I was to blame for the documentation error. The warnings are because of xref missing a comma afterwards (which for the two places were sane, even regarding the oxford comma). <manumanumanu>Now bootstrapping to see if I really was to blame. If so I will probably go into hiding (even though noone else seems to be hit by this) <manumanumanu>wingo: my srfi-171 patch broke installing on os x due to my malformed documentation. I am so sorry. A patch is on it's way to the mailing list in about 1 minute. <wingo>np, i had't even noticed yet :) <manumanumanu>wingo: Patch coming whenever my email provider is up... Sorry again. <dsmith-work>wingo: FWIW, I've not seen any problems with aarch64 *wingo finishing new quick-and-dirty compiler... <mwette>wingo: Sweet! Will the debugger provide better visiblity into variables? <mwette>And I guess bootstrapping will be snappy fast. <wingo>manumanumanu: is to reduce compile time & space. result will run slower than cps but compiler will run faster <wingo>for -O0 and similar. i think debugging may be more straightforward also but that's a secondary thing. <dsmith-work>wingo: an since intel/amd is also fine, /me thinks those segfaults must be isolated to the armv7 jit code. <rlb>civodul: if I understand correctly, wondering about the possibility of making GUILE_SYSTEM_EXTENSIONS_PATH "official", i.e. documenting it (or doing something else), so that projects that build on guile have a supported (when it's available) way to augment the load-extension/dynamic-link path. <rlb>Or rather, some documented way to provide a dir with your own libs that should "come first" without having to change LD_LIBRARY_PATH. <rlb>(which affects all subprocesses as mentioned in dynl.c) <rlb>...and better yet, some way to do that after guile starts up so that a (shell/env) wrapper isn't mandatory, e.g. (set! %system-extensions-path (cons ... %system-extensions-path)) as per %load-path. <civodul>rlb: GUILE_SYSTEM_EXTENSIONS_PATH is really for Guile's own .so files <civodul>so what we'd need is GUILE_EXTENSIONS_PATH <civodul>but that'd be synonymous with LTDL_LIBRARY_PATH currently *rlb would be quite happy with that :) <rlb>It's not a critical concern right now, just wondered. <rlb>i.e. I can just set LD_LIBRARY_PATH -- all the lib names are quite unique, but it might be nice to have a somewhat "cleaner" option like that. <civodul>LTDL_LIBRARY_PATH is a bit cleaner than LD_LIBRARY_PATH, but yeah <rlb>Good suggestion, thanks -- if nothing else, I'll switch to that. <rlb>civodul: and it'd be nice to have %extensions-path too, so you can adjust it from within guile, and programs won't be required to have a shell/env wrapper. <rlb>(if we were to add GUILE_EXTENSIONS_PATH) <iv-so>has anyone here successfully used guile-git with guile 3.0? <mbakke>iv-so: Guix uses it pretty heavily <iv-so>ik, i am trying to package guix for my distro and am stuck with dysfunctional guile-git rn <civodul>iv-so: how dysfunctional? does "make check" fails? <iv-so>like this: git/bindings.scm:66:8: In procedure git_libgit2_init: Function not implemented <lfam>I was on Guix System, and I used `guix pull --core-updates`, then using that, built Guix from source. I saw that error later when trying to do `./pre-inst-env guix system reconfigure ...` <mbakke>iv-so: in what context do you get that error? <lfam>It went away after I built Guix from source using the master branch, and then reconfiguring <mbakke>could be a GUILE_LOAD_PATH issue <iv-so>mbakke: when i eval (use-modules (git)) for example