IRC channel logs
2014-02-08.log
back to list of logs
*davexunit is working on a GLSL shader module for guile-2d *nalaginrut need to take up after vacation... <davexunit>Fuuzetsu: just doing the simplest thing for now and explicitly passing the agenda. <nalaginrut>davexunit: I was trying a cooperative server before the vacation, same principle to yours, the difference would be the scheduler, and each thread won't be infinite <nalaginrut>I think game object thread would be infinite till user end the program <davexunit>nalaginrut: do you think the approach that's being taken for 2.0.10 is flawed in any way? <davexunit>because we would probably prefer that people use *the* coop-repl-server instead of rolling their own. <nalaginrut>is it recent patch? I haven't read it, just end the vacation <nalaginrut>and I'm afraid to open my mail client now, it's very slow, too many filters, although I configured mu/getmail/emacs, I realized I have to write procmail script myself, and I'm writing it with Guile, hmm... <nalaginrut>davexunit: anyway, I don't think it's bad idea to implement a new one, because the inner server in Guile is actually a web server <nalaginrut>you never know what kind of server users want, and Guile core can't cover them all <davexunit>nalaginrut: guile 2.0.10 will have a new module (system repl coop-server) <davexunit>it's a repl server that plays nice with single threaded programs and ones that use an event loop. <davexunit>nalaginrut: it's up to the user to integrate it as they see fit. <davexunit>the server is still multithreaded, but the actual evaluation happens from within a single thread. <davexunit>mark weaver and I have done a bunch of work on it. <davexunit>nalaginrut: I submitted the original patch and mark weaver is doing additional work. *nalaginrut opening Evolution <nalaginrut>davexunit: I saw you blog, seems it's hard coded for poll? <davexunit>the thread that should do the evaluating needs to poll the server at regular intervals. <nalaginrut>is there anyway to specify other event handler, say ,epoll if I have <nalaginrut>I'm sorry, it's not some kind of judge of you contribution, it's cool thing! ;-) <nalaginrut>I'm aiming a high performance server design without compatible concerning, so I think it's fine to co-exist with the inner server <nalaginrut>but I'm glad that your contribution introduced mvar as a co-product <davexunit>mvars are actually *not* going to be be in the next release. <davexunit>so instead mark weaver is changing it to use regular mutexes and condition variables. <Fuuzetsu>considering these problems, I'm scared to think what other bugs you guys have *nalaginrut removed several filters and ML to accelerate his mail client... <mark_weaver>Regarding O_NONBLOCK, you asked "wouldn't dupping the fd be sufficient, somehow? is the object with the O_NONBLOCK the same object with the seek position?" <mark_weaver>I have answers for you. Yes, the object with the O_NONBLOCK is the same object with the seek position -- that's 'struct file' in fs.h -- but 'dup' returns a new fd pointing to the same 'struct file'. <mark_weaver>and actually, the 'dup' manpage specifically mentions that both fds share a file position. <wingo>i was later told that CLOEXEC is the only per-fd flag <wingo>we would have to have some marker about whether a fd is "ours" or not, or whether to assume it is nonblocking... <mark_weaver>btw, if you read the 'fcntl' manpage, you'll see that FD_CLOEXEC is set with F_SETFD, whereas O_NONBLOCK is set with F_SETFL. <mark_weaver>F_SETFD set file descriptor flags; F_SETFL set "file status" flags. also note the different prefixes of the flags themselves. <mark_weaver>note that O_APPEND also falls into the same category. <mark_weaver>yeah, we could do that as an optimization, but we still need a solution for fds that point to ofiles that are not exclusively ours. <mark_weaver>and also, we could consider that a file that starts out exclusively ours may not end that way. and also, that even files that are exclusively ours could be shared by the guile user herself, and that they may assume that O_NONBLOCK is off. <mark_weaver>Bernstein has been asking for better support for nonblocking I/O for a long time, on the linux (kernel) mailing list and on his web site. <mark_weaver>I can think of just one portable POSIX solution, really, but it's wasteful. <mark_weaver>have a separate thread do the I/O, and talk to it via a pipe. <mark_weaver>it seems that pipes also have better properties in terms of non-blocking behavior. disk reads and writes actually block even with O_NONBLOCK. <mark_weaver>(lots of things don't really support non-blocking properly) <mark_weaver>I wonder if anyone has actually submitted a patch to Linux to add the nonblock_read and nonblock_write syscalls that DJB is asking for. <mark_weaver>on platforms that support it, we could use asynchronous I/O. <wingo>vectors.c is 666 lines long. <wingo>i had no idea you could sort arrays in guile; neat <lloda>scheme@(guile-user)> (sort (make-typed-array 'f64 1 10) <) <lloda>ERROR: Wrong type (expecting non-uniform array): #f64(1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0) <lloda>the first line was the output for (sort (make-typed-array #t 1 10) <) <lloda>it does feel silly to convert each 'f64 to SCM before you call the predicate :-/ <mark_weaver>if the sort procedure was written in scheme, and was able to inline the predicate, the conversion from 'f64 to SCM could be avoided when we support unboxed float ops. <mark_weaver>in the meantime, it would still be good for 'sort' to support sorting uniform arrays/vectors. <mark_weaver>wingo, civodul: I'm a bit concerned about deprecating so many interfaces without a compelling reason. <mark_weaver>for example, the ntohl et al stuff, which combined with the uniform-vector-* interfaces have been recommended by the manual to do network I/O. <mark_weaver>I don't see what we're getting in return for breaking existing code. <civodul>ntohl et al. are really an omission in the manual <civodul>we've had the bytevectors API for 3 years <mark_weaver>an omission? what do you mean? they were just removed from the manual within the last day. <civodul>well, i mean the example in question <civodul>which i still have in my installed copy here ;-) <civodul>it makes sense to remove that example IMO <civodul>i think removing uniform-vector-* will allow us to do vector == tc7 in 2.2, right? <mark_weaver>I agree that the docs should recommend using the bytevectors API. <mark_weaver>but I don't see why we're removing this stuff, and why we're so impatient to get rid of it. <civodul>my understanding was that it was for the vector thing, which in turn allows for better optimization <mark_weaver>I wish that devs didn't feel the need to keep supporting 1.8, but the fact is that many still do want to. and they'll have to now have two copies of their network code, one for 1.8 and one for 2.2 I guess. <civodul>that said, i think the mailing list is underused :-) <civodul>the GnuTLS tests use uniform-vector-*, for instance <civodul>because that's the only thing that works for both 1.8 and 2.0 <mark_weaver>I recent months, I've had vastly better results talking to Andy on IRC than on the ML. <mark_weaver>(I'm not complaining, just giving my reasons for using IRC :) <civodul>i'll try to catch up with the commit list <mark_weaver>I'm also worried about some of the recent changes on stable-2.0 which will break existing code. <wingo>mark_weaver: the example didn't work. <wingo>there was make-uniform-vector <wingo>which is not an interface in guile <wingo>two versions of network code is unavoidable when you bring 1.8 into the picture; it doesn't even have binary i/o <wingo>sorry i've been bad at the mailing list recently, i need to be serious about my mail :/ *wingo been avoiding it for a while <mark_weaver>it's okay. I can sympathize. sometimes in order to get real work done, you need to drop some things on the floor. and you've been getting quite a bit of real work done, which is much appreciated. <Yawar>hi folks. (fairly) new to guile. am trying to write a unit test runner for a unit testing framework i've written. i've come up against the main problem: how to load an scm file, find all the anonymous thunks contained within, call them one by one, and get their return values. can anyone point me in the right direction? :-) <mark_weaver>blindly calling all the anonymous thunks in a file seems like a very bad idea to me. who knows what some of them might do. <mark_weaver>and non-anonymous procedures of no arguments are macro-expanded into anonymous thunks. <mark_weaver>after macro expansion, lots of things might become thunks. <mark_weaver>FWIW, we just recently added SRFI-64, which is a fairly portable test framework for Scheme. It'll be in Guile 2.0.10, due out soon. <Yawar>mark, i understand that things like eval and load could be dangerous, but i don't see a way around them if you're going to implement an automated test runner <Yawar>hmm ... does srfi-64 have a test runner? <dje42>mark_weaver: Is stable-2.0 the branch to use for testing 2.0.10? <mark_weaver>Yawar: btw, although SRFI-64 will be included in Guile starting with 2.0.10, you can use the reference implementation (linked from that page) right now. <wingo>man, this ra0 patch series is tricksy... <mark_weaver>yeah, there's a reason why it's been sitting around for so long... <mark_weaver>Yawar: FWIW, I'm not trying to block you from doing what you want, but there's not really any good way to do it. Before macro expansion, you can't even parse the code properly, because you can't understand the meaning of the macros in general. After macro expansion, there will be lots of thunks that wrap internal bodies of things, which wouldn't make sense for you to call. <Yawar>mark_weaver: thanks for the links. if i'm not mistaken with the srfi-64 test runner you actually have to write down some code, somewhere, to call each of your test suites with the runner. i'm trying to do something more automated. <mark_weaver>Yawar: can you explain the basic idea of how you want it to work? <dje42>like pytest? [just guessing] <Yawar>sorry not familiar with pytest :-) *mark_weaver is familiar with neither of those. <Yawar>in case y'all haven't used rspec, i'll explain a bit more... <Yawar>i'm planning to put a bunch of files inside a 'test' or 'spec' directory, each file containing something like: <Yawar>each suite form returns a lambda which, when called, will return the given lists of passed and failed tests <Yawar>i want to load each file, call the lambda returned by each suite form, and gather up the return values of passed and failed tests <mark_weaver>the macro can expand into code that does what you want. <mark_weaver>and that's pretty much what srfi-64 does anyway. I'm not sure how this is different. I guess I still don't quite understand. <Yawar>well, srfi-64 has test runner functions (macros?) that you have to call for each of your test suites, you have to manually code these calls <Yawar>so each time you create a new test suite you have to write up a new test runner call to actually run that test suite <mark_weaver>well, you never have to manually code anything in scheme. you can always write macros to generate the code. <Yawar>i'm hoping to have a guile script that discovers all the test suites in a certain directory and runs them automatically <mark_weaver>but I guess what I'm saying is this: instead of asking for a list of all thunks in a file, instead, just load the file, possibly with some set of macros predefined. would that work? <mark_weaver>and of course, you can traverse a directory structure to find the files easily enough. <Yawar>right, walking the directory is a fairly simple problem. but getting the return values from each lambda in the file is my immediate problem <Yawar>i.e., when i don't know beforehand how many there are in the file <dje42>IIUC, Yawar wants the files that are loaded to be the tests (define (test-foo ...) ...) (define (test-bar ...) ...) and that's it. It's up to the test harness to find those functions, run them, and collect hte results. <Yawar>right, i.e. i want the files that are loaded to each _contain_ a set of tests <Yawar>and then it's the harness' or runner's job to execute them <dje42>Have a convention for names of top level functions that are the tests. test-foo or some such. ? <wingo>lloda: you can't remove the array from the array handle and replace it with the shared-array-root; otherwise you have no GC reference that will keep the dimensions alive <wingo>because the dimensions belong to the array. <mark_weaver>well, if you constrain the top-level syntax of these files to some set of forms that you can understand, then you can just 'read' each one from the file, and then basically do the macro expansion by hand to get the lambda. <mark_weaver>and then use 'eval' to evaluate the lambda expression and then call it. <dje42>I'm not sure I'd want to constrain the top level forms *too* much. <mark_weaver>well, there are better ways of doing this, IMO, but I get the impression that Yawar would just prefer a solution that uses the syntax he's already decided on. <mark_weaver>if the top-level forms were simply the lambda expressions, e.g. (lambda () ...) then it's trivial. <mark_weaver>just 'read' each form, pass it to 'eval', which returns the thunk, and then call the resulting thunk. <dje42>That's one way sure. Part of what Yawar would like [IIUC] is the code that actually does this to be part of srfi 64 (or whatever). <mark_weaver>if he wants the toplevel forms to be of the form (define (foo) ...), then he needs to extract the lambda from that, which means expanding the 'define' macro into (define foo (lambda () ...)) and then extracting the lambda out of that. <dje42>[I only read srfi 64 once, so I don't remember it well enough.] <Yawar>dje42: actually, i don't have a requirement for srfi-64. it's just that mark pointed me at it and i found it didn't have something i need... <mark_weaver>Yawar: if you're willing to use a different keyword other than 'define' in the top-level syntax, then it's trivial. you could use SRFI-64. <mark_weaver>e.g. instead of (define (foo) ...), something like (test (foo) ...), then you can have 'test' be a macro that expands into the SRFI-64 syntax.. <mark_weaver>and then all that remains is to traverse the filesystem and load all of the files. <Yawar>mark, it might be as trivial as you say. in my test files, i don't need to have 'define'. the only top level i _need_ would be the 'suite' form which when called, returns a lambda <mark_weaver>why does it need to return a lambda? why not have it just perform the actual test? <lloda>wingo: hm. which patch are you looking at? <mark_weaver>Yawar: you could make 'suite' be a macro that expands into whatever you want. instead of returning the lambda, it could add the lambda to a global list that's later iterated over by your test runner. <Yawar>mark_weaver: hmm, let me think for a sec. my original justification was that i need to collect the numbers of passed and failed tests from each 'suite' call <mark_weaver>you can provide arbitrary code that gets called after each test, and can look at things like whether it passed or failed, the expected and actual results, etc. <Yawar>right, i was reading the doc for s64, but again each test runner appear to be limited to running a single test suite in the same source file. <mark_weaver>no, there's no constraint that it be in just one file. <mark_weaver>you can establish a test runner (a custom one if you like), and then load up a bunch of files, and all the results will be accumulated together. or group them. it's very flexible. <mark_weaver>and you could write custom macros to make the syntax pretty much whatever you want. <lloda>wingo: I suppose it's 'Use underlying vector implementation directly in array handles' <Yawar>mark, the s64 runner is doubtless very flexible, but my feeling is the amount of work i'd have to do (which admittedly may not be much) to get it to load up a bunch of files, run their tests, and accumulate the results, would be equivalent to the amount of work to write my own runner for my framework which does this job <wingo>the "use underlying vector implementation" patch <Yawar>the key here might just be using 'read' ... i'll look into that. thank you! <mark_weaver>Yawar: it's trivial. if you want to do your own thing, that's fine, but personally I think it would be a mistake. <mark_weaver>all you have to do to load a file is 'primitive-load'. <mark_weaver>that basically all the forms of the loaded file as if they were in the same file. <Yawar>i look at it as a learning exercise. <lloda>doesn't the handle itself protect the dimensions? <Yawar>thanks for your input though, i now have several things to explore. <lloda>it seems off to have the array handle keep a reference to itself. It must not be used to access the array when the root can provides direct access. <mark_weaver>Yawar: fwiw, even if you do your own thing, I think it's better to define a 'suite' macro that adds the thunk to a list of accumulated thunks. then you can just load the file. <mark_weaver>the list could either be a global, or a fluid/parameter (basically a dynamically-scoped variable). <Yawar>ah, actually i wanted to avoid global state :-) <mark_weaver>(it's not unreasonable to prefer to treat the file as data containing code fragments, rather than as code that accumulates the list of thunks). *mark_weaver goes afk for a while. <lloda>wingo: if the handle itself doesn't protect the dimensions, then I still want to have the root there and never use the reference to the array to do anything. <lloda>actually I'd be even happier if the array handle structure was gone. as far as I can tell, it's only needed to make unarray-like objects pass as arrays (strings, bitvectors, etc. and the roots themselves). The dimensions are right there in the array SCM for actual arrays... <wingo>lloda: why do you want that? <lloda>because then once I have an array I don't need to check whether this is an actual array or one of the other n types <lloda>all the rank/dimensions/shared array ops can be done directy on the array SCM b/c they wouldn't depend on the type of the array <lloda>but that's going too far right now, isn't it <wingo>so it seems possible to avoid the array handle structure inside the implementation of tc7-array <wingo>but it's not something that's a good idea for guile as a whole <wingo>the array handle structure is (i think?) a useful unifying C interface to arrays <lloda>the way it'd work is that (vector ...) is not an array. If you want one, you do (as-array (vector ...)) which will be a nearly-trivial op. <wingo>so including vectors and uniform vectors and such. <lloda>I see how that makes sense, so I'll stop. <wingo>we need one generic interface to array-like objects (and neither more nor less :) <lloda>so about array -> root. Is keeping the array ref in the handle really necessary? <wingo>i think it is from a gc perspective; and it is from a compat perspective too <wingo>i don't know how to deprecate that properly, and there is a lot of code out there <lloda>then can we keep both the root & the array ref in the handle? <lloda>and never ever use the array ref for anything <wingo>i think i would like to remove the whole "impl" thing <wingo>it was always internal, just introduced in 2.0, and it seems that open-coding the implementation in array-handle.c is doable <wingo>if we include the vref / vset members in the handle <wingo>that's another way to cut through the layers <lloda>that would be much better than going through the impl table every time <wingo>you mean for tc7-arrays dispatching on their vector, i guess; yes. <wingo>i didn't like the approach of just saying "bv_handle_ref never looks at the dims, therefore it's ok to play this game switching the impl" <wingo>better to have it all in one file <lloda>I did it b/c before this, the shared-array-root could be an array itself. <dje42>Is there any easy compilation (autoconf) test to verify the guile one is getting is >=2.0 (e.g. and not 1.x) ? <lloda>wingo: but after this, that can't happen, the root is always a raw vector (vector/bytevector/etc) and those don't have any dims/ndims/fields <wingo>dje42: pkg-config check for guile-2.0 versus guile-1.8 ? <wingo>there is also the GUILE_PROGS macro from guile.m4 <wingo>see the manual, it defines ways to require different guile versions <lloda>wingo: that's the whole point: the reason why it says SCM_ARRAY_IMPLEMENTATION (scm_tc7_array,...) and that had to be removed <wingo>sometimes it is difficult to figure out what the point of some patches is ;-) <lloda>this is what that patch does (+SCM_ARRAY_IMPLEMENTATION (scm_tc7_array, 0x7f, NULL, NULL, NULL)) and I only could remove this line finally in the top commit. <lloda>it's being done now and that's what matters <lloda>so I accept that /* this works because the v's impl NEVER uses dims/ndims/base */ looks irresponsible <lloda>it was scaffolding, I suppose <dje42>Is there a preference of pkg-config over guile-config? <wingo>dje42: depends; pkg-config is better for selecting versions and getting compile flags, and guile-config uses it internally, but guile-config is good for executable programs (guile, guild) <wingo>making sure that executables match up with libraries is tricky :) <wingo>use guile.m4 (make sure you get it from a recent guile) if you are using autoconf <dje42>I need to support cross-compilation scenarios which makes things a bit tricky to. What gdb does for python is allow the user to provide a private python-config program that provides the desired values. E.g. configure --with-python=/foo/bar/my-crosscompiled-python-config <wingo>pkg-config can also deal with cross-compilation, usually via a cross-pkg-config <dje42>Cool. I still have to support {cross-,}pkg-config not existing, but I can just make the user write a script that works the same. <wingo>you can also have the user set specific environment variables to get the needed information <wingo>in the case of gdb probably you need the info for the library more than the guile executable so pkg-config is probably the way to go <dje42>If I go with pkg-config, I'm going to need --with-guile and --with-guile-version, whereas if I go with guile-config I can just have --with-guile (without getting too hacky). <dje42>I'm not punting on pkg-config, just thinking things through. <wingo>if you use guile.m4 you can express version preferences <wingo>i.e. 2.0 first, then 1.8; or 2.0 only; or 2.2 first, then 2.0; etc <wingo>probably you should not automatically jump to 2.2 <dje42>I'm a bit afraid of some developer having to cope with some weird bug on some weird system and guile.m4 getting in the way. [It's easier to work around such things without the extra layer of abstraction.] <dje42>Maybe just unnecessary paranoia though. <dje42>[The kind of thing I'm thinking of is, e.g., the check for IRIX siginfo support in gdb/configure.ac for the python support.] <wingo>to be fair i don't think i had the bandwidth to review this patch in all the last 5 months <lloda>you did a ton of other work on Guile <wingo>yep, but it takes a special thing to be able to dedicate 30 hours to someone else's patch series :) <lloda>and you have my thanks for that! <wingo>heh, thanks (and apologies of course) to you too :) *wingo off to sleep, ttyl