<ArneBab>Would it be possible to use guile to create a webserver which uses GnuPG for encryption instead of SSL? <ijp>well, of course it would be possible <ijp>but are there any pgp-based transport protocols? <mark_weaver>ArneBab: out of curiosity, what's your plan on the client side? <ijp>SSL/TLS is at least understood and standardised <ArneBab>the plan on the client-side would be a specialized client, too. Just with a simple spec, so other clients and servers can be written as well. <mark_weaver>The centralized PKI system is fundamentally flawed, but that's distinct from the TLS protocol itself. <mark_weaver>Do you know of a problem with TLS itself that would motivate this change? <mark_weaver>If we change the way that browsers authenticate the TLS public keys, you can effectively change the PKI policy. <ArneBab>nothing except for the authentication - if TLS can switch to ECC <mark_weaver>ah, so you'd have to break compatibility with existing browsers anyway. <ArneBab>using GnuPG would just be a fun-project: testing it. <mark_weaver>well, in that case, I guess it's reasonable to switch to gnupg :) <mark_weaver>anyway, it should be quite trivial to implement this in Guile's web server and client. <ArneBab>the black hat conference reported, that RSA and similar algorithms are not too unlikely to be broken within 5 years. <ArneBab>but GnuPG has an upgrade path to ECC in version 2.1 <mark_weaver>ArneBab: what's the basic method for encapsulating an http session within gnupg messages? <ArneBab>mark_weaver: currently I’m just in the “that looks like a nice idea”-state. <ArneBab>I think the gnunet guys do something much more ambitious <ArneBab>I did not know that the talks were recorded - nice! (I tried to find them last weeks and failed…) <mark_weaver>though we still need to create an index.html, and link to it from the main pages in audio-video.gnu.org. <ArneBab>mark_weaver: I just dented and tweeted it around - I hope that was OK… <ArneBab>that’s fully OK for me: It works perfectly and with low maintenance overhead. <ArneBab>that’s the term for sending a notice from status.net/GNU social <mark_weaver>ah, I guess I should be embarrassed for not knowing that. I've never done any kind of micro-blogging (send nor receive), so I'm quite ignorant of them, proprietary and free alike. I'm glad you're on top of that stuff :) <mark_weaver>I should probably educate myself on this at some point. <ArneBab>no probs, not everyone has to run a newswire :) <ArneBab>and there wasn’t yet a release of the new GNU social code from the most recent statusnet <mark_weaver>I do feel quite passionately that we need free decentralized replacements for facebook, g+, twitter, etc. I spend quite a bit of time trying to convince my friends not to use them. <ArneBab>status.net/GNU social tackles twitter - and is quite good at that <ArneBab>also it has a twitter-bridge, which allows me to post exclusively on status.net but reach twitter-users, too. <mark_weaver>I was going to ask about that. I remember learning quite recently that pump.io is BSD, whereas status.net is AGPL, is that right? <ijp>is that all pump.io is? <davexunit>yeah, I think it's actually the Mozilla public license <davexunit>there's also diaspora for a more facebook-style social network. <ijp>a "client" pushing to various other services <davexunit>statusnet and pump.io are by the same author. <ijp>well, I can pop that one out of the toresearch list <ArneBab>yes: He decided to no longer write on the system which just needs a basic PHP server and to instead create a new, incompatible system which requires nodejs <davexunit>yeah there's a bit of controversy over the change to pump.io <ArneBab>He said that identi.ca grew to expensive (~7k€ per month) <ArneBab>and that pump.io delivers the same with 20% of the resources. <davexunit>the ease of development of PHP applications is attractive, but we'd be living in the past to not want to use better, more modern languages and frameworks. <mark_weaver>I can certainly sympathize with wanting to move away from PHP. <ArneBab>(though from the tests I read, he could also have switched to the HipHop PHP instead) <ArneBab>yes - that change killed pump.io for me <ijp>yech, pump.io does the bundling thing <ArneBab>AGPL ensures, that I will always be able to get equivalent features somewhere else - or that I can even host it myself. <mark_weaver>technical aspects can be fixed, but license problems are likely impossible to fix. <ijp>davexunit: including all the dependencies, because no-one uses a package manager <ijp>nobody, not even when it's the recommended method for installation in the readme <davexunit>ijp: oh really? I figured that with node applications you just specify what packages need to be downloaded via npm <ijp>davexunit: well, it has a colophon section where it lists various js packages as included <ijp>why those arent' packaged for npm, I don't know <davexunit>ijp: thanks again for your help with streams. I've decided that the reactive programming stuff is a bit too difficult for me to try to tackle now. <ijp>ported from haskell, it's insane <ijp>it's a stream of functions that operate on that stream <ijp>so if you actually run it, you will get (4 4 4 4) <ijp>because (stream-length s) would be 4, and so, (stream-car s) becomes 4, and then (stream-ref <> 1) becomes 4, and then (stream-ref <> 3) becomes 4 <davexunit>this will take a lot of studying to understand <ijp>davexunit: cut is just a shorthand for lambda <ijp>(cut list 1 2 3 <> 4 5 <> 7) -> (lambda (x y) (list 1 2 3 x 4 5 y 7)) <davexunit>I often here spreadsheets used as examples of reactive programming <ijp>it indicates an argument to the function <ijp>there is also <...> which can be used as a rest argument at the end of a cut form <ijp>it only goes one deep, so you can't do (cut list (list <>) 1) <ijp>Notation for Specialising Parameters without Currying <davexunit>I have heard of currying, but don't know what it actually is. <mark_weaver>davexunit: if you load the guile reference manual in emacs: i cut RET <ijp>currying is the idea that you can represent a function of N arguments with functions of one argument by nesting <ijp>(lambda (a b c) ....) -> (lambda (a) (lambda (b) (lambda (c) ...))) <ijp>it's how haskell and ML do multi-argument functions <ijp>it is distinct from, but often conflated with, partial application <davexunit>hmm I can't find my guile manual in the info buffer. <davexunit>I think I didn't install the docs on this machine. <davexunit>yup. found it. I forgot to install it on my desktop machine. <mark_weaver>knowing how to use the guile manual efficiently is very useful. <davexunit>time to confess: I've been reading it in my web browser <davexunit>time to learn how to use the emacs info reader <mark_weaver>although there's a slight learning curve, and it looks relatively primitive by today's standards, I think you will find that emacs info mode will save you a lot of time. <davexunit>wow this is great. "i cut RET" brought me right to what I wanted. <davexunit>I've been leaving the procedure index open in firefox and searching for the function I want. <mark_weaver>and of course, C-h i from anywhere in emacs will bring you to the *info* buffer. <davexunit>I'm consistently impressed with the help that this channel provides <mark_weaver>one of the first things I do when I set up a new debian system is to populate its info files with the manuals I care about. unfortunately, that's not as easy as it could be in Debian. some files have to be installed outside of dpkg. <mark_weaver>partly because of Debian's rejection of the GFDL invariant sections, but also I think because some of the relevant Debian developers apparently don't use info files. <mark_weaver>some of the gnu info manuals are in the nonfree section, and some aren't in any debian package. <ijp>davexunit: any manual with invariant sections is non-free according to the dfsg <ijp>there are other reasons, but that's a commonly cited one <ijp>dfsg is the debian free software guidelines <mark_weaver>well, docs that have some small bits that cannot be changed. <mark_weaver>e.g. no removing the GNU manifesto from the emacs manual. <ijp>"The idea of invariant sections is that they give you a way to express nontechnical personal opinions about the topic." <ijp>it's sort of a protection against being misrepresented <davexunit>yeah, like how some people license opinion pieces under CC-BY-ND <davexunit>it's free culture, but altering someone's personal opinion isn't right. *davexunit just finished watching the emacsy presentation by shanecelis <mark_weaver>yes, e.g. RMS grants only verbatim copying of his essays, not modified versions. <davexunit>I wasn't aware that RMS persuaded shanecelis to license emacsy as GPLv3+ <mark_weaver>however, RMS believes it's important for the licenses of reference works to allow distribution of modified versions. <mark_weaver>as he formulates it, the relevant difference is between "functional works" and "creative works". <mark_weaver>functional works include both software and reference works. <davexunit>Looking for some design opinions: A game can be built from many "scenes" that represent each piece of the game like the main menu, high score table, world map, battle screen, etc. <davexunit>So, I would like to model this with a <scene> record type. <davexunit>the part that I'm stumbling over is how to define a scene in such a way that encloses it's mutable state nicely. <davexunit>any scene can have some arbitrary amount of state: assets, player object, map, etc. <davexunit>a scene would have callback procedures that the game loop would call to draw the scene, update it, or when input happens. <davexunit>would this be a place where GOOPS would best solve my problems? <mark_weaver>GOOPS might be nice for this, though I don't know what problems you're trying to solve here. <davexunit>sorry, I'm having a hard time articulating my issues. <mark_weaver>even simple SRFI-9 records allow mutable state nicely. <ijp>how structured are scenes? <davexunit>mark_weaver: yes, they do. However, a main menu has different state variables that a battle screen. <davexunit>this is where GOOPS could come in by subclassing the <scene> class. <ijp>if the types of state is vary, it might be best to have a hashtable, or even module, in the scene record <mark_weaver>if you want to do inheritance, then GOOPS will be helpful. <davexunit>ijp: yeah I was thinking about using a hashtable as a generic place to hold all of state specific to a scene instance. <davexunit>but with the hashtable there's now another layer to work through to get at the data. <mark_weaver>though I acknowledge that there are probably cases where it's an appropriate technique. <davexunit>inheritance could work here, but I want to evaluate my other options. <davexunit>if I have a hashtable, it seems to me like I would be writing a really crappy vtable. <davexunit>so I might as well use GOOPS and inheritance. <ijp>davexunit: I'm thinking more as a "grabbag" for the scene specific kinds of state <mark_weaver>in some cases, you can put the procedures directly into the records. <ijp>but if you go with goops, then inheritance takes care of that <mark_weaver>so it depends whether you need to extra complexity of inheritance and GOOPS. <mark_weaver>I don't mean to suggest one way or the other. GOOPS is very nice. <davexunit>ijp: yes that's what I was thinking of doing. a scene *always* has properties like title, on-update, on-draw. but each scene has some extra "stuff" too. <davexunit>I was thinking of just keeping each scene in its own module with state as private variables. <davexunit>but I think it would be better to instantiate a scene when its needed and be able to instantiate the same scene many times. <davexunit>(define (make-a-scene) (let ((player (make-player))) (make-scene #:title "Foo" #:update (lambda () (update-player player))))) <mark_weaver>sure, closures are every bit as efficient as records. the only disadvantage is they're harder to inspect. <davexunit>that is true. I'm not currently writing unit tests, but that would certainly make it very hard to test a scene. <davexunit>I have learned from my experience in writing unit tests at work that trying to hide too much implementation complexity in private procedures leads to code that is very difficult to verify. <davexunit>mark_weaver: will you be attending the GNU 30th event at MIT? <davexunit>trying to find out if there will be any guilers about. <ijp>heh, Jeff Bezos is way at the bottom <ijp>can't beat the power law <davexunit>does anyone have any thoughts about how to write a REPL server that passes its compiled thunks off to another thread to execute? <davexunit>I'm working with an application that is pretty strictly single-threaded and runs an event loop of its own. <davexunit>and I want to allow for live coding in this environment that is safe and works as the user expects. some procedures only work in the context of the main thread. <ijp>well, if you provided your own client, then you could just wrap it there <ijp>if you wanted to do it with an arbitrary client and the normal repl server, you could define a langauge that merely wraps scheme, and tell your user to do ,L foo-lang <davexunit>ijp: I took the (system repl server) module and made my own modifications <ijp>of course, that one is kinda yucky <mark_weaver>I still like the lock-based method I outlined before, where the lock is almost always held by the main thread, but the main thread has a periodic job that involves unlocking the lock for a short time. <ijp>making your own, hmm, I'd need to look at the code <ijp>As an aside, I think we need to provide a way to stop individual servers <davexunit>mark_weaver: the issue now is that some code, particularly code that will call OpenGL or SDL functions will *not* work from this thread. <davexunit>the mutex lock was a great first step, though. <ijp>but then you need to find a way to get the individual servers id, which is problematic <davexunit>ijp: well, I was thinking that the server either knows how to get what it wants into a data structure that the other thread will pick up, or a hook will be run that will then do the same thing. <ijp>actually this may not be a big issue, if we can stop the server from a client... does ,q work? <davexunit>I don't think the server needs to be stopped. I already have the server and game loop interacting happily, sans the OpenGL issues. <davexunit>OpenGL contexts work on a per-thread basis, and SDL isn't thread-safe whatsoever. <davexunit>guile-2d already has a mechanism for scheduling thunks to be run in the event loop. <ijp>hmm, no, ,q just quits the client <mark_weaver>yeah, but you don't want to make those data structures thread safe, or else you'll add a lot of expensive locking. <davexunit>the way I've dealt with this thus far, is that either the game loop is running or the REPL is, because of the mutex. <mark_weaver>well, first off, I think we need a different kind of REPL hook, one that accepts one argument: a (usually compiled) thunk that will evaluate the user's expression. <mark_weaver>that thunk is already computed as part of what the REPL does. <davexunit>since, in the context of a game loop running at 60hz, it's rare that the game loop needs to pause for the REPL to work. <davexunit>so, as a before-eval-hook, I set a repl-waiting flag <davexunit>when set, the game loop unlocks the mutex and lets the REPL do its thing. <davexunit>so, with that in place, I think the scheduler need not be thread-safe. <mark_weaver>that's the idea, but I'm not sure the details are quite right. <mark_weaver>the repl-waiting flag is obviously shared between two threads. <mark_weaver>so you have to be careful about accessing outside of synchronization. <mark_weaver>but in general, you have to be very careful about writes being observed in a different order by another thread. <mark_weaver>for example, it would *not* be safe to use this approach if you set! a shared variable to point to a new structure it just allocated. <mark_weaver>because a thread might see the new pointer before it sees the writes that initialized the structure it points to. <mark_weaver>suppose you made it not just a flag, but a list containing the expression to be evaluated. <mark_weaver>then here's what wouldn't be safe: outside of a lock, you observe that this shared variable is no longer #f but instead a pair. then you proceed to read the CAR and CDR of the pair without locking... that's not safe. <mark_weaver>because even if the writer is careful to initialize the CAR and CDR before writing the pair's address to the shared variable, <mark_weaver>a reader in another thread might see the shared variable point to the pair *before* it sees the writes to the CAR and CDR, which it might see as garbage. <mark_weaver>welcome to the fun of weakly-ordered memory models... <davexunit>but as this is just a #t or #f value, I'm alright? <davexunit>I tried to take care to make sure this one simple flag would be all I needed. <mark_weaver>however, if you see a #t, make sure to lock before reading any other shared state. <mark_weaver>and of course you'll need more shared state: a way to pass the compiled thunk to the main thread, and a way to pass back the results for printing on the REPL. <davexunit>if the game loop sees a #t, it unlocks, sleeps a bit, and then locks again. <mark_weaver>those other two pieces of state should be protected by the mutex. <davexunit>the REPL will lock the mutex in the before-eval hook, and unlock in the after-eval hook. <mark_weaver>so, the REPL should grab the lock *before* setting that variable. <mark_weaver>how are you going to arrange to run the code in the main thread, using only before and after hooks? *mark_weaver looks at the REPL code. <davexunit>I won't be. that's the part I don't know how to implement yet. <davexunit>I thought you were saying something else wasn't good enough. <davexunit>but yeah, before/after hooks won't get the thunk into the main thread, of course. <davexunit>I tried to modify the REPL server code, but I could only create a REPL that crashed when you tried to eval something. :( <mark_weaver>well, 'repl-prepare-eval-thunk' and 'repl-eval', in system/repl/common.scm, are the relevant bits. <mark_weaver>so I guess 'repl-eval' is the thing to change. instead of doing (% (thunk)) right here, pass (lambda () (% (thunk))) to some user-definable hook. the default would be to simply call that thunk. <mark_weaver>in your case, you could put that thunk (lambda () (% (thunk))) into a shared data structure (first locking the mutex, since that protects the shared variable) <mark_weaver>and then unlocking, and then waiting for the results to be available, which would require an additional sychronization step. <mark_weaver>A condition variables might make sense to wait for the results. <davexunit>I typed % into my REPL and got an undefined variable error <davexunit>okay, so repl-eval returns a thunk. that thunk gets placed into the scheduler <mark_weaver>repl-eval doesn't return a thunk. it actually runs it. <davexunit>well the modification you suggest is to return a thunk that runs it, right? <davexunit>rather than just executing it as it currently does. <mark_weaver>no, I don't think you should change the 'repl-eval' interface. <mark_weaver>I think you should change 'repl-eval' so that it can optional do something else with 'thunk'. <mark_weaver>specifically, if a suitable repl option is set to some procedure P, then it should do (P thunk) instead of (% (thunk)) <davexunit>ah, so we could add a third, optional, argument with a default value of % <mark_weaver>'repl-eval' already takes an argument 'repl' that contains repl options. <mark_weaver>the customization should be stored in that 'repl' object. <mark_weaver>I haven't looked deeply at that code, but that's the basic idea. <davexunit>so repl-prepare-eval-thunk will generates the thunk, and the thunk that is created can vary depending upon REPL options. <mark_weaver>see 'repl-default-options' in system/repl/common.scm <mark_weaver>not quite. I don't think you should change what 'repl-prepare-eval-thunk' does. <mark_weaver>the reason is that 'repl-eval' takes care to run (thunk) within a prompt. <mark_weaver>and you'll want that prompt to be run in the same thread that actually runs the code, presumably. <davexunit>okay. I'm all sorts of wrong. thanks for your continued patience. <mark_weaver>also, in general, it's best not to change the semantics of these procedures, if it can be avoided. <mark_weaver>the semantics of 'repl-prepare-eval-thunk' is to compile a thunk that, when run, will evaluate the expression the user typed. <mark_weaver>the semantics of 'repl-eval' is that it should compute and run that thunk. <mark_weaver>therefore, I think 'repl-eval' is where the hook should be applied. <mark_weaver>so I would add a new option to 'repl-default-options', where the default value is (lambda (thunk) (% (thunk))) <mark_weaver>and then, in 'repl-eval', get that procedure using something like (repl-option-ref repl 'run-thunk) <mark_weaver>and then replace the (% (thunk)) with (run-thunk thunk) <mark_weaver>where 'run-thunk' is bound to the result of (repl-option-ref repl 'run-thunk) <mark_weaver>in your case, you'd pass 'thunk' to the main thread, and then the main thread would arrange to do (% (thunk)) <davexunit>so that will allow the thunk to be scheduled for execution in the main thread. <davexunit>the REPL will need to block and wait for the results to come back, and that's where the condition variable comes in? <mark_weaver>now, your custom 'run-thunk' will need to wait for the result values to come back from the main thread, and only then return. <mark_weaver>I'm a bit fuzzy on the best way to do all this synchronization. <davexunit>well you've gotten nearly all of the way there. <davexunit>I can work in something primitive that will handle the synchronization and see how it goes. it can be improved later. <davexunit>once this problem is solved, I shouldn't have to worry about threading in guile-2d ever again. :) <mark_weaver>btw, does the "repl-waiting" flag really only have one writer? who clears that flag? <mark_weaver>I might be missing some more clever way to do this, but the straightward approach using condition variables is to simply have one condition variable and one mutex per direction of information flow. <davexunit>the REPL sets it in before-eval and then clears it in after-eval. <davexunit>it's cleared *before* the mutex is unlocked. <mark_weaver>well, that's suboptimal because the main thread might then see the flag set again, before the REPL thread has had a chance to clear it. <mark_weaver>ideally, after evaluting the thunk, the main thread shouldn't have to wait for the REPL thread to wake up. <mark_weaver>so, here's a straightfoward condition-variable-based approach: <davexunit>well, it works fine now, but when I implement the changes you've suggested it will be a problem. <mark_weaver>basically, after putting the thunk in a shared variable, the REPL thread should signal a condition variable to tells the main thread that the REPL is waiting. <mark_weaver>the main thread periodically waits on that condition variable with zero timeout. <mark_weaver>well, I'm rushing this. please give me some time to think on this a bit more carefully. <davexunit>sure. I can hack on this for some time and see where I get. <davexunit>threading is such a pain, but providing a REPL is so important. <davexunit>if for no other reason than I'm tired of the write-run-kill-loop I'm in now. <mark_weaver>we should probably think about how to make this kind of thing easier. <mark_weaver>for starters, we really need some better synchronization primitives. <mark_weaver>with the right primitives, this kind of thing is trivial. <ijp>Simon Marlow, a lead GHC developer, has a new concurrency book out <mark_weaver>things like that would be trivial to implement in Guile, but we just need to craft some nice APIs and libraries and probably bundle them with core guile. <ijp>actually, how is your TODO list currently, mark? <mark_weaver>I'm just overwhelmed with the size of my TODO. We really need more help. <ijp>do you think it would make sense to have it on the website, or in the repo? <mark_weaver>I should probably post my current one. or better yet, convert them into a bunch of wishlist bug reports. <ijp>it would be rather more useful than the Ideas page <davexunit>I think Guile is a great place for people with the proper domain knowledge to contribute. now if only my feelings could translate into a wider user base... <mark_weaver>I think the bug tracker might be the right thing for this. <ijp>yeah, we do already have a few feature request reports <mark_weaver>davexunit: so this would be trivial with MVars. You have one MVar to pass thunks to the main thread, and another MVar to pass the list of result values back to the REPL thread. <mark_weaver>and it's not hard to implement MVars using the lower-level primitives we have in Scheme. *mark_weaver looks at the Haskell MVar spec *davexunit encourages mark_weaver <nalaginrut>BTW, what about adding let-values or define-values? <mark_weaver>we already have let-values. I agree that define-values would be nice, and I plan to do it in master at some point. <davexunit>hmmm. I made my own copies of all of the repl modules that I haven't yet copied and replaced "system repl" with my namespace "2d repl" and now the REPL throws an error every time I try to eval. <davexunit>sorry. I make my own copies of the (system repl ...) modules <mark_weaver>I don't know what you mean by: replaced "system repl" with my namespace "2d repl" <davexunit>so I could start to hack on the ones I needed. <davexunit>changed (define-module (system repl ...)) to (define-module (2d repl ...)) <mark_weaver>I think you're making this harder than it needs to be. <davexunit>but I figured that I needed to copy the modules since I'm going to be modifying them to suit my own purposes. <mark_weaver>I intend for this new repl option to go into guile stable-2.0 <mark_weaver>so why don't you just add this capability to guile, and maybe send in the patches? :) <mark_weaver>in the meantime, you can have it in your private tree. <davexunit>I was trying to do the "in the meantime..." part by copying the entire system/repl directory. <mark_weaver>well, the other solution is to simply redefine those procedures at run-time within a guile session. <mark_weaver>just like in emacs when you want to change something. <mark_weaver>(well, rerun "make" that rarely takes more than a few seconds) <davexunit>I already had 2 of the repl modules copied in my private tree. <davexunit>but since i was changing common.scm, I wanted to copy all of the modules. <mark_weaver>make copies in a private directory that's in your GUILE_LOAD_PATH, but leave the names the same, and make your changes. <mark_weaver>and make sure to set the GUILE_LOAD_PATH before running Guile, so that your modified REPL code gets loaded instead of the core version. <mark_weaver>and you should only need to copy that one file system/repl/common.scm <mark_weaver>that's the only one that needs modifications, I think. <mark_weaver>well, the easy way to set the load path is using the -L option to guile. <davexunit>I have to maintain the directory structure right? ./system/repl/common.scm <mark_weaver>right, so if you run guile -L $FOO, then $FOO/system/repl/common.scm should exist <davexunit>quick question: I see that a REPL option has 3 elements. The name and 2 others. <davexunit>it looks like the the 2nd element the default value. what is the third? <mark_weaver>when the user sets the option, the value they're trying to set is passed to the converter, which validates it and returns the value that will actually be stored as the option value. <mark_weaver>for example, look at the converter for the 'prompt' option. <davexunit>some converters are #f, does that mean that no validation is done? <mark_weaver>it takes the value that the user gave, which could be #f or a string or a thunk or a more general procedure. <mark_weaver>yes, see the detailed logic in 'repl-option-set!', in the same file. <mark_weaver>if the third item of the list isn't a procedure, then there's no conversion: the value provided by the user is set directly. <mark_weaver>davexunit: do you know how to look up tags in emacs? <mark_weaver>I confess that I still haven't been fully geiser-ized yet, so I may not be the best person to explain it. I've been relying on the non-geiser tag lookup in emacs. <mark_weaver>but basically, when things are set up, you can just type M-. with the cursor on a global identifier, and it takes you right to the source code for that global identifier. <mark_weaver>and then there's M-* to "pop the stack", i.e. go back to what you were looking at before M-. <mark_weaver>(and quite often, this process of looking things up recurses) <mark_weaver>knowing how to use tags properly makes a *huge* difference in being able to figure out how any piece of Guile works relatively quickly. <davexunit>very cool. I opened a geiser REPL and now M-. is bound to geiser-edit-symbol-at-point <mark_weaver>so if you build guile from source, then you can just do "make tags" in the build directory. <mark_weaver>and ask it to load the main tags table in the top-level build directory. <mark_weaver>I think you can just add an entry to 'repl-default-options' <mark_weaver>look in your chat history here. I outlined how to do it. <nalaginrut>after read wingo 's 2.2 TODO and AOT plan, I can't wait to use Guile3.0 <davexunit>mark_weaver: sorry, I guess I'm not being clear. I also changed repl-eval to use that REPL option. <mark_weaver>davexunit: to change the option, type ",option run-thunk <procedure>" <davexunit>now, to do this every time I use the REPL server, I could use repl-default-option-set! <davexunit>I'm looking for where I have the REPL object so that I can. <mark_weaver>maybe it should take an additional keyword argument to allow other customizations other than 'lang' and 'debug' <mark_weaver>or course, that's inconvenient, because (system repl server) won't pass any arguments to 'start-repl'. maybe more plumbing is needed here (which should be as generic as possible).. <mark_weaver>of course, 'parameters' (dynamically scoped variables) could be used to avoid the plumbing. <mark_weaver>I have a strong aversion to using parameters, but sometimes there's no other sane solution. <mark_weaver>and they are certainly reasonable to use in the short term. <davexunit>I'm just hacking the call right into start-repl for now. <mark_weaver>probably the simplest expedient hack without breaking other repls is to use a parameter. <mark_weaver>for the record, I think the right solution here is to add keyword parameters to both 'start-repl' and the procedures in (system repl server), to accept a list of repl options to set! before running the repl. <nalaginrut>well, it's so big change to rip out GLIL/assembly/bytecode/objcode, things gonna change for evolution ;-) <mark_weaver>but of course it must be able to run all code in guile before we make it the default compiler. <ijp>is there any way to skip a test in check-guile <ijp>anyway, I'm getting the occasional hang on threads.test, but otherwise the rest seem to pass on master <nalaginrut>well, I haven't tried master, I have to stick to 2.0.9 for Artanis at close future <ijp>I did an strace while compiling psyntax because it seemed to be taking forever, and it seems to be mostly FUTEX operations <ijp>I'm assuming that is gc related <ijp>near the end, there are also a number of one character writes, which is quite funny <ijp>mark_weaver: someone posted a problem on #scheme that might interest you <wingo>heya late-night mark_weaver :) <mark_weaver>I just posted a proposed implementation of Haskell-style MVars for Guile. *mark_weaver is sick of having no modern concurrency primitives <wingo>good, up before the dawn this morning to haul brush off to the dump <wingo>settling down to prepare a talk on minimalt <wingo>did you see that paper when it came out? pretty interesting stuff <wingo>cr.yp.to/tcpip/minimalt-20130522.pdf <mark_weaver>I haven't seen it, and indeed I'm interested, thanks :) <wingo>they are still closed atm, though they claim they'll open up soon <wingo>currently targeted to run on "ethos", a new OS that runs on xen <wingo>anyway they have a posix layer too, so i'm going to try to work off of that <wingo>would be fun to reimplement, but i have enough of those projects ;) <ijp>the tag line on that site is kinda long <ijp>"An operating system to make it easier to write applications that withstand attack" <wingo>mark_weaver: is that the entirety of the mvar interface? <wingo>we should certainly add it if it's just that; i'm just surprised at how short it is <mark_weaver>I didn't implement the last four primivites listed on that page, namely modifyMVarMasked (with and without trailing underscore), mkWeakMVar, or addMVarFinalizer. <wingo>mark_weaver: since they are still a low-level interface perhaps they could just go in (ice-9 threads) <wingo>i don't have any strong opinions there <mark_weaver>also, they have a dedicated "newEmptyMVar", whereas I just use case-lambda with a single 'new-mvar' procedure. <mark_weaver>if they become popular, I could imagine them getting replaced with a faster implementation. <wingo>i wonder if it makes sense to implement our own mutexes in scheme <mark_weaver>I would think they should be as fast as we can make them. <ijp>which is always a danger with Haskell <mark_weaver>I think it's worth having some simple commonly-used sychronization primitives be implemented in C or even as VM instructions. <wingo>i don't know what that set of primitives is though <mark_weaver>Yeah, neither do I. There are so many to choose from :) <ijp>it's even worse than delimited continuations <mark_weaver>Do other people here have preferred sychronization primitives that they wish were in Guile? <ijp>sychronization? not really <mark_weaver>The other system I want to implement at some point is Software Transactional Memory (STM), even though I think that shared memory is basically the wrong approach. <ijp>I remember learning to use semaphores, but I can't really say I miss using them <mark_weaver>but for those who want to have multiple threads mutating a shared state (of which there are many), STM is the only sane system I know of that has good properties. <taylanub>Apple's Objective-C frameworks have condition variables that double as mutexes, since one always uses a condvar with a mutex. Not sure if that's a significant improvement. <wingo>yes stm sounds good, but it's not exactly a primitive... <mark_weaver>well, it's primitive in the sense that it makes sense to support it at a low level, and in fact the Intel Haswell architecture will have hardware support for it. <wingo>so, for me one question is whether to assume a platform's mutexes are good enough, or whether to implement our own <wingo>as with java's biased locking, etc <wingo>i would look at futexes to see if they use more interesting primitives <mark_weaver>it would certainly be good to have much lighter primitives available. <ijp>that reminds me, earlier I mentioned a lot of futex syscalls while compiling psyntax <ijp>am I right in saying that's to do with libgc? <wingo>ijp: that's probably when making gensyms <ijp>ah right, those have a global counter <wingo>that problem is fixable if we have gensyms create their representation when they are printed instead of when they are created <wingo>since most are used just for identity... <ijp>didn't either you or mark_weaver have patches for that already? <ijp>I assume this means it is on the giant todo <mark_weaver>however, I ran into some nasty complications, due to the weird nature of the symbol hash table. <mark_weaver>the problem is that the symbol table uses the _symbol_ itself as the key. <wingo>its hash is the hash of the string <mark_weaver>the mapping which I'd expect to be from a string to a symbol, is instead from symbol to symbol. <wingo>that would make sense, though a weak set can consume less memory <mark_weaver>I don't remember the details, but I remember this made the locking quite difficult. <mark_weaver>my patch also made a change that is thread-unsafe, I think. It was before I educated myself about modern weak memory models. <mark_weaver>wingo: how does it map merely to "its existence"? how does one find the canonical interned symbol from some possibly redundant one that was constructed solely to do the symbol lookup? <wingo>mark_weaver: see symbols.c:104 in master <mark_weaver>oh, I guess the lookup matches by checking the backing string, and then returns the pair so that the canonical key is recovered. <wingo>note that weak sets have a mutex in them <mark_weaver>for lazy gensyms, iirc, we need something that will atomically try to intern a symbol, and then report whether the symbol was newly added. <wingo>i.e. loop in scm_i_str2symbol if the symbol you get back isn't the same one you tried to intern <mark_weaver>I seem to remember that there was a complication, but I'm forgetting what it is off hand. <mark_weaver>it had to do with the fact that for lazy gensyms, the symbol object itself already exists. <mark_weaver>and initially, the symbol object is marked as "not yet named". <mark_weaver>now, I mustn't mark it "named" until I know what the final name will be. <mark_weaver>but in order to intern it, I need to modify the object to include a string that I want to try. <mark_weaver>now that opens a window where other threads will see that the symbol has a name.. but it might not be the name it ends up with. <mark_weaver>in fact, at that point, if the name is wrong, then that means it's a duplicate symbol with the same name. <mark_weaver>I need to be able to try-to-intern the symbol under a proposed name before I mutate the symbol at all. <wingo>because string->symbol at the same time with the name that it got could see a symbol flagged as not-yet-named <wingo>perhaps that's not an issue though <mark_weaver>symbols flagged as not-yet-named shouldn't be a problem. <mark_weaver>lazy gensyms mean that any symbol could be flagged that way. <mark_weaver>the key is that whenever you ask for a symbol's name, then it has to transparently find a name for it, in a thread-safe way. <mark_weaver>I think it's not a problem, but not for the reason I just gave. <mark_weaver>if 'string->symbol' returns a symbol that's marked 'not-yet-named', that could only happen because the 'maybe-intern' operation succeeded. <mark_weaver>which means that even though it hasn't happened yet, the name of the symbol is guaranteed to match the name it was interned as. <mark_weaver>what's needed is a way to atomically try to add a mapping from string->symbol in the symbol table, where we provide the symbol. <mark_weaver>such that if the string is already interned, we get the real interned symbol. <mark_weaver>but if it hasn't already been interned, the symbol object that we passed in is used. <Arne`>shanecelis: I could test a leapmotion last weak, and that’s really cool! How did you get guile working with it? <ijp>Arne`: he's not on, fwiw <Arne`>damn, thanks! (I did not check that before writing…) <ijp>the way I tell is when emacs doesn't complete the nick <Arne`>auto-complete completed anyway… <Arne`>maybe it’s set too aggressive… <mark_weaver>wingo: I don't think this will work though, because the 'intern' process assumes that the string value is already in the symbol object. <ijp>Arne`: and when that fails, if erc-hl-nicks highlights or not :) <mark_weaver>wingo: but we need to be able to try-to-intern a symbol that's still marked "not yet named". <wingo>there are intern variants that take the name and hash explicitly <wingo>see e.g. scm_from_latin1_symboln <wingo>so it shouldn't be a problem to have a symbol whose hash and contents are taken from somewhere else <ijp>Arne`: it's important to have redundancy. <ijp>Arne`: it's important to have redundancy. <Arne`>woah, and the site went down… <mark_weaver>scm_from_latin1_symboln doesn't do the job atomically. *Arne` ’s site is up again, no clue what happened. <wingo>mark_weaver: the key thing is that scm_str2symbol is atomic; the initial lookup is just optimistic <wingo>if it succeeds, then we avoid making a string object, which is a pretty nice win <mark_weaver>well, just to rehash more clearly, we need an atomic operation that is given PROPOSED-NAME and NOT-YET-NAMED-GENSYM, and either interns NOT-YET-NAMED-GENSYM under PROPOSED-NAME (if it wasn't yet interned), or indicates failure. <mark_weaver>sorry, I'm having trouble thinking clearly right now, I probably need sleep :) <mark_weaver>the piece I still don't get is how to call 'scm_c_weak_set_add_x' in such a way that it won't look in the symbol itself for the name to compare. <mark_weaver>if the key is the symbol to be entered, then somehow we need to give the predicate access to more information than just the key. <mark_weaver>I need to study the relevant code in master. before I looked only at stable-2.0. *mark_weaver just discovered 'rob_from_rich' :) <mark_weaver>okay, I guess I don't see any remaining problems with implementing lazy gensyms for master. <mark_weaver>but it'll be hard to tell until I actually try to do it. <wingo>btw a review of weak-set.c and weak-table.c would be really nice if you ever get time for it <wingo>i've looked at it a lot but i still might have made a mistake <mark_weaver>right now, what I really need to do ASAP is to figure out how to make psyntax deterministic w.r.t. generated marks. <mark_weaver>without creating unintended mark collisions, which would be bad. <mark_weaver>one idea in the back of my head is that it might be helpful to dynamically map them to fresh gensyms when a .go file is loaded. <mark_weaver>but it needs more thought, and it might be tricky to get it right. <mark_weaver>(and I'll take the opportunity to think about source tracking for immediates) *wingo tries to force guile out of active memory, with minimalt :) <mark_weaver>which I guess should be arranged by having a variable of 'read' that returns syntax objects. <mark_weaver>things like MinimaLT, GNUnet, and deterministic builds are all good things to be working on these days... <ijp>nalaginrut: heh yeah, davexunit posted that earlier <ijp>yeuch, a copy paste error and I got a page telling me how to fix the site's xmlcon fig ***sneek_ is now known as sneek
<nalaginrut>well, I'll have an interview with RMS if he is lucky to get the position ;-/ <taylanub>It's saddening that significantly more people know of Torvalds than Stallman. <ijp>taylanub: I don't think that's the right interpretation of those results <taylanub>E.g. I know that in my workplace, there's 5-7 people who'd know who Torvalds is, and would get the joke and vote for him on that page, but they mostly have no idea who Stallman is. People tend to not even know the name "GNU". :\\ <davexunit>mark_weaver: your mvars module looks very cool. <Arne`>taylanub: Linus somehow managed to look successful, while in my understanding, the movement around copyleft software mostly gets driven by RMS <Arne`>But Linus fits the image of the successful nerd, who is non-threatening, because he actually acts like any other businessperson, while Stallman is a threat, because he represents a change in society. <Arne`>which is why Linus is not that interesting to me… <Arne`>Besides: Can guile modules be distributed *easily* with guix? <Arne`>taylanub: I have another question on guile PR: Is there an explicit mission statement and slogan? <Arne`>If yes, I’d like to include them in my project-communication article… <taylanub>I don't know, but I suppose the mission statement would be, roughly, to bring the Emacs experience to the rest of GNU, or in more dry terms, to create an embeddable and extensible Scheme-centric run-time to glue together as many GNU applications as possible. <Arne`>So emacsy would be the ideal representation of guiles mission? <Arne`>taylanub: What do you mean by glue together? <taylanub>From what I get, Emacsy is mostly for bringing an Emacs-like UI to an application ? <taylanub>I think "Emacs experience" mostly refers to Elisp, and not the UI. <Arne`>So being able to adapt every program to my needs? <Arne`>Essentially being able to extend any program as user? <taylanub>Yeah .. if all applications made their main functionality available through Guile, one could use any UI they want, write scripts that rely on and extend that functionality, etc. <Arne`>Ah, so all programs could provide an API which guile can access? <taylanub>A highly flexible, customizable, extensible, dynamic environment. <taylanub>That's the ideal, to have all applications provide some data types and SUBRs (Scheme procedures defined in C) that offer an API that opens up the functionality of the application to everything. <Arne`>that sounds pretty cool - it’s actually more than just offering an extension language. <davexunit>and not only that, but whole applications can be written in Guile. :) <davexunit>which makes for an even more extensible program. <Arne`>Actually that is what I would prefer using guile for <taylanub>Yes, as Guile Scheme itself gets highly efficient, and modern hardware very performant, one can even write some critical code directly in Scheme. <Arne`>Many programs nowadays have no performance critical code anymore… <davexunit>I'm writing a hackable 2D game development library completely in Guile. <Arne`>davexunit: is that guile-2d? <davexunit>(with maybe some C for the performance critical bits) <Arne`>I’m looking forwart to testing that! The description (similar to pyglet) sounds like it could be exactly what I need. <Arne`>though I hope that you’ll manage to build it without needing to resort to C: That makes deployment much easier. <Arne`>“here’s guile, and here’s the game” <davexunit>but you could also ship the .so if you really needed? I depend on other libraries that also need to be compiled and installed. <Arne`>did you see how pyglet managed to get that fast in pure python? <Arne`>shipping an .so is a huge hassle compared to just shipping scripts <davexunit>but it has performance issues in roughly the same areas that guile-2d does <davexunit>such as rendering a large number of sprites that are moved/rotated/scaled each frame. <Arne`>it’s fast enough for the games I experimented with (which are slightly retro, though) <Arne`>Any game I write will have to be shipped to GNU, MacOSX and Windows… <davexunit>guile-2d requires a >= 2.0 version of guile. <Arne`>me neither, but that’s only a matter of time, I think