IRC channel logs

2013-12-23.log

back to list of logs

<linas>hi tylanub, my users don't have access to the standard guile repl ...
<linas>they have a custom repl inside the app.
<linas>thanks though... I'm trying to figure out how to encourage users, and one way is better/easier docs
<mark_weaver>module/system/repl/command.scm is the relevant code
<mark_weaver>specifically 'meta-command'
<mark_weaver>I'm not sure this is considered public API, though.
<linas>ah. Well .. if it breaks ... not like I have hordes of users to support.
<linas>most of them want to do everything in python, even though, for this particular app, python is verbose/inefficient
<linas>cest la vie
<mark_weaver>I wonder if it might be better to hook in at the same level that readline does, so you're just passing a string to the REPL.
<linas>?
<linas>my apps don't use the guile repl; they're servers of variious sorts
<linas>you've got to telnet/ssh into them to get a human-accessible prompt
<mark_weaver>are you familiar with (system repl server) ?
<linas>never heard of it :-)
<mark_weaver>that *is* a public API and might do exact what you need.
<mark_weaver>*exactly
<mark_weaver>see "REPL Servers" in the manual.
<linas>found it .. pondering
<mark_weaver>also, keep in mind if you ever need to run in the debugger
<linas>well, the app isn't written in guile
<mark_weaver>well, I mean what happens if the user enters an expression at your REPL and it causes an error. it would be nice to be able to enter the debugger and investigate, no?
<linas>the app is about 50% c++ and 20% scheme and 10% python, and its got a built-in repl thingy
<linas>yes.
<linas>Hm
<linas>I see
<mark_weaver>more generally, there is benefit to having just a single REPL that is nicely tuned and supported, rather than reimplementing one from scratch.
<linas>yes, I agree ...
<linas>I'm working with 10 year-old code that I suspect predates the REPL server
<mark_weaver>*nod*
<mark_weaver>one possible issue is that (system repl server) puts REPLs in their own thread, which means potential thread synchronization issues.
<linas>I think that's fine. I've long wanted guile to be propely multi-threaded .. in the 1.8.x days, it didn't work very well, maybe its better now
<linas>it works!
<linas>(spawn-server (make-tcp-server-socket #:port 18001))
<linas>excellent!
<linas>Yeah, I'd opened a bunch of threading bugs against guile years ago; never quite got around to trying again.
<linas>the app is btw opencog which has its own channel # opencog here on freenode
<mark_weaver>ah, nice! I'm aware of opencog, very exciting work!
<mark_weaver>note that in guile-2.0, ports are not thread-safe. you must ensure that I/O operations for a given port are done by only one thead at a time.
<mark_weaver>(on the master branch, which will become guile-2.2, ports are thread-safe)
<linas>I doubt I'd have to use the same port, multi-threaded.
<linas>Mostly just need to manipulate gazzillions of .. err "atoms" with relatively unsophisticated scheme methods.
<nalaginrut>morning guilers~
<b4283>morning
<nalaginrut>heya
*nalaginrut love define-values ;-)
<dsmith>hey hey
<taylanub>If I got it right, this temporary implementation of `define-values' uses mutation so inhibits some optimizations, but once the proper implementation lands I wonder if one could simply use that for implementing "truly private top-levels" and "immutable modules" as I had in mind http://lists.gnu.org/archive/html/guile-devel/2013-10/msg00046.html
<mark_weaver>taylanub: yes, that's exactly right.
<mark_weaver>taylanub: oh sorry, I didn't read what you wrote carefully enough.
<mark_weaver>taylanub: I don't know about "truly private top-levels" and "immutable modules". I'm sympathetic to what you're trying for, but we have to consider backward compatibility issues and longstanding history, etc.
<mark_weaver>'module-variable' returns a variable object. the assumption of mutability is baked into the API.
<taylanub>Right, some extra machinery is needed in any case for "immutable modules", though private top-levels could be done with `define-values'.
<taylanub>I imagine we could pass flags to `define-module' for creating new types of modules, maybe changing the default in some big, compatibility-breaking update. (Arguably, reading private bindings of a module, and mutating exported bindings of a module, are both things that should be considered bad/broken code anyway...)
<mark_weaver>if we were to do something like this, I think perhaps the right way would be to actually expand this new kind of "immutable mutable" into a single top-level form, containing a closure with lexical variables.
<mark_weaver>but the last time I brought up the idea of deprecating '@@', wingo squashed the idea pretty hard.
<taylanub>mark_weaver: Do you mean here http://lists.gnu.org/archive/html/guile-devel/2009-09/msg00023.html ?
<mark_weaver>yeah
<taylanub>From what I see in Andy's reply, the base point is that it's neat to *have* mutable modules with openly visible private bindings (perhaps even as default), but having the choice of alternatives isn't bad either. And there's even explicit consideration/endorsement of alternatives via compiler flags...
<mark_weaver>I have to go offline for a while. ttyl!
<taylanub>Laters :)
<taylanub>.oO( Oh, I forgot that truly private top-levels are only useful in immutable modules; otherwise one can always add a procedure to the module that calls set! on a private top-level that previously was never set!, breaking any assumptions an optimizer could make. )
<taylanub>.oO( Well, that problem is actually lifted by making "truly private top-levels" to be *not* top-levels, and instead locals only visible during the initial definition of the module, e.g. with (define-values (...) (let (private ...) (values ...))). )
<taylanub>.oO( Which in turn makes it quite senseless to add new procedures to a module, since any new procedures won't be able to reference these fake "private top-levels" of the module, i.e. the new procedures gain nothing from being part of the module and could as well had been in another module; thus our module could as well had been immutable... )
<tupi>hello guilers!
<dsmith>tupi, Hey
<taylanub>I wonder what the optimal way of implementing immutable modules would be. Implementing immutable variable objects ? Disallowing getting the variable objects in the module (instead only the values directly) ?
<davexunit>immutable modules :(
<taylanub>davexunit: ?
<davexunit>I don't see how immutable modules would even work.
<davexunit>and I imagine that having them be mutable makes things like Geiser's module awareness work.
<taylanub>davexunit: I detailed all my thoughts in http://taylan.uni.cx:8080/doc/guile-optimize-2.txt , and additionally just wrote http://taylan.uni.cx:8080/doc/guile-optimize-3.txt .
<davexunit>links don't work :(
<taylanub>oh ..
<taylanub> http://taylanub.github.io/doc/guile-optimize-2.txt and http://taylanub.github.io/doc/guile-optimize-3.txt
<ArneBab_>taylanub: Can you give a bit more motivation, why modules should be immutable by default?
<ArneBab_>davexunit: backlog says on immutable modules: http://lists.gnu.org/archive/html/guile-devel/2013-10/msg00046.html
<taylanub>ArneBab_: So e.g. a top-level (define (loop) (blah) (loop)) can be compiled to a fast loop if the module itself doesn't mutate it.
<taylanub>Essentially, to make it possible to statically analyse a whole module and be able to tell what top-levels will conform to what invariants etc.
<taylanub>And one can still swap out a module as a whole for a new full version of it, so it doesn't really make the Guile run-time as a whole less dynamic, just more painful to redefine a part of a module.
<taylanub>During debugging it would be disabled so you can continue to use C-M-x in Emacs and such things.
<taylanub>And a next step could be to allow static linking of modules, for which they need be immutable, so we could finally e.g. static-import the (guile) module and have calls to car, cdr, +, -, vector-ref, etc. be inlined. (Many of them already are, actually, but it's technically a bug that they are; you get no error when redefining them yet a procedure you compiled uses their old definition...)
<ArneBab_>taylanub: that sounds like changing the programming flexibility for the sake of optimization. And having vastly different coding-time and run-time behaviour sounds like a bug-hive…
<ArneBab_>taylanub: would it be possibly to somehow freeze a part of a module, but not all of it?
<ArneBab_>taylanub: that way, performance critical parts could be frozen while the rest stays easy to change
<ArneBab_>taylanub: maybe even keeping the same programming model, but requiring a recompile of all frozen parts when they get changed
<taylanub>ArneBab_: Yeah, moving top-levels to inside the lexical scope of something else does it. E.g. for the loop example, (define (loop) (define (actual-loop) (blah) (actual-loop)) (actual-loop))
<taylanub>But the way I'm envisioning things, there will really just be two `define-module' flags: 1) "use-define-values" will turn the module body (which consists of several top-level forms so it's a bit tricky) into one `define-values' call, putting all non-exported bindings into a lexical scope (thus truly private/invisible and not referencable or mutable through the module API), and 2) "make-immutable-module" will flag the module as a whole
<taylanub>(and/or each of its bindings) as being immutable, which basically gives way to an optimizer making 100% sure that certain bindings are never mutated.
<taylanub>And that logic is already present for doing the same analysis on lexical scopes, if I'm not mistaken.
<ArneBab_>ah, ok, so it would just be syntactic sugar to simplify writing that
<taylanub>`define-values' will be added soon apparently, so re. making non-exported top-levels really private, yes, but currently we have no machinery for making a module or an individual binding immutable.
<ArneBab_>from what I gather, this could regain full mutability by creating a module-set! which copies and rebinds the module
<taylanub>Hence I was pondering on what the best way to add that would be. I think marking a module as a whole as being immutable should do it; then the only over-head would be within calls to module-mutating procedures, which *should* be a rare occurrence, I think.
<taylanub>ArneBab_: Possible ...
<taylanub>No wait, how exactly do you mean ?
<ArneBab_>so the effect would be a different performance behavior: Mutating a module would be slower (because it would be a full copy) while anything else could be faster.
<taylanub>A module that was compiled as being immutable might have had some optimizations being done to its code which relied on the assumptions that the bindings won't be mutated, like removing type-check.s
<taylanub>checks.*
<taylanub>So even for lifting immutability, a recompilation is due...
<ArneBab_>taylanub: I mean, that the code (not the compiled version) would be copied.
<ArneBab_>yes
<taylanub>Ah OK
<ArneBab_>recompile on mutation
<ArneBab_>(and with a cache for the recompiles, setting an already existant value could just reuse the old compile)
<ArneBab_>(which would be close to a tracing compiler ☺)
<taylanub>By the way, are you aware of any use-cases of module-mutation other than 1) bootstrapping hacks, 2) working on the module (e.g. C-M-x usage), 3) bad practices :P ?
<ArneBab_>taylanub: in python module mutation is used for a very natural way of using singletons
<ArneBab_>(in python, every module-variable is a singleton)
<ArneBab_>what I do: import core; core.win = Window() → every module which imports core can access the Window() instance
<taylanub>I think such things are done with parameters ? In worst case one can simply export a setter procedure to get the exact same behavior.
<ArneBab_>I need that, because I create the window based on commandline flags and I do not want to push the commandline handling into the windowing module.
<taylanub>Could you elaborate on the singletons, or link to something I could read about it ?
<ArneBab_>I’m checking whether I find a good writeup
<ArneBab_>taylanub: OK, it’ll be faster to elaborate myself…
<ArneBab_>In python modules are only imported once, so if you have 3 files which all import the module, only the first import will really import the module and the others will simply bind a reference to the module to a variable.
<ArneBab_>due to that, every variable of a module is a singleton automatically.
<ArneBab_>and this makes it really easy to define shared state
<ArneBab_>(and capsule it: “This module contains all shared state”)
<adhoc>ArneBab_: thats also a function of the name space in python
<taylanub>Ah, interesting. The same thing could be done with Guile modules if I'm not mistaken.
<ArneBab_>I think yes
<ArneBab_>For classes, python also offers properties: start with a variable, and when you need special handling, add getters and setters.
<taylanub>Well anyway, I don't see this strategy being threatened much by making modules immutable by default. One could have, as you suggested, a "state" module which is simply declared mutable.
<ArneBab_>→ instead of creating special accessing functions, just capture the call to module-set! and to accesses of the variable and return the output of some function instead.
<ArneBab_>then there are two different types of modules with vastly different behavior.
<ArneBab_>aside from performance: what’s the reason to make a module immutable?
<taylanub>none, as far as I'm concerned
<taylanub>WDYM with the former two lines ?
<ArneBab_>I mean that when you work with a module, you always have to know whether it is mutable or immutable, and that increases complexity
<ArneBab_>I can already see the bug reports: „Why can’t I set the variable? The tutorial says I can!“
<ArneBab_>or worse: „module-set! is broken!“
<ArneBab_>also going from experiments to working code gets harder (experiments will likely use mutable modules)
<ArneBab_>that’s why I asked whether it would be possible to minimize the impact of the immutability. Normally only 10% of the code are performance critical, so it would be nice if just that code could be declared as immutable.
<ArneBab_>and if it would be possible to add edit-as-copy sugar which allows mutating immutable modules with module-set!, this could be a really easy and effective optimization
<ArneBab_>(for the Guile scheme programmer, not for the programmer of Guile scheme)
<adhoc>python has those moddules, but you mess with instances of them, not the state of that imported module
<adhoc>and the whole messy generator head space
<taylanub>How often does one "work with a module" other than using (importing) it, without being a developer of it anyway ? If a newcomer is just tinkering around, they'll probably hit many more errors than "tried to mutate immutable binding", and obviously it should be documented that `module-set!' won't work on immutable modules.
<taylanub>I don't see how going from experiments to working code would be harder. Working code itself generally doesn't use module mutation, so turning a module immutable will be transparent in the normal case.
<adhoc>taylanub: in python and others, you subclass and mutate that ..
<taylanub>Automatic mutable-recompilation is a fine idea I guess, as long as the user gets some kind of notification of what's happening.
<ArneBab_>taylanub: all module-set! from outside will be efficient while you develop the module, but it will either be inefficient (with automatic recompilation) or impossible once you switch to working code.
<ArneBab_>(why doesn’t working code use module mutation?)
<taylanub>Since the module system is for the organization of program components, and reorganization of program components is not usually part of a program's logic, I see it as being abnormal for a program to make use of module mutation. The big exception is interactive environments like Emacs, where said reorganization is usually still limited in scope such that immutable-by-default should be fine.
<taylanub>I have to say though, I don't think it's that important what behavior is default.
<taylanub>Since changing it should be a single switch, and compiling a module should take seconds.
<taylanub>(Static linking is another thing ...)