IRC channel logs
2026-09-16.log
back to list of logs
<cosmicusr>Hello, so I wanted to ask how does one properly handle the case when you need exactly-once processing of messages to OCapN objects. In particular, it seems that the issue is that, if CapTP is run over regular TCP, for example, that TCP connection can die at any time without the peer knowing that the connection died. <cosmicusr>Since there are no CapTP-level acks, the peer doesn't know which CapTP operations were actually delivered to the other peer. Even if there were acks, the connection itself could die before the ack is sent but after the operation is received. So after reconnecting, you're risking either message loss or message duplication depending on whether you attempt to redo the delivery of messages to objects. <cosmicusr>Is the proper fix to implement a netlayer that fixes this problem or to add idempotency to messages or something else entirely? <dthompson>this channel has been very quiet for a long time! <dthompson>a captp-level ack would be resolution of a promise. <dthompson>you're right that there's no guarantee that a message has made it to the recipient. <dthompson>it is indeed possible that a message could be received and that the connection could close before the response is sent <dthompson>exactly-once delivery is thus something you could layer on top <dthompson>for example, I've implemented causal broadcast on top of ocapn that prevents messages from being processed more than once <cosmicusr>But a resolution of the promise is not really an ack, is it, because I kind of interpret it to be a completion of the function that is launched by the message rather than acceptance of responsibility over the message - although I think that distinction rarely matters. <dthompson>it's up to the communicating parties to determine what resolution of a particular promise means <dthompson>if alice wanted to wait for a signal from bob before doing something else then a promise is the way to do that <cosmicusr>So what is it that needs to be done, idempotence like in HTTP needs to be added to objects at the application level? <dthompson>there's not a single answer. idempotence is one approach. <dthompson>it's definitely a problem to be solved by a higher level of abstraction than ocapn, though. <cosmicusr>Well I guess I'm asking what's the proper way to handle it from the point of view of the capability theory <cosmicusr>It also can be handled by a netlayer I think <dthompson>it is possible to define a netlayer whose session is longer than any actual, say, TCP session. but there be dragons. <dthompson>and I would say that capability theory is orthogonal to this issue <dthompson>are you trying to build a particular application? <cosmicusr>To talk about something else: why is there a distinction between desc:import-object and desc:import-promise? I couldn't figure it out last time I read the spec <dthompson>cosmicusr: promises are special and it's important to know when you're dealing with them because they are a layer of indirection. <cosmicusr>Not exactly, I have this hobby project (which I'm doing very inactively) in which I'm attempting to create a federated instant messaging protocol that is supposed to be (in my opinion) better than XMPP and Matrix. It's a project mostly for fun. Basically, the protocol ultimately turned out to suck I believe, and the current version is not even the first version. Thought maybe it could be made on top of CapTP of some kind or something <cosmicusr>Even though it might not necessarily be a good fit <dthompson>for example: if promise A gets resolve with another promise B, you know that the first promise hasn't settled yet <dthompson>cosmicusr: I think chat over ocapn would work quite well. at spritely we have several prototypes of different chat architectures. <dthompson>and in fact we use a (admittedly primitive) ocapn chat application every day as part of dogfooding the protocol <dthompson>to avoid duplicate message delivery in chat, you could use content-addressed messages. receiving a duplicate becomes a no-op. <cosmicusr>dthompson: Not sure I entirely get it, I think I missed that part entirely, why isn't promise A settled? If that's the case, what changes in the protocol from typical handling of promise resolution? <dthompson>what I mean is that a promise can be resolved with another promise, forming a chain of promises <dthompson>promise chains introduce the ability to do promise pipelining and shortening <dthompson>which are the fancier parts of ocapn that take some time to understand <dthompson>a promise is settled when it finally is resolved with some non-promise value. <cosmicusr>Well I get that, you can do promise pipelining on a promise which resolved to another promise which resolved to another promise, so any messages sent to the initial promise would get forwarded across the chain, right? <dthompson>well rather than being forwarded they get sent to the eventual result of the promise <dthompson>well, on second thought, I think the answer is "yes" :) <dthompson>the shortening optimization is used to reduce the length of promise chains <dthompson>whereas pipelining allows queueing up messages to objects that haven't resolved yet <cosmicusr>But aren't peers already supposed to know that the promise created in response to some specific message would get resolved with another promise ahead of time, kind of like they know what args they need to send to some object exported to them by other peer (they know what the object represents ahead of time, even though its name doesn't seem to be exchanged anywhere in the protocol)? <dthompson>if I'm understanding your question then no, a peer has no idea, in general, how a promise is going to resolve. <cosmicusr>How do they know what messages to send to objects then? <cosmicusr>No, for example, say peer A has some object exported to it by peer B. Then peer A uses op:deliver to deliver a message where the first arg is "open-dir" and the second arg is some other string. So a promise is created and then resolved by peer A to export a new object for peer A. Then Peer A delivers a message to the new object, perhaps "open-file" and peer B once again resolves with some new object. <cosmicusr>Then, maybe, peer A delivers message to the newest exported object with the first arg "stat". <cosmicusr>So it seems to me that it would make no sense for peer A to send these messages if peer A had no idea what these objects represented <cosmicusr>So peer A knows, ahead of time, the application-level API and in particular that the first object is perhaps of type "Directory", which has a method called "open-dir", which takes in the name of the directory (string) and returns another Directory, and a method "open-file", which returns an object of type "File", which has a method "stat". If one is to talk in OOP-like terms anyway. <cosmicusr>By "returns", I mean that a promise is created, which is then resolved with an object of that type. <cosmicusr>That info isn't exchanged in the protocol, so it seems that peer A needs to know this interface ahead of time, so why can't peer A just know that some method resolves with a promise ahead of time too (which seemingly would make the distinction between import-promise and import-object pointless)? <dthompson>"peer A needs to know this interface ahead of time" - correct, capabilities are opaque by design. <dthompson>I'm not sure what to make of that last part. every op:deliver results in a promise <cosmicusr>If you mean "that some method resolves with a promise", I meant that the promise created by a op:deliver resolves with another promise <dthompson>there's no static typing or anything like that <dthompson>leaving aside return types, I think even if you could assume that there would still be other reasons for distinguishing promises from other objects. <dthompson>ocapn implementations are expected to do things with promises that they don't do for other objects, such as listening for shorter promises. <gibson042>object references are stable from the perspective of an OCapN peer, while promises are unstable until they are fulfilled or broken, which obligates the peer to do separate bookkeeping for them. desc:import-object vs. desc:import-promise explicitly exposes that distinction. <gibson042>and importantly, independently of any particular application semantics that sit above OCapN <gibson042>for when cosmicusr returns: "unstable" as in "subject to change", particularly via state transition... for example, a message sent to a promise after it is fulfilled can and should be delivered directly to its fulfillment value, which obviously isn't possible *before* fulfillment (during which time such messages must instead be held in a queue). There is no analogous state machine for object references, and if promises were not in the data model at <gibson042>all then there could be no protocol-level shortening.