IRC channel logs
2023-02-22.log
back to list of logs
<civodul>tsyesika & i had interesting conversations at FOSDEM BTW! <civodul>not yet, but in the not-too-far future maybe? <civodul>cwebber hinted at this a couple of years ago, and tsyesika had good ideas too <jfred>This reminds me I have an email I've been meaning to write to guix-devel about that <jfred>But work and home things have been so hectic 😅 <haugh>composing promises to reason about the future state of a shepherd service with the ability to restore any local service to a known working state would make me into some kind of Operations superhero... <haugh>I don't have my head all the way around Goblins yet but I'm finding it difficult to retain a healthy level of skepticism <haugh>As a child I had a textbook whose cover featured an illustration of the first flight of the Wright brothers' glider. In the foreground, a man stood holding his top had uselessly beside him, looking up at a machine whose potential was simply outside his capacity for reasoning. I think about that guy a lot <EWEW>haugh: I'm curious, which pieces would you say are unclear? <EWEW>Or is it more about the broader picture and how it ends being applied/results "in the real world" <haugh>EWEW, yes, like a lot of concepts I've learned from the GNU folks, the difficulty (but also the ultimate flexibility) is in fitting them together according to particular purpose. For example I think Guile's delimited continuations are an incredibly construct but I don't have a single practical application for them, lol. Right now I'm thinking about how Guix and Goblins both provide local transactionality and therefore what we're calling "time <haugh>travel," but at different layers of the system. <EWEW>There's a phrase I learned from a friend while talking about having expectations of something. <haugh>I think "time travel" completely changes my relationship to persistent data. I make a living by walking into operations centers and creating/improving relationships between monolithic applications. The big question has always been "where do we persist the state?" You don't want to make partial copies of all these different databases, but you also don't want to get married to any of the existing apps (any more than the client already is). The thing <haugh>is, I only every create persistence in order to survive systems failure. If each of my nodes has the ability to "re-instantiate" (?) both its memory and its filesystem, suddenly I don't think I need any new tables! <EWEW>"Обіцянка цяцянка, а дурневі - радість" roughly, Prmises are toys, but a joy for a fool. Which the real meaning of the phrase to me in a practical sense is that. Practicality will come from something once you have it, but not before. <EWEW>reading what you wrote now. <EWEW>"Where do we persist the state?" So my take on this question is, the state is already persisted as the union of the states of all monoliths. The real difficulty is in all of the "in-flight" interactions between the monoliths where there are no guarantees on when/if those interactions become concrete in one or more monoliths. <EWEW>by controlling when/if these in-flight operations will become "concrete" simplifies the universe <EWEW>because by not controlling that, you get this explosion of possibilities. operation 1 X {operation 2, ..., operation N} <EWEW>but it's naive to say you could say ahead of time that say the operations will be committed to state in a well defined linear order say. operation 2, operation 1, operation 5, operation 3,... <EWEW>but time traveling as a mechanism could let you find a middle ground <EWEW>between "anythng is possible" and "i know exactly what will happen" <EWEW>Caveat I should mention, this is just me thinking out loud. <EWEW>but with time traveling you can say, okay i don't know which way these operations will be committed. I don't even know in which order they need to be committed to have every monolith's state be a proper non-broken state <EWEW>but i can be confident that if i do get to a broken state, i can rewind and try committing them in a new, different, order that gets the ultimate state of all monoliths to a non broken one <EWEW>and if you can get all of them to a non-broken state you can now confidently say the answer to your question of "where do we persist the state?" <EWEW>"the state of all monoliths" <EWEW>you don't need to develop from first principles some meta system that keeps them all working together properly <EWEW>let me try to put it another way actually. The problem of persisting state between many monoliths with the introduction of communication between any of them. It's really a problem of you have multiple systems that each define constraints on what their valid states are. With communication between them each singular monolith now needs to further restrict what their set of valid states is. <EWEW>so that monoliths don't contradict each other. <EWEW>time traveling as a mechanism is sort of like a process which allows you to search for a satisfiable answer that meets the constraints of all monoliths concurrently <EWEW>where the satisifiable answer is one that modifies the state of every monolith to bring them as a group to non-contradictory states. <haugh>EWEW, You are technically correct in your first assertion that the entire state in such a system is already accounted for; the issue is that there are massive bottlenecks between the existing monoliths. There are the obvious limitations of network interfaces and APIs (intentional or... otherwise), but more importantly, the Truth of a given datum depends on context provided asymmetrically across the whole system. Either you persist the work you've <haugh>already done or you commit to repeatedly re-interpreting massive amounts of data. Not an easy choice, especially when inter-organizational policy gets involved, but I'm losing the plot. You said, "With communication between them, each singular monolith now needs to further restrict what their set of valid states is." I consider this to be a sometimes-unavoidable failure. I build a layer of business logic which attempts to tolerate the <haugh>"untruthfulness" of individual monoliths. Now I'm worried about flying offtopic, but I think I agree with your statement that time-traveling is "searching." It means I can reason without incurring a specific kind of corrosive cost. <EWEW>I don't think it's losing the plot, those considerations are of practical importance. <EWEW>On that I would need to think more. <EWEW>but those are good points for sure. <haugh>In addition to monoliths being untrustworthy sons of bitches that will eventually kill me, I have to consider the limitations of my own code, and how my changes are propagated through the meta layer. Some of the Guix folks don't think that time travel has a big impact on continuous integration, but it's all I can think about, especially if my changes don't have to be grounded in database mutation. <haugh>EWEW, oh absolutely they are and I want to thank you for the opportunity to articulate some of this crap. It's been on me like wet clothes. I just want to be cognizant of the fact that a lot of my issues are pretty specific to my choice of business and in many ways I get what I deserve and that's not anyone else's problem :) <vvvvmvr[m]>I can't get into specifics but I will say that in my job we have grappled with synchronizing state remotely, and have arrived on the Message Queue approach as the best way for our use case. However, this approach works best if the flow of data is very clear. If you have chatting back and forth, it breaks down. And also, it relies on a single "broker" to be authoritative (though broker quorums do make this at least more fault tolerant). <vvvvmvr[m]>Again, this is within a trusted network, so isn't really relevant to decentralization. But I still find it an interesting approach. <EWEW>haugh: Glad to be of service. Of course I just like thinking about things, so from my perspective the cost to talk about anything is basicaly zero :) <haugh>I vaguely recall some talk related to Lisp productivity where someone said "Our Java programmers are productive all day, but our Lisp consultant just stares off into space for 5 hours, writes for 15 minutes, and then goes home!" <Zarutian_iPad>one guy I worked with sat thinking for fifteen minutes between moving an probe of an osviloscope between test points <Zarutian_iPad>he was chasing a fault in a rather expensive piece of equipment <Zarutian_iPad>to sate my curiosity I been looking into how Unity (or at least the Netcode library they provide) does multiplayer <vvvvmvr[m]>hmm. unity networking is pretty yucky. unless it's improved lately. Last I fiddled with that, there was a 3rd party lib Mirror that was far better <Zarutian_iPad>though Unity Netcode uses network ticks and bundles up changed NetworkVariables and RPCs at every tick <Zarutian_iPad>note Unity Netcode is not doing physics interpolation or such <Zarutian_iPad>as cwebber can attest to I have talked extensively about the OpenCroquet slash Snes9x Netplay does things. <Zarutian_iPad>vvvvmvr[m]: looks like it is based, conceptually on UNET which is Unity Netcode predecesor so they share some similiarities <Zarutian_iPad>which pretty much looks like it follows the Quake II and Quake 3 Arena model conceptually <Zarutian_iPad>btw many moons ago I also looked into how Second Life and Roblox do netcode <vvvvmvr[m]>I've found the current approaches to netcode I've seen are not very good. very hard to reason about and prone to errors. <Zarutian_iPad>having auto synchronizing variables (Unity Netcode NetworkVariables) and RPC (which basically grew out of Q2 impulse stuff iirc)? <vvvvmvr[m]>Mainly the tough part is trying to figure out what code is run on what side. Which takes a bit to get used to <vvvvmvr[m]>Also, the SyncVar stuff tends to be problematic because it can be slow, and you can't easily tune it ... idk <vvvvmvr[m]>I'm no expert on this though, it's just from my own tinkering. <vvvvmvr[m]>I don't think I can comment on Vats yet as I am still learning them and getting used to it. <Zarutian_iPad>well there is the aforesaid OpenCroquet slash Snes9x Netplay that many Real Time Stratedgy games use <Zarutian_iPad>basically all user commands/impulses go through the server and is reliably broadcasted to all participants and spectators <Zarutian_iPad>I like to think of that server as doing the role of the Eye of the Hourglass <Zarutian_iPad>that is the impulses/commands come in like sandgrains in the top bulb and go through the pinch point one by one <Zarutian_iPad>this ensures the sequence of them to all the participants and spectators is the same <Zarutian_iPad>then these impulses/commands get fed into the local deterministic replica of the gamestate and game rules <Zarutian_iPad>meaning you only have to program your game like you had local single computer multiplayer <Zarutian_iPad>one kind of technique you could use in the transport (like Unity Netcode and Mirror call it) when using udp instead of tcp is to send part of the history buffer of user impulses/commands in every udp packet <Zarutian_iPad>you could also add rollback ontop by regularly delta-snapshot the game state&rules, have the replay run it as fast it can when catching up and then lerp the visible scenegraph objects locations, rotations, velocitiesm etc from the current stale game state version to the new more up to date version <Zarutian_iPad>longish drop outs would look a bit gloopy but I think people wouldnt mind <vvvvmvr[m]>networked games are weird. So, I wouldn't want to speculate on how it would 'feel'. There are so many hacks used to make things seem 'right'.