Show HN: Rivet Actors – Durable Objects build with Rust, FoundationDB, Isolates

(github.com)

129 points by NathanFlurry 4 days ago | 25 comments

Hello! We posted a Show HN for Rivet last year for our container orchestration project (https://news.ycombinator.com/item?id=37188659). In that time, a lot has changed that I think HN will find interesting.

Rivet is open-source actor infrastructure similar to Cloudflare's Durable Objects. Rivet itself already serves millions of MAU in production using our current container runtime – primarily for multiplayer games – and Rivet Actors are a new extension to support actor-like workloads. Rivet Actor's core primitives are RPC, state, and events.

Actors are powered by Rust, V8 isolates (supports Deno), and FoundationDB. An architecture diagram is available here for [1]. If you're not familiar with FoundationDB, you're overdue to watch Dave Rosenthal's talk [3]. (I firmly believe it's by far the best permissively licensed database; if only it had a well maintained SQL layer.)

Here's where Rivet's architecture gets fun – we don't rely on a traditional orchestrator like Kubernetes or Nomad for our runtime. Instead, our orchestrator is powered by an in-house actor-like workflow engine – similar to how FoundationDB is powered by their own actor library (Flow [4]) internally. It lets us reliably & efficiently build complex logic – like our orchestrator – that would normally be incredibly difficult to build correctly. For example, here's the logic that powers Rivet Actors themselves with complex mechanisms like retry upgrades, retry backoffs, and draining [2].

One of the reasons we built Rivet Actors is because we tried to replace most of our Redis-based realtime infrastructure with Durable Objects. The architecture allowed us to build realtime features much faster & efficiently, but the platform & APIs were needlessly rigid and difficult to use. Our goal is to build an actor-like platform that includes the bells and whistles required for developers to benefit from the actor model without the learning curve of tools like Erlang/OTP, Akka, or Orleans.

Rivet Actors provides a few key benefits in flexibility over Durable Objects:

- Open-source (Apache 2.0) – built to be self-hosted and deployed on-prem

- Provides observability out of the box, no Logpush required

- Rivet Actors support the Deno runtime, so NPM & JSR just works

- @rivet-gg/actor [5] framework provides RPC, state, and events out of the box for faster bootstrapping; you can modify and deploy it yourself

- Supports both V8 isolates & Docker-compatible containers so you can run any software you'd like, like Godot/Unity servers or video transcoding

- Also supports TCP & UDP (we run games!)

- Provides vanilla HTTP API for easy use with existing apps

- Full control over regions

There's plenty more that I don't have space to talk about. Give our docs a read if you'd like to learn more [6] or read about internal design decisions [7]. I'll be in the comments answering questions!

Cheers, Nathan

[1] https://github.com/rivet-gg/rivet?tab=readme-ov-file#diagram

[2] https://github.com/rivet-gg/rivet/blob/a3db31f3b5c351061d665...

[3] https://www.youtube.com/watch?v=9g84y_60VGM

[4] https://apple.github.io/foundationdb/flow.html

[5] https://jsr.io/@rivet-gg/actor/doc

[6] https://rivet.gg/docs

[7] https://rivet.gg/docs/internals/design-decisions

iFire 4 days ago | next |

If I wanted a Rivet integration with Godot Engine and/or Elixir, how would I do it for shared durable objects in a virtual reality environment?

I have code for Godot Engine Foundationdb access, but I'm cutting too many encapsulation layers.

NathanFlurry 3 days ago | root | parent |

You have Godot wired up to talk to FoundationDB? That's incredibly cool, curious about your use case.

Godot runs using our container runtime on Rivet. The FDB API for containers is currently undocumented. I'd be happy to share early access on Discord – https://rivet.gg/discord

Games running on Rivet usually have a hybrid of actors that handle persistent storage (e.g. a matchmaker) and the game server itself.

iFire 3 days ago | root | parent |

I aim to build a collaborative VR world editor in Godot using mvsqlite for shared scene persistence.

I was salvaging someone else's toy project. The main problem with my research was write performance in SQLite on FDB. I didn't have the capacity to integrate Hctree, which was in the pipeline.

* https://github.com/V-Sekai/godot-mvsqlite

* https://github.com/V-Sekai/mvsqlite

* https://github.com/V-Sekai/elixir-mvsqlite

* https://sqlite.org/hctree/doc/hctree/doc/hctree/index.html

Let's connect on discord.

Hixon10 2 days ago | prev | next |

> Here's where Rivet's architecture gets fun – we don't rely on a traditional orchestrator like Kubernetes or Nomad for our runtime. Instead, our orchestrator is powered by an in-house actor-like workflow engine – similar to how FoundationDB is powered by their own actor library (Flow [4]) internally. It lets us reliably & efficiently build complex logic – like our orchestrator – that would normally be incredibly difficult to build correctly. For example, here's the logic that powers Rivet Actors themselves with complex mechanisms like retry upgrades, retry backoffs, and draining [2].

It is a bit unclear for me, do you use actors themself to develop Rivet Actors, or it is another actor-like workflow engine, not the final product?

(I would be super happy to read an article, which explains architecture, main blocks of the system, gives some an example)

FjordWarden 3 days ago | prev | next |

How is this different from the Durable Promises idea by ResonateHQ? It seams to me a bitter easer to get started with as a small standalone project. Would you mind comparing this solutions to that?

NathanFlurry 3 days ago | root | parent |

I skimmed a bit about Durable Promises, and if I understand correctly, they’re similar to workflow tools like Temporal but come with high-quality language bindings that let you write workflow steps using async/await.

We built something almost identical in Rust to let us use async/await for long-running & failure-prone workflows. This powers almost everything we do at Rivet. We have a technical writeup coming later, but here's a couple links of interest:

- Rough overview: https://github.com/rivet-gg/rivet/blob/58b073a7cae20adcf0fa3...

- Example async/await-heavy workflow: https://github.com/rivet-gg/rivet/blob/58b073a7cae20adcf0fa3...

- Example actor-like workflow: https://github.com/rivet-gg/rivet/blob/58b073a7cae20adcf0fa3...

Rivet Actors can start, stop, or crash at any time and still continue functioning, much like Durable Promises.

However, their scope differs: Rivet Actors are broader and designed for anything stateful & realtime, while Durable Promises seem focused on workflows with complex control flow.

Rivet Actors can (and likely will) support workflow-like applications in the future, since state management and rescheduling are already built-in.

For a deeper dive, Temporal has a writeup comparing actors and workflows: https://temporal.io/blog/workflows-as-actors-is-it-really-po...

pvg 4 days ago | prev | next |

Related discussion in 2023 https://news.ycombinator.com/item?id=37188659

NathanFlurry 4 days ago | root | parent | next |

Yep! Same repo, lots of changes. Since we were on HN in 2023, we've:

- Rewrote Nomad from scratch with our own in-house workflow engine

- Launched Rivet Actors and isolate runtime

- Overhauled our OSS experience to make self-hosting easier

- Rewrote frontend from scratch with shadcn to be like the cool kids

- Replaced Nebula with mTLS

- Completely stripped out Redis for non-caching use cases

We've also opened up the platform to non-gaming use cases.

Animats 3 days ago | prev | next |

What kinds of web sites need this level of complexity? They must be doing something really hard and tightly coordinated, such as collaborative editing, right?

Does this have enough performance to solve the server cost problem of Improbable's metaverse back end? That's a remote object system too expensive to run 24/7.

NathanFlurry 3 days ago | root | parent | next |

Hey!

> What kinds of web sites need this level of complexity?

The most common use cases are high contention & strong consistency workloads where using an OLTP database doesn't work well. A few examples:

- Collaborative docs

- Local-first & CRDT sync (e.g. Yjs)

- Stateful multiplayer games, often async multiplayer (e.g. chess)

- Agentic workflows with frequent memory updates

- Chat

- Scheduling software

- Rate limiting

Actors can also be used for stateless realtime workloads that benefit from edge computing, similar to what you would use existing serverless edge functions for.

The actor model is widely used in larger companies. The best-known example is how WhatsApp used Erlang to scale to 900 M users with 50 devs [2]. It's also widely used at Discord, LinkedIn, Microsoft, etc. Take a look at Erlang/Elixir, Akka, and Orleans for prior art.

> Does this have enough performance to solve the server cost problem of Improbable's metaverse back end?

Actors usually have a single responsibility (e.g. document or chat room) while Improbable focused on building expansive worlds without clear boundaries between entities. Our docs include tips on scoping the responsibility of actors [1].

The actor model encourages actor-actor communication, sort of similar how entities in the ECS model communicate in games.

> That's a remote object system too expensive to run 24/7.

Actors automatically go to sleep when unused. They provide the benefit of realtime workloads without the cost of keep idle compute sitting around.

[1] https://rivet.gg/docs/scaling#tips-for-architecting-actors-f...

[2] https://www.wired.com/2015/09/whatsapp-serves-900-million-us....

declan_roberts 3 days ago | root | parent | prev |

If you have a really tiny bit of compute that must happen on the edge, I guess it makes sense.

NathanFlurry 3 days ago | root | parent |

For example – I'm a big proponent of local-first apps, but even those require a sync server. You can easily implement one in under 100 lines of code with actors.

cpursley 3 days ago | prev | next |

Here’s my once a month comment that I should probably put on autopilot:

“at least half of show HN posts are just reimplementations of what already exits in Erlang/Elixir”.

declan_roberts 3 days ago | root | parent | next |

The actor model existed before Erlang, and Erlang doesn't have some kind of a monopoly on it. Akka is pretty good too.

Same as every OOP isn't "something that exists already in Java"

NathanFlurry 3 days ago | root | parent | prev | next |

Haha, I’m with you there. I was considering calling that out in the post.

I dearly love Erlang & co., but its learning curve is way too steep for most developers today. Our goal is to bring the benefits of Erlang/Akka/Orleans/etc. to more developers by:

(a) supporting mainstream languages, and

(b) lowering the technical and conceptual barriers to entry.

foota 3 days ago | prev | next |

Out of curiosity, what's the sort of consistency (in the sense of linearizable etc.,.) provided here? If an RPC is handled by some actor, is it guaranteed to have seen all previously accepted messages that completed up to that point?

If so, how does that work?

Edit: I guess this sort of comes by forcing everything to round trip through foundationdb for state? As long as the actor reads the state from foundationdb before handling the response, and writes to foundationdb before sending a reply, it should guarantee the same sort of consistency you'd get from foundationdb?

What about calls to other actors? If I make a potentially state changing RPC call to some other actor as a part of handling an RPC, do those commit together, or is it possible for the other actor to commit without me?

NathanFlurry 3 days ago | root | parent |

We document as many design decisions like this as we can. Here’s a related bit on serial vs. parallel RPC/message handling [1].

> I guess this sort of comes by forcing everything to round-trip through FoundationDB for state?

If you’re using the KV API directly, this is correct.

Actors also have a `this._state` property, which is automatically written to FDB after each RPC call if modified [2]. This allows developers to rapidly prototype by writing vanilla JS code like `this._state.count += 1` without having to worry about writing state and its associated edge cases.

> What about calls to other actors? If I make a potentially state-changing RPC call to some other actor as part of handling an RPC, do those commit together, or is it possible for the other actor to commit without me?

Not at the moment. You’d need to use a 2-phase commit for that.

[1] https://rivet.gg/docs/internals/design-decisions#parallel-rp...

[2] https://rivet.gg/docs/state#state-saves

foota 3 days ago | root | parent | next |

Got it, thanks for the reply!

It might be interesting if there were a way to represent a part of a transaction to commit as a sort of effect of an RPC. I don't know the details, but I'm aware of a non-public system that does a similar sort of thing to coordinate a distributed transaction between many different RPC services that are otherwise opaque.

E.g., you make a bunch of RPC calls, some part of the framework is tracking the read set and writes that need to be made, and then when the root of the RPC is ready to commit it can all atomically commit or fail (if there was a transaction conflict or something).

Pozzuh 2 days ago | root | parent | prev |

> written to FDB after each RPC call

What does "after" mean in this case? Is the state change durable in FDB before the client sees the result of the RPC call?

wdb 2 days ago | prev |

Looks interesting. I am wondering if I could get this to run with F5 XC