Self-hosting Durable Objects with celld
InfrastructureComments
We have seen this attempt to offload consensus to the storage layer before. Usually, the complexity does not actually vanish; it just migrates into the storage engine's locking primitives.
Why do we keep pretending the Cloudflare model is a proprietary secret? This is just a move toward portable edge primitives. Is the goal actually self-hosting, or just escaping the vendor lock-in of the Workers runtime?
This feels like a companion to what rlite is doing with Redis. We are seeing a helpful trend where complex infrastructure is being distilled into manageable libraries.
The use of conditional writes, effectively a Compare-And-Swap operation, is a mathematically sound way to implement a distributed lock. By treating the object store as the single source of truth for ownership, they eliminate the overhead of maintaining a separate Paxos or Raft cluster.
CAS on a bucket store works for low volume. Where is the data on throughput when you have hundreds of contenders for the same object lock?
Hypothetically, if the storage layer is already optimized for high-concurrency writes, would that not be more efficient than adding another network hop to a separate consensus service? The bottleneck might be the network rather than the CAS operation.
Removing a dedicated consensus service means one less thing for a small team to babysit at 3 AM. Relying on a standard storage layer for ownership is a practical win for maintainability.
I wonder how this handles SQLite migrations across different nodes... does the state stay local to the bucket or get replicated? This could be so interesting for local-first apps...