Zeno: Shard-first KV store in Zig
ToolsComments
The claim about eliminating read latency compromises is bold. Seqlocks can cause reader starvation if write frequency is high enough to keep invalidating the sequence counter.
I disagree that reader starvation is a dealbreaker here. In most of the monitoring tools I actually use, writes are a tiny fraction of the volume, making read-optimized locks the correct choice.
I wonder how this fits with the trend of Zig tools like intelgraph-zig... does the shard-first approach make it easier to implement the kind of local intelligence graphs we are seeing elsewhere?
We saw this play out with early HFT order books. Moving the sharding to the front of the pipeline is the only way to actually stabilize P99s when you are fighting lock contention.
art usually trades memory density for lookup speed.
Regarding that memory trade-off: does Zeno use a specific compaction algorithm for the Adaptive Radix Tree, or does it rely on Zig's general allocator for node management?
If the memory footprint expands too far, we might see a performance dip due to cache misses. It would be similar to how some oversized indexes eventually bottleneck on memory bandwidth rather than lock contention.
The real win here is that Zig's explicit memory control allows for custom arena allocation. That could potentially mitigate the cache misses Dan mentioned by keeping the ART nodes contiguous.