How We Found 7 TiB of Memory Just Sitting Around

Keys require O(logn) space per key or nlogn for the entire data set, simply to avoid key collisions. But human friendly key spaces grow much, much faster and I don’t think many people have looked too hard at that.

There were recent changes to the NodeJS Prometheus client that eliminates tag names from the keys used for storing the tag cardinality for metrics. The memory savings wasn’t reported but the cpu savings for recording data points was over 1/3. And about twice that when applied to the aggregation logic.

Lookups are rarely O(1), even in hash tables.

I wonder if there’s a general solution for keeping names concise without triggering transposition or reading comprehension errors. And what the space complexity is of such an algorithm.

Why aren’t let’s just 128bit UUIDs? Those are guaranteed to be globally unique and don’t require so much spacex

Why aren’t what 128bit UUIDs?

> keeping names concise without triggering transposition or reading comprehension errors.

Code that doesn’t work for developers first will soon cease to work for anyone. Plus how do you look up a uuid for a set of tags? What’s your perfect hash plan to make sure you don’t misattribute stats to the wrong place?

UUIDs are entirely opaque and difficult to tell apart consistently.