←back to thread

ULID: Universally Unique Lexicographically Sortable Identifier

(packagemain.tech)
68 points der_gopher | 4 comments | 01 Dec 25 20:45 UTC | HN request time: 0s | source
Show context
sedatk ◴[09 Dec 25 22:26 UTC] No.46211578[source]▶
Whenever ULID comes up, I need to remind that it has a sequential ID generation mode in its spec which is prone to conflicts on multi-threads, processes or hosts which kills the purpose of a "universal" identifier. If you need a sequential ID, just use an integer, preferably one that's autoincremented by the database.

It's best to stick to UUIDv7 because of such quirks of ULID.

replies(6): >>46212131 #>>46212144 #>>46212658 #>>46212724 #>>46213359 #>>46213609 #
cpburns2009 ◴[09 Dec 25 23:26 UTC] No.46212131[source]▶
> I need to remind that it has a sequential ID generation mode in its spec which is prone to conflicts on multi-threads, processes or hosts which kills the purpose of a "universal" identifier.

Can you expand on how this can actually cause a problem? My understanding is different processes and hosts should never conflict because of the 80 bits of random data. The only way I can conceive of a conflict is multiple threads using the same non-thread-safe generator during the same millisecond.

replies(1): >>46212222 #
sedatk ◴[09 Dec 25 23:39 UTC] No.46212222[source]▶
You're right, not hosts or processes in that case. I forgot about random part as it's been a while since I looked at it. However, a single instance of a ULID generator must support this mode, which means that on multi-threaded architectures, it must lock the sequence as it still uses a single random value. That again, kills the purpose of a client-side, lock-free generation of universal identifiers as you said.
replies(2): >>46212315 #>>46212388 #
0x457 ◴[10 Dec 25 00:03 UTC] No.46212388[source]▶
You only need to lock sequence if you care about IDs being ordered within a millisecond. That generally only matters when you create a batch of IDs at once, in that case you don't need to lock anything: generate ULID, keep incrementing sequence in that batch either by doing on the same thread, or by moving it from thread to thread. Kinda like creating an iterator and zip'ing it with iterator of thing you need IDs for.

I've switched to using UUIDv7 tho. It made sense to use ULID before v7, but now ULID only has one thing going on - smaller string representation. That doesn't matter if your storage can store UUIDs natively (i.e. as 128 bit integer)

If your goal is to have global order intact, then neither ULID nor UUIDv7 is going to work for you.

replies(2): >>46212558 #>>46212830 #
1. sedatk ◴[10 Dec 25 01:05 UTC] No.46212830[source]▶
> You only need to lock sequence if you care about IDs being ordered within a millisecond

Yes, and that's when sequences are only used. I guess that's to avoid hogging the CPU or emptying the OS entropy pool during high loads.

However, that "optimization" is a failure mode if you're not aware how ULID internals work. It's easy to shoot yourself in the foot by blindly trusting ULID will always generate a unique ID across threads without blocking your thread. That's a sneaky footgun.

> That generally only matters when you create a batch of IDs at once

No, any web service instance can receive requests at arbitrary times, and sometimes in the same millisecond zone. The probability is proportional to the number of concurrent users and requests.

> If your goal is to have global order intact, then neither ULID nor UUIDv7 is going to work for you.

Agreed.

replies(2): >>46213015 #>>46246298 #
2. jasonwatkinspdx ◴[10 Dec 25 01:32 UTC] No.46213015[source]▶
> or emptying the OS entropy pool during high loads.

Just a heads up that's not really a thing. If the CSPRNG is initialized correctly you're done. There's nothing being depleted. I know for ages the linux docs said different, they were just wrong and a maintainer was keeping a weird little fiefdom over it.

replies(1): >>46213229 #
3. sedatk ◴[10 Dec 25 02:10 UTC] No.46213229[source]▶
Thanks for the heads up, then it’s one less reason for ULID to adopt this weird behavior.
4. 0x457 ◴[12 Dec 25 17:25 UTC] No.46246298[source]▶
> No, any web service instance can receive requests at arbitrary times, and sometimes in the same millisecond zone. The probability is proportional to the number of concurrent users and requests.

Yes, but does it matter that you have out of order IDs within the same ms for concurrent requests? That's why I said batch. I only ever been an issue for me when I've chosen ULID as an ID for an event log (if the command produced more than one event, random bits will ruin the order)

> However, that "optimization" is a failure mode if you're not aware how ULID internals work.

That's not ULID internals, that's whatever library you're using. The rust implementation I've used, for example, will generate random bits unless you implicitly increment, and that requires `&mut`