The principles of database design, or, the Truth is out there

Why have both a database ID and UUIDv7, versus just a UUIDv7?

There is a security principle to not expose real identifiers to the outside world. It makes a crack in your system easier to open.

Actually, it should be a database ID and an encrypted database ID, which doesn’t require storing a second ID. Even better, you can make that key unique per session so that users can’t share keys. For security reasons, it is a bad idea to leak private state, which UUIDv7 does.

A single AES encryption block is the same size as a UUID and cheap to compute.

> A single AES encryption block is the same size as a UUID and cheap to compute.

I didn’t realise this! The UUID spec mandates some values for specific digits, so I assume this would not be strictly valid UUIDs?

Idk that reeks of security through obscurity to me. Your authorization/permission scheme has got to be fubar'd if you're relying on obscuring IDs to prevent someone from accessing a resource they shouldn't.

I'm sure I'm missing something obvious, but I'm not sure what other threat vectors there are, assuming it's in conjunction with other layers of security like a solid authorization/access control scheme.

I guess I'm not the biggest fan for a few reasons. I'd rather try and design a system such that it's secure even in the case of database leak/compromise or some other form of high transparency. I don't want to encourage a culture of getting used to obscurity and potentially depending on it instead of making sure that transparency can't be abused.

Also, it just feels wasteful. If you have two distinct IDs for a given resource, what are you building your foreign keys against? If you build it against the hidden one, and want to query the foreign table based on user input, you've gotta either do a join or do a query to Get the hidden key just to do another query. It just feels wasteful.

EDIT: apparently RFC 4122 even says not to assume UUIDs are hard to guess and shouldn't be used for security capabilities. So if it shouldn't be depended on for security, why add all this complexity to keep it secure?

They would not be valid UUIDs, it is an opaque 128-bit value.

To be honest many companies are not using strictly standard conforming UUIDs everywhere, and UUID has become a synonym for opaque 128-bit identifier. Consumers of the value just need them to be unique. All standard UUID versions are sometimes prohibited in environments, except maybe UUIDv8, so this semantic ambiguity is helpful.

Technically, you could make it present as a standard UUIDv4 or UUIDv8 by setting a few bits, as long as you remember to add them back if you ever need to decrypt it. The entropy might be a bit off for a UUIDv4 if someone actually checks but you can guarantee the uniqueness.

Using AES to produce a UUID-like key is an old but useful trick. Both ARM and x86 do that encryption in hardware — it is cheaper to generate than the standardized UUID versions in most cases.

So you've got a database ID, either serial or uuid? And you encrypt it when you send it to the user, maybe encrypted against their JWT or something to maintain stateless sessions?

And I guess if the user references specific resources by ID you'd have to translate back? Assuming the session has been maintained correctly,which I guess is a plus for stateful sessions. And it doesn't really matter if you get a collision on the encrypted output.

I spent enough time in the nibling comment talking about my doubts about that advice not to publicly share the identifying key. But I'll add one more point; it feels like a bunch of added complexity for marginal benefit.

The point you may be missing is that the key itself contains information about records in the database that you don’t have access to. There are many famous examples in literature (e.g. the German Tank Problem [0]) of useful attacks on known unique serials to infer hidden information without access. In a database context, the keys for the records you are authorized to see can tell you much about the parts of the database to which you have no access.

Strong randomization of record serials mitigates these attacks.

[0] https://en.wikipedia.org/wiki/German_tank_problem

Can you explain this a bit more or link to something? I don’t really understand. What’s encrypted? A guid? A monotonic integer ID? Is the encrypted ID only used for user facing stuff? How is it decrypted? What do you gain by this?

I thought we were talking about UUIDv7, which is random enough to make this not a problem right?

The only thing UUIDv7 exposes is its creation time, which isn't tremendously useful or secret information.

I don’t have a link. I’ve never seen a good writeup but the practice is really old.

It is literally encrypting whatever type you are using as a handle for records that you send the user, typically an integer, a UUID, or some slightly more complex composite key. The only restriction is that the key should not be longer than 128-bits so that you can encrypt it to a UUID-like value; in practice you can almost always guarantee this [0]. The encryption only happens at the trust boundary i.e. when sending it to a random user. That encrypted key is not stored or used at all internally, it just changes the value of the key the user sees.

Most obvious ways of generating handles in a database leak private state. This is routinely exploited almost to the point of being a meme. Encrypting the handles you show the user prevents that.

An advantage of this is that if you are systematically encrypting exported keys then you can attach sensitive metadata to those keys at runtime if you wish, which can be very convenient. You have 128 bits to work with and a unique serial only needs 64 at most. If you are paranoid, there are schemes that enable some degree of key authentication. And while well beyond the scope here, similar constructions work nicely for compact keys in federate data models.

At scale, the cost (storage, compute, etc) of all of this matters. Encryption of keys, if done intelligently, is atypically efficient on all accounts while providing explicit inspectable security guarantees.

[0] There are reasons you might want to expand the exported key to a 256-bit value, particularly in high-assurance type environments, but the advantage of 128-bits is that it is literally drop-in compatible with UUIDs almost everywhere.

The idea being to expose uuid instead of the natural index.

It’s been downgraded as people use uuids more.

That said, security through obscurity is an effective layer, particularly for slowing an attack.

Slowing lateral movement is valuable.

Sorry I'm a bit confused.

I'm in agreement that a natural key shouldn't be used as the primary key for a record.

I was responding to a comment about having a hidden "database ID" (which I interpreted as being a serial key?) and a public "Uuid", and questioning the utility of that hidden database ID versus having a public UUIDv7 as the sole primary key, followed by questioning whether the utility of obscuring that primary UUIDv7 is worth the complexity of having to manage multiple artificial keys.

I agree that security through obscurity is a valuable layer in a multi-layered security position.

I guess I just don't think obscuring a Uuid primary key is worth the added complexity in most systems.

I see it like adding a second front door to your house with a separate set of keys. Sure it'd be more secure, but it's an added pain and doesn't help if you don't have a sturdy doorframe, or smash-resistant windows.

UUID is PUBLIC id, use it to look up the bigint numerical id, when necessary.

one should not divulge scale, placement in numerical sequence, etc wtr to integer id, hence ouvlic UUID, which is basically unguessable token

Why have a bigint numerical ID at all?

In this case obscurity is security: leaked information from the database is directly useful and a secret in itself, not only a stepping stone that might or might not facilitate further exploitation.

Which mode of AES do you mean? AES ECB seems to fit your example, but wouldn't it still expose the incremental nature UUID v7 with enough samples (see https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation, the Linux Tux image example)? Other AES modes produce way longer results, even when the ID is encoded with Base58 or the like.

Which mode of AES do you mean? AES ECB seems to fit your example, but wouldn't it still expose the incremental nature UUID v7 with enough samples (see https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation, the Linux Tux image example)? Other AES modes produce way longer results, even when the ID is encoded with Base58 or the like.

fast lookup for internal usage. it's significantly faster than a UUID on every DBMS i have ever seen

That's a good reason! I think with UUIDv7, because it's sequential it's indexes are faster than UUIDv4. Still larger than bigints though. I'd like to do a benchmark at some point