Counterintuitive Properties of High Dimensional Space (2018)

Yea - high dimensional spaces are weird and hard to reason about... and we're working very frequently in them, especially when dealing with ML.

Luckily if you do enough math it becomes much easier to reason about such spaces

- How do you even visualize an 11-dimensional space?

- oh that's easy - you just visualize an N-dimensional space and then set N equal to 11.

I think of high-dimensional spaces in terms of projection. Projecting a 3-dimensional space onto a 2-dimensional space loses information and the results depend on perspective. Same with an 11-dimensional space being projected onto a 10-dimensional space.

I find that this metaphor works pretty well for visualizing how a vector-space search engine represents how two documents can be "similar" in N-dimensional term-space: look at them from the right angle and they appear close together.

Yeah, stopping that need to visualize everything is one of the mechanisms usually adopted for working in high-dimensional space.