According to this document (http://i-web.i.u-tokyo.ac.jp/edu/training/ss/lecture/new-doc...) it would appear that directory entries have one extra level of indirection and share space with the page cache and hence can be pathologically evicted if you read in a large number of files; compiling/reading lots of files for example.
On Linux however the directory entry cache is a separate entity and is less likely to be evicted under readahead memory pressure. Also it should be noted is that Linus has spent a largish amount of effort to make sure that the directory entry cache is fast. Linux's inode cache has similar resistance to page cache memory pressure. Obviously if you have real memory pressure from user pages then things will slow down considerably.
I suspect that if Windows implemented a similar system with file meta data cache that was separate from the rest of the page cache it would similarly speed up.
Edit: I should note, this probably wouldn't affect linking as much as it would affect git performance; git is heavily reliant on a speedy and reliable directory entry cache.