Working in the terminal gets you to make the best use of a single most efficient input source, the keyboard. Which, coincidentally, is what you need for coding. In visual ides (pretty much all of them), some actions can only be performed by using a mouse (because, well, it exists). In editors like nvim that's not the case and everything is accessible via keyboard, usually in some pretty logical and structured way, so that eventually you get used to longer key chords etc for doing various things.
That, plus the terminal (itself) is at your disposal because, uh, you're in it.
That, plus vim emulation in all major ides be it vscode or jetbrains is pretty wonky and not comparable to the real thing.
Not much to do with 'true hackers', it's sad there's people who actually think that.
(all of the above being purely subjective ofc; this can be an infinite argument)