I think SGI failed to understand that there was a point where desktop PCs would be good enough to replace dedicated workstations. Continuing to make hardware that's much better than the best PCs wasn't going to save them after PCs crossed the good-enough line - whatever they had, would be relegated to increasingly rarefied niches - the same way IBM now only makes POWER and mainframes - there is no point of making PCs, or even POWER workstations anymore for them, as the margin would be too narrow.
SGI could double down on their servers and supercomputers, which they did for a while, but without entry-level options, their product lines becomes the domain of legacy clients who are too afraid (or too smart) to port to cheaper platforms. And being legacy in a highly dynamic segment like HPC is a recipe for disaster. IBM survived because their IBMi (the descendant of the AS/400) and mainframe lines are very well defended by systems that are too risky to move tied to hardware that's not that much more expensive than a similarly capable cluster of generic and less capable machines. As the market was being disrupted from under them, they retreated up and still defend their hill very effectively.
The other movement they could do was to shift downwards, towards the PC, and pull the rug from under their workstation line. By the time Microsoft acquired Softimage and had it ported to NT, it was already too late for SGI to even try that move, as NT was solidified as a viable competitor in the visual computing segment, running on good-enough machines much, much cheaper than anything SGI had.