Maybe SMR's, thorium, 4th gen, etc will work out, but maybe not.
That's pretty niche, though. Think about trucks, tanks, aircraft, generators for outposts, etc. It might be cool if you could safely package a zillion nuclear reactors for those use cases, Terminator style, but I'd guess that reactors are a better fit for centralized, permanent power generation.
Oh, here it is: https://media.githubusercontent.com/media/openmsr/msr-archiv... page 24/139 budgets 48400 "pounds" for "reactor and reactor shield", which is 22 tonnes, about 110kW/tonne.
The smallest nuclear submarine was NR-1, which had a total displacement of 400 "tons": https://en.wikipedia.org/wiki/American_submarine_NR-1 so the reactor must have weighed less than that.
The 10MW version of SSTAR was supposed to weigh 200 tonnes, 50kW/tonne, while the 100MW version was 500 tonnes, thud 200kW/tonne: https://en.wikipedia.org/wiki/Small,_sealed,_transportable,_...
A 4.95-kg americium thermal reactor design outline has been published: https://www.researchgate.net/publication/239521070_The_Small...
Arleigh Burke class destroyers have 80MW of propulsion and displace 8300 to 9700 tons, compared to which the SSTAR's 500 tonnes is almost insignificant. So weight isn't an issue for ships, and weight doesn't get ridiculously high until you're down below the megawatt scale.
So, you may be right that existing proven reactors won't scale down to a single truck or tank. There isn't a known physical reason it's impossible, or even impossible to do safely, but it hasn't been achieved.
Probably you are right that many small reactors would be more dangerous, but warships exist so that they can go into dangerous situations. You have to weigh the risk of a reactor problem against the risk of being unable to fight because you have no fuel. And we've certainly seen that many militaries have little concern for sailor safety.
Despite all this, no navy has switched all their ships to nuclear fuel. The only explanation I can come up with is that it's unsustainably expensive.