There’s an attack vector in there somewhere.
Kinda struggling to think of what, beyond the well understood risks of using password-based SSH at all. But that's easily ameliorated by sticking it behind Wireguard or something similar. I think this is a pretty welcome change vs turning off FV entirely which I've had to do with Mac servers in the past.