How to secure your new VPS: a step-by-step guide

I'd strongly suggest adding:

  echo "sudo ufw disable" | at now +5 minutes

as an emergency recovery mechanism before you first run

  sudo ufw enable

That way, if you've screwed up and locked yourself out with your new firewall rules, you can just wait 5 minutes and log back in (instead of paying for remote hands at your datacenter, or blowing away your vps and rebuilding from scratch).

Remember to re enable the firewall or stop the at job if everything works for you.