> I've been putting off digging into AWS for years now
In my opinion people end up in these billing situations because they don't actually "dig in" to AWS. They make their pricing easily accessible, and while it's not always easy to understand, it is relatively easy to test as most costs scale nearly linearly.
> the rational fear of waking up to massive charges.
Stay away from the "wrapper" services. AWS Amplify, or Cloudformation, or any of their Stack type offerings. Use the core services directly yourself. All services have an API. Getting an API key tied to an IAM user is as simple as clicking a button.
Everything else is manageable with reasonable caching and ensuring that your cost model is matched to your revenue model so the services that auto scale cost a nearly fixed percentage of your revenue regardless of current demand. We take seasonal loads without even noticing most years.
Bandwidth is the only real nightmare on AWS, but they offer automatic long term discounts through the console, and slightly better contract discounts through a sales rep. Avoid EC2 for this reason and because internal bandwidth is more expensive from EC2 and favor direct use of Lambda + S3 + CloudFront.
After about 3 months it became pretty easy to predict what combination of services would be the most cost effective to use in the implementation of new user facing functionality.