←back to thread

192 points beedeebeedee | 2 comments | | HN request time: 0s | source
Show context
peterkos ◴[] No.41900587[source]
I'm reminded of a time that an intern took down us-east1 on AWS, by modifying a configuration file they shouldn't have had access to. Amazon (somehow) did the correct thing and didn't fire them -- instead, they used the experience to fix the security hole. It was a file they shouldn't have had access to in the first place.

If the intern "had no experience with the AI lab", is it the right thing to do to fire them, instead of admitting that there is a security/access fault internally? Can other employees (intentionally, or unintentionally) cause that same amount of "damage"?

replies(12): >>41900622 #>>41900627 #>>41900641 #>>41900805 #>>41900919 #>>41901069 #>>41901814 #>>41903916 #>>41909887 #>>41910021 #>>41910134 #>>41910235 #
EE84M3i ◴[] No.41900919[source]
I'd like to learn more about the AWS incident, but when I google "us-east1 intern" I get this comment. Do you have a link?
replies(1): >>41902508 #
1. rafram ◴[] No.41902508[source]
Probably this: https://aws.amazon.com/message/41926/
replies(1): >>41910064 #
2. donavanm ◴[] No.41910064[source]
No. That was operational modification of system state using existing tools. The “miss” was an intended subset filter that was not interpreted correctly.

> an authorized S3 team member using an established playbook executed a command which was intended to remove a small number of servers for one of the S3 subsystems that is used by the S3 billing process. Unfortunately, one of the inputs to the command was entered incorrectly and a larger set of servers was removed than intended.

As of a while back that entire state management subsystem, which dates from the very beginning of AWS, has been replaced.

Source: me. I was oncall for (some of) the incident management of that event.