Amazon Go

I hate it when companies offer a "how this works" section that doesn't actually tell you a damned thing about how it works.

* How does my Amazon account get associated with the items I take?

* How are items detected when leaving the store? If my friend and I walk out side by side, how does it know (if it does) which items are mine and which are hers?

* What happens when someone picks up an item and leaves without first doing whatever check-in/registration/setup is necessary?

Exactly. They say, it uses Deep Learning, Computer Vision and blah blah.. (Add 2 more buzzwords). But, for christ's sake, say how!

Sensor Fusion. Duh.

RFID, image recognition with cameras... what is it? Can someone go and check it out? ;)

* What happens when someone throws a product in my bag. Who pays it?

Its probably too creepy in terms of privacy to detail how it works which in turn will give themselves PR. I can only assume it uses arrays of cameras and sensors with deep learning. Glorious amounts of tracking data.

As a customer-facing thing, "how it works" means "what do I have to do to get stuff". Like, how does the process work on a UX level.

These are purely assumptions based on what I saw on the video alone:

>If my friend and I walk out side by side, how does it know (if it does) which items are mine and which are hers?

Looks like the entry/exit is the same type of set-up you find at most large office buildings with the tap in/out gates (see screenshot: http://i.imgur.com/e7fDglY.jpg). I assume it only lets one person out at a time which also suggests your friend could only enter the store if they had an amazon go account and tapped in themselves.

>What happens when someone picks up an item and leaves without first doing whatever check-in/registration/setup is necessary?

Again, based on those gates, I'd assume you can't actually access the store without going through the necessary set-up first.

Personally, I'd like to know what happens if/when my phone battery dies in-store.

So will they have weighing scales to walk over? And also register eaten candy bars and toilet visits?

How it works sections aren't designed for the technically inclined user they are designed for the layman.

Or if you lose your internet connection. or if your app crashes.

They don't have to get intensely technical but they could say a little more than basically, "magic".

I'd assume the sensors can do a couple things - sense your proximity to your goods (the closest) and sense the difference in time between when you leave and when your goods leave (the smallest). I'd also assume that they use logic along the lines of "your goods are those which have been closest to you (and not on the shelf) for the most time since you entered the store.

I'd assume trying to forcibly enter the store without registration will set off an alarm. I can't wait to show up with 15 of my friends and run amok in the store - it'll present an interesting legal experiment (unless they just get us for trespassing).

Which is already addressed earlier in the video. The "How it works" section of the video adds no practical details on how to use the system. It's just a slot to drop in a bunch of impressive-sounded jargon.

It looks like you don't need to scan anything when exiting. So your phone could die once you're in the store and you'd still be able to check out.

I'm guessing the Amazon Go app generates a one-time-use QR code that gets scanned on entry and then security/tracking cameras follow you all over the store. If the cameras see you in front of an item when its RFID sensor detects it was picked up, they make an educated guess that you picked it up. Then they can re-scan all the RFIDs as you exit for extra confirmation.

Except to a layman it is basically magic. Using X technology we did Y so you can do Z. I'm sure in coming months you'll get the in depth technical explanation you are looking for but this announcement isn't for you it's for the average person.

That's a rather defeatist attitude to have about explaining technology to laymen.

For example (not saying this is how it actually works): "Scanning your phone when you enter lets us know who you are. Then, our advanced vision tracking system follows you around the store and lets us see what you're choosing. Finally, scanners in our turnstiles verify your purchases as you're leaving the store."

yep. full on dystopian tracking was my first thought.

i believe this is it too, mainly because of their mentions of machine learning and computer vision in the post.

I think you meant http://i.imgur.com/e7fDglY.png

My current best guess is, basically what you've suggested. The Amazon Go page says "Computer Vision" which means "Cameras, Cameras Everywhere!". It also says "Sensor Fusion," I bet that means at the very least "Wifi and Bluetooth," with which they'll use to place your location on a virtual map of the store. That location could then be compared with the computer vision object tracking location to "double check" that you really are at that location. And so on.

if you don't notice it and don't put it bag you pay for it, just like in a normal store. That's what I'd assume at least.

This is an advertisement announcement not a technical breakdown. There is no need to even say that much.

Oh. I have to admit I ignored the video and thought the comment was about the "How does Amazon Go work?" paragraph on the page.

They probably don't have RFID sensors on the shelving, and if they did they would only be able to detect when you walk several feet away from the shelf, and would have huge problems if two people were standing right next to each other at the shelf.

It looks like they have a row of cameras along the top of each shelf that will be used to detect when you pick up and place items back down.

The RFID tags are mostly useful at the gates to confirm the visual data and feed back into the machine learning algorithm.

> Sensor Fusion

I wonder, when did the obvious way you use sensors become a buzzword?

Yeah that was pretty terrible. I half expected to hear the word synergy in there somewhere.

Sensor fusion? You mean multiple sensors?

It's not "dystopian" if you're on private property and have accepted their terms of service. You can avoid surveillance by, you know, choosing not to shop there.

If there is no way for me to delete my personal data after i leave the store, i find it distopyan.

If one day all the stores are like this, I guess you may argue I can still grow my own food, and everything is fine.

Nope, get back on narrative! If it involves tracking me it's dystopian and literally 1984, especially if I've voluntarily signed myself up to be tracked

> If there is no way for me to delete my personal data after i leave the store, i find it distopyan.

By "personal data", if you mean the video footage of your shopping, then it will probably be deleted after a few days/weeks. It will just be used to train (reinforce) the machine learning model, and will be discarded eventually. But your shopping history will always be there, as it is now in online shopping sites...

I'm more interested to know how it handles one person taking something off the shelf and handing it to another person.