Amazon Go

I really wouldn't trust 'picked up off the shelf' detection, not without the whole thing looking like a giant vending machine. RFID tags on products probably works better.

I don't think the system works like that.

From my first quick take of the video, the app+turnstyle is used to identify you to the store. The video system then tracks your position as you walk around.

When you walk out, the items are recognized and tallied by a large RFID sweep. Funneling you back out through a turnstyle makes sure the vision system knows it's you. Notice that you don't need to barcode yourself on the way out, and the exit system is phone agnostic (it's not checking for an NFC or Apple Pay tag or anything).

The whole "tracking individual items as they come on and off the shelves" task is a very complex thing. But tracking bodies as they walk around a 1,500 square foot room isn't that hard.

I can't help but picture Indiana Jones swiping the golden idol from the pedestal with one hand, and quickly replacing it with a bag of sand using the other hand.

Not as crazy as it sounds. With RAIN Rfid you can get the xyz location of a tag from ~30ft away accurate to 6" in realtime.

Examples: https://www.youtube.com/watch?v=p1LykdRWTfk https://www.youtube.com/watch?v=-hCA8L7v-R8

I think RFID tags are too expensive to put on every product. I may be wrong on this.

I have a feeling you're wrong -- I see RFID tags on almost everything these days, even tiny cheap products.

RFID has some privacy concerns, since the tags continue work when you are outside the store and somebody can scan what you are carrying.

Using these as EAN barcode replacement also has the problem that the system can't tell if you bought the product from Amazon GO or if you just happen to be carrying it with you when you enter the store.

Quick search on Alibaba shows that on volume these are just $0.01 each (and probably lower when you buy larger volumes). Of course sticking these to each product is inconvenient and I would assume the tags are not yet built in in most grocery product packaging.

https://www.alibaba.com/product-detail/China-Professional-ma...

Yep, in smallish bulk (low 1000s), you can get them for about 10-20c a tag (and these are "self printed" ones you can print in-store as needed via dedicated RFID printers) (e.g. https://www.atlasrfidstore.com/rfid-label-4x2-for-the-zebra-...)

I am sure that amazon/walmart/tesco/whoever will need more than 1000 at a time and so obviously get a larger discount. Probably down to well under a penny or two - starts to make a lot of sense when you factor in the other benefits (lower staff, easier stock-tracking/taking etc)

RFID probably works better if you control the packaging, for instance, for deli counter products. But for packaging that you don't control, you either have to slap an RFID tag on when the items get to the store, negotiate with suppliers to start to include RFID tags in their packaging for items shipped to your store, or arbitrarily limit yourself in terms of which products your store may sell. Not to mention that, if your store slaps on RFID tags itself, it raises the OpEx of running the store as well as needing to come up with tamper-resistant tags, because somebody would be able to steal a product by discreetly removing a tag and then walking out of the store.

In essence, RFID tag technology already exists, and there's a reason why it hasn't taken off in grocery stores.

RAIN Rfid tag is < 10cents.

Also, since the app is still on your phone there is no reason they can't be using it to gather data as well.

> because somebody would be able to steal a product by discreetly removing a tag and then walking out of the store.

That's already handled in grocery -- people can mess with regular barcodes if they want. it's a known cost of business

Also never underestimate Jeff Bezos' willingness to lose money in the short term.

1% is a decent average margin in a grocery store (http://smallbusiness.chron.com/industry-standard-gross-margi...).

So, it would require a $10 item to break even on a €0.10 tag.

> When you walk out, the items are recognized and tallied by a large RFID sweep.

But the video clearly shows the items being recognized as someone takes them from a shelf (and puts them back). The items don't need to be recognized at the time you walk out, the store just needs to know that you've walked out.

I don't think that helps in the use case presented here.

What is really needed is a way to talk to multiple RFID tags quickly without any crosstalk. Think of a handbag with a dozen candy bars in it. How do you scan them all quickly without missing any?

The goal has always been to scan something large in one pass, say a shrink-wrapped pallet of items that could number in the hundreds or thousands. Obviously this is technology that Amazon could benefit from as well, I wouldn't be surprised if there's a lot of technology overlap here.

You mean that floating list in the air?

I think that's just visually giving you an idea of what's happening. Like I said earlier, I'm giving a simple naive presentation of how the system might work. Or, at the least, how I would design it without dealing with finicky shelf sensors. Ask anyone that has ever worked in a hotel with in-room minibars how well those things pan out in real life.

Unless that store is just a mockup and not how it really looks and works, I see nothing on those shelves or in the sky above it that is watching you put that cupcake back on the shelf.

The real way to find out is if someone in the demo video can pull their phone out of their pocket mid-trip and see their current inventory and total price. But I didn't see that in the video.

The girl in the video at 1:26 looks at her list and total, but after she's been through the exit turnstyle.

They did say sensor fusion. I'm not sure if they have indoor location tracking in the app but it seems reasonable.

Agree with you, would be much easier to just checkout once when you leave.

The video makes it look like it's constantly up to date, but that sounds quite complicated (despite dropping the ML, DL buzz words).

I don't think you can make that conclusion. Presumably this store will be saving money on cashiers by instead using RFIDs.

Also the cost of labor and/or machines to stick them on.

And if they know who you are because of your phone, and they are doing inventory on a really regular basis (every 6 hours, maybe even less), they might actually be able to track down who it was.

The article I quoted claims 1% is the _gross_ margin, in which case my argument is valid. I now think that is incorrect, as other high-ranking Google results claim it is the _net_ margin.

> How do you scan them all quickly without missing any?

Most RAIN Rfid readers have a read rate of 800+ tags/second. Door portals are a solved solution and now phased arrays are on the market which scan all items in realtime. Take a look at the impinj marketing material for more information. For example the RS2000 chip: http://www.impinj.com/products/reader-chips/indy-rs2000/

But if using RFID saves them more than the cost of the technology its a win.

Basically if Savings from reducing sales persons > Cost of New Technology, you'll add the new tech.

Bring a Faraday's cage, or rather Faraday's bag. Use it once you've paid to store your goods.

Given the turnstiles that the users entered/exited from I'd have guessed there is RFID in play too.

Someone can already walk around a grocery store and see what's in my cart or they can just watch me checkout. Buying groceries isn't exactly a private experience unless you buy your stuff online. Securing tags in things like a passport or credit card is important since identity theft is real and hardware to read the tags is cheap. Knowing what things I buy in the grocery store might help some marketers know what sells but I can't really think of any nefarious use of that kind of data.

Maybe you could use it to make some guesses about a person. An older guy buying frozen dinners, beer, and not much else is probably single or divorced. A woman buying prenatal vitamins is probably pregnant. Someone buying both Special K and Fruit Loops might have a family. But someone can already make these determinations from using their eyes. RFIDs might make it a bit easier but don't really add any new infosec facet to this grocery store experience.

> You mean that floating list in the air? I think that's just visually giving you an idea of what's happening.

A friend at Amazon said he tried to trick it by taking two items at a time and it got it right every time, so yeah it does appear to track when you take the items.

Most libraries RFID all their books now so you can just drop them on a scanner that detects all the books you have, swipe your library card and go.

I don't think that's actually what Amazon Go is doing though.

I would assume RFID would be more accurate to begin with. Seems like there's a lot of room for errors with their proposed technology.

BUT what if their goal isn't to make razor thin margins at grocery stores, but to test out their tech on human tracking. You've got to believe that they have a more long term plan than making a better self-checkout.

They can now tie in your Amazon Go app with you as a person. Most likely all the sensors and cameras in the store will be both tracking you via your app with Bluetooth, but also perhaps body heat sensors and the like that track you based on your unique body pattern.

Then they can tie this in with your Amazon Prime account to better improve recommended purchases to you, they can tie this in with your Prime Now account and your home address and more immediate delivery needs. Drones, Kindle, Echo as well.

The goal is deeper analytics and tracking on individual humans. That's how Facebook makes their money, and Amazon can perhaps have deeper data on individuals than any other company tracking browser cookies.

That's a definite plus. I was just saying that the long-range position tracking might not be as important.

Did the system know before checkout, or after?

Alibaba sellers will list almost anything for $0.01 each, though, regardless of actual selling price.

It doesn't have to be strictly cheaper, as it also adds convenience. Not having to wait at the checkout will be worth something to customers, possibly quite a lot in the case of a shop targeting richer customers.

Of course it's trivial to tell. When individual items are tracked, it's exceptionally clear if the item is from the store or not.

> Someone can already walk around a grocery store and see what's in my cart or they can just watch me checkout.

The problem with electronic surveillance is never the surveillance part. It's the part where we can surveil billions of people at once without the need for billions of spies.