Tesla Optimus Bots Were Remotely Operated at Cybercab Event

To be honest, the autonomous control of the robot seems like the easier part of the equation. (doing it safely in a room with guests, unguided... thats another matter). The physical limitations and packaging are a big challenge, and I dont think I saw Optimus lift anything remotely heavy.. just pull a beer tap.. a decision that probably speaks volumes about current limits of the technology.

To apply my first point to reality: put an Optimus in its current state/capability, on a commercial 0-turn lawn mower, plug Optimus into the mower's power takeoff, and have someone in another country remotely pilot the mower. That right there is worth every commercial lawn service having at least one on their crew TODAY.

The appeal of hot swapping an operator real time on the equipment you already own, whether it's a push lawn mower or a huge mining truck, provides enormous value right out of the gate. Especially in tasks where the Optimus can handle 90% of the task autonomously but needs to step aside or oversight for the last 10% of the job. Compare to a business model that requires purchase of all new very expensive and unique equipment.

> To be honest, the autonomous control of the robot seems like the easier part of the equation.

I agree but it is frustrating watching Elon like Michael Copperfield but thinking it is real like a 4 year old.

I don't see a clear advantage of Tesla against other competitors if he will launch it in a couple of years.

I've worked in robotics for over 10 years, at state of the art labs and high quality startups.

There are really only two hard problems in robotics: Perception and Funding.

Perception, especially around a bunch of people, with depth, mapping, understanding traffic and gestures, all in real time etc etc will be a huge problem for these machines for a while.

Funding though? I doubt that's an issue right now.

What stood out to me is that the speed of the movements makes it clear the autonomous balance control just isn't there. You can simulate this: try to move your upper body while standing up without changing the balance or stance of your legs and hips - unless you move slowly you can't do it, whereas if you put any force or momentum behind it you'll feel yourself straining pretty hard to stay upright.

When you see the bits and pieces of behind the scenes for Boston Dynamics it's clear that's where a lot of the magic actually is (and also if you look at how say, Atlas moves) - by necessity it looks much more "natural" because to get any power or speed behind the motions the whole robot needs to actively compensate the movements (obviously having enough power behind the drive system to actually do it is also critical).

I would imagine latency would be an issue if companies were considering teleoperation using staff in a country with cheaper labor. For example I work with people in India and China and they regularly complain about the several hundred ms of latency they get when using their American VDIs. That off the shelf lawn mower is going to be hard to control safely with all that delay, and there's also the risk of connection drops and the like. You would need a specialized mower with collision detection/etc. to handle this, and at this point you might as well discard the robot and just have a remotely operated mower instead.

However there are cases where this can work well, say in a factory handling dangerous chemicals with the teleoperator in an adjacent room. Or maybe it's doing some sort of task where delays and connectivity loss are acceptable.

Let's see, New York to Mumbai over the Earth's surface is maybe 12,500 km, assume a direct fiber optic cable where light travels noticeably slower than in a vacuum at 200,000 km per second... So a minimum of 62.5 ms one way with the best terrestrial equipment.

While one can play network games at 125 ping, it relies rather heavily on tricks that only work in a virtual environment. (Back in the '90s I used to play with 300 ping, no latency compensation, uphill both ways.)

Don't get me wrong I think this guy's idea is incredibly stupid. But, have you ever operated a mower? They're not fast. A few hundred ms of latency on a mower is no problem at all.

That surprises me. I thought motion planning and motor control would be harder - old memories of Asimo falling helplessly trying to climb stairs, the clunkiness of a robot aligning itself perfectly with a drawer before executing a scripted-looking action to pull the handle, the obvious recorded sequence Atlas uses to get up from a fall. I know Boston Dynamics does impressive acrobatics, but it's all legs and no arms.

Are kinematics and planning solved now? I want to move into the field so I'm trying to learn.

Realistically it's in the several hundred range. I just did a ping using Vultr's Looking Glass from New Jersey to Mumbai and got around ~240ms on commercial fiber. For people working from home in India (with cable/DSL overhead + distance from the IX) connected to servers in LA I regularly see 300-400ms.

Also keep in mind in a VDI or teleoperation setting there's not only network latency but additional delay from the video encoding, compression/packetization, and decoding on the other side plus a bit of buffer. Honestly I think cloud gaming is a good test case for this - and in my experience that only works well when you have fiber and have the game server in the same city as you (basically <10ms).

I'm also a roboticist. Perception and funding are hard. But don't forget battery energy density, and the power-to-weight ratio and energy efficiency of actuators. Also very very hard, and Moore's law helps not at all.

Autonomous cars are in a nice niche since they store vast energy for actuation anyway, it's OK to be heavy, and the controls are relatively simple. They are limited by perception and decision making.

Humanoids are way more limited by energy storage and actuation. Animals are absurdly efficient.

David Copperfield is the magician’s name, if that’s who you meant

> Asimo falling helplessly trying to climb stairs

IIRC, that wasn't a control problem but a mechanical failure of a gearmotor shaft.

Tangential question: are there any actuators out there that mimic the animal muscle tissue, i.e. swelling laterally in order to shorten a tendon and pull a joint? This seems like a very elegant method compared to servos with all sorts of slack and rigid positioning. I'm not a roboticist so I'm not familiar with state of the art in actuators.

They exist, but they're inefficient compared to ordinary motors.

Battery density is only an issue if these things are spending most of their time moving long distances. If you are targeting a drop in replacement for a human worker who is spending most of their time at a workstation, it can be plugged in while working. Even in a scenario where the robot can not be connected to power while working, that's easily solved with redundancy - get two robots, one works while the other charges. Obviously better battery life is a nice to have, but it's not an impediment to large scale adoption the way other big robotics problems are.

> easily solved with redundancy - get two robots.

Yay, twice as expensive.

And power tethers on robots suck so hard. Try it sometime, you’ll hate it.

Yes, my fault! Thanks! David Copperfield, the illusionist.

My point is that if you want to make a fully functional android last longer, have it take bathroom breaks and change out its lithium backpack.

If you want to make a energy-unconstrained robot into a fully functional android, you have much bigger, fundamental problems.

Here's some closed-course manipulation with arms: https://www.youtube.com/watch?v=vuG-qNgLHws

There are others. Controls is hard! You need investment and to solve difficult engineering problems. But we have a pretty good idea that those things are solveable and can demonstrate success b/c they are engineering challenges, not things we fundamentally don't have an approach for yet.

And how much more expensive is the twice as efficient power system that hasn't been developed yet?

Nearly all robots in actual use have tethers, it's really not a big concern. Further there are other methods of providing power, such as induction. For any situation where long range mobility is really a concern, you probably don't want a humanoid robot to begin with.

How much of that is actually perception though?

"Where can I put my feet safely?"

"What is the orientation and 6 higher order velocities of my body?"

etc.

I've been told that a perfectly observable / estimable system is trivially controllable. It's one of the reasons I believe perception is upstream of everything - interaction dynamics alone are nearly impossible to just wave away with models.

I don't even work in perception. But I know that everything is fine until you try to go online with perception in the loop. Then you are behind the perception team's debugging nearly all the time.

A problem that can be solved by spending 2x is not the type of fundamental problem I'm referring to (easily solved by "Funding" or otherwise known as a system design constraint and part of everyday engineering albeit very difficult and skillful engineering)

The leaps that would be required to make a mannequin with motors intelligently interact with crowds (in groups no less) at a publicity event cannot be solved with 2x funding jumps, and I'm arguing they are largely perception-, sensing-, mapping- and self-modelling- based.