Magma: A foundation model for multimodal AI agents

Looking at industrial robots they don't mimic how humans do things, and hence, they are efficient. That's why I don't understand how these propsals to teach robots how humans do things will make any sense.

To have robots at homes, they will need their tools to be efficient. It will not be the same washing machine, oven, or dishwasher that we use now, there will be new ones made for robots.

This. The paucity of imagination in the AI space is mind-numbing.

But Humans generalize very well across tasks. You can have an employee driving a forklift, then stop pick-up a pallet that blocks his way and continue.

Because many spaces are built for humans. We already have specialized robots that vacuum our floors, but cleaning the surfaces, removing cables and tidying up is still constrainted to human-shaped entities

Airplanes don't fly by flapping their wings.

You are considering issues of like 0.25x - 4x efficiency gain or loss while ignoring that the real issue is possible vs impossible. If I have a household tobit that can vacuum the floor taking 4x as time as me I don't care. It's still extremely useful. This is why we human-imitatinf approached are used everywhere. It won't be the most efficient solution but the data contains information that helps make it possible at all.

Because of adoption and flexibility.

what do you think how long it takes to align all washing machine manufactures to make a 'Robot Washingmashine'? Also if its optimized for robots, you need a robot to use it.

The progress in ML/AI is so fast, we can easily skip this tedious assumption that we have to form our env for robots instead of teaching robots how to interact in our env.

and no a robot doesn't need to be efficient at all. A robot can do things 24/7. If i'm out of the house and a robot needs 4 hours to clean my kitchen and this doesn't cost me a lot besides of a little bit of energy, i could care very little how long it takes as long as it doesn't take too long.

Fashions in AI.

Everyone is piling on Transformers and Diffusion (and in robotic, humanoids) today; but for most of the history of AI, we've been making things so simple they can only mono-task, and the only way to make commercial sense of that is to be much more efficient (on one of the many axies) than humans.

Now we have models that seem (at least at first glance) to cover the full breadth of what humans can do, so the question has become: can we make them perform at a decent skill level, rather than like someone who is book-smart enough to pass the tests but has almost no real experience of anything.

Yet we shouldn't not forget, we create things to make our lives easier, we create tools that assist us - not the other way around!

Why is it important to be efficient at home for a robot? If you have an industrial robot churning out parts 24/7 in a factory shaving of 3 sec on a 30 sec process is very valuable. But does it really matter if the robot folds your laundry in 3 min or 5 min? Probably not unless you are running a dry cleaner business.

But they could... Its just not made yet.

Could does not mean should, though.

No, I'm not considering speed. I'm considering efficiency. A simple example: I tried to do plastering for a room in my house a few months ago, and as a first-timer, I did a terrible job. There are many things I learned later by hiring a professional plasterer:

- Mix type.

- Mix consistency.

- Mix timing and water amount (which he adjusted as he worked).

- Uneven walls.

These things are based on experience, room type, wall type, wall alignment ... etc. There is no way that a robot will do the same job as a man; it has to be done differently. Using the same space as humans will not make generic robots useful. Your vacuum example is perfect, I have one and had to manually add/remove the water container as the robot will not be able to do that. Even if it does, it has to return to the dock, unload, and start again. A human would remove it at any point and place it somewhere else.

And robots will not do that either, what if the employee used hearing to determine if there is a hazard (another moving vehicle around) before jumping to pick a pallet? How would the robot know by just “looking”? How to prioritise visuals, audio, sense … etc?

Well, difference between 3 and 4 hours meanwhile might be important if robot is noisy.

how would an AI robot oven be different. I as a human want to use AI robot oven.

Yes it actually solved the fun parts of vaccuming and left tedious parts to humans.

When it comes to integration, efficiency isn’t the most important quality attribute (“ility”). Having the interfaces be easily used by humans manually is more important. It’s why http1 has been dominant for so long… it’s easy for humans to understand and manually use without needing complex tooling. Yeah, eventually these things get replaced once machine-to-machine efficiency becomes a real pain point, but it’s far down the road. Not the first thing you try to tackle.

I will happily replace my dishes with steel if a robot can load and unload the dishwasher

Ok but these are going to be humanoid robots, so it makes sense for them to use the appliances that have been designed for use by humans. I don’t really care if tasks like doing the dishes or doing the laundry take 1 hour or 10 hours, as long as they get done.

I don't understand why you think a human-inspired general purpose robot won't be able to learn to deal with the issues in your bullet point list.

We're already seeing in AI that given human examples they can "learn" to act the same way, and a robot wouldn't have to go from never having done any plastering to being an expert plasterer in a single shot. They can be trained once using the expertise of professionals (both explanations, and videos of them doing work) such that they don't make the same mistakes you make on a first attempt - and once trained once, that knowledge can be rolled out to all robots running that software rather than needing to teach each one individually. Hell, even without expert advise in the training, they could even learn how to do it by trying it in a demo room (or a virtual environment) thousands of times until they figure out what does and doesn't lead to the desired end result...

For me doing the dishes is a 10-15 minute chore in the sink with some hot water but most people don't seem to object that their dishwasher takes 3 hours to do the same job with superheated steam or whatever. It still saves them the 15 minutes.

There's no reason to expect models won't be able to handle this even better than humans.

I think a robot that can work with existing machines will be much better than one that requires new, complimentary ones - at least initially.

Don't forget that the dishwasher also uses less water and the dishes get sterilized by the steam.These features may or may not be important for a particular user.

These are orthogonal to form.

Yes but it wouldn't be ideal to have 100s of complex, specialized robots running around at home, if they behave like humans, they can complete tasks _optimally_.

You're not looking at hyper efficiency for most of the tasks anyway, see the robot vacuum for example, they are not quick and are slower than humans but absolutely useful.