←back to thread

Launch HN: Flywheel (YC S25) – Waymo for Excavators

103 points jashmota | 1 comments | 24 Sep 25 16:48 UTC | HN request time: 0s | source

Hey HN, We're Jash and Mahimana, cofounders of Flywheel AI (https://useflywheel.ai). We’re building a remote teleop and autonomous stack for excavators.

Here's a video: https://www.youtube.com/watch?v=zCNmNm3lQGk.

Interfacing with existing excavators for enabling remote teleop (or autonomy) is hard. Unlike cars which use drive-by-wire technology, most of the millions of excavators are fully hydraulic machines. The joysticks are connected to a pilot hydraulic circuit, which proportionally moves the cylinders in the main hydraulic circuit which ultimately moves the excavator joints. This means excavators mostly do not have an electronic component to control the joints. We solve this by mechanically actuating the joysticks and pedals inside the excavators.

We do this with retrofits which work on any excavator model/make, enabling us to augment existing machines. By enabling remote teleoperation, we are able to increase site safety, productivity and also cost efficiency.

Teleoperation by the operators enables us to prepare training data for autonomy. In robotics, training data comprises observation and action. While images and videos are abundant on the internet, egocentric (PoV) observation and action data is extremely scarce, and it is this scarcity that is holding back scaling robot learning policies.

Flywheel solves this by preparing the training data coming from our remote teleop-enabled excavators which we have already deployed. And we do this with very minimal hardware setup and resources.

During our time in YC, we did 25-30 iterations of sensor stack and placement permutations/combinations, and model hyperparams variations. We called this “evolution of the physical form of our retrofit”. Eventually, we landed on our current evolution and have successfully been able to train some levels of autonomy with only a few hours of training data.

The big takeaway was how much more important data is than optimizing hyperparams of the model. So today, we’re open sourcing 100hrs of excavator dataset that we collected using Flywheel systems on real construction sites. This is in partnership with Frodobots.ai.

Dataset: https://huggingface.co/datasets/FlywheelAI/excavator-dataset

Machine/retrofit details:

  Volvo EC380 (38 ton excavator)
  4xcamera (25fps)
  25 hz expert operator’s action data
The dataset contains observation data from 4 cameras and operator's expert action data which can be used to train imitation learning models to run an excavator autonomously for the workflows in those demonstrations, like digging and dumping. We were able to train a small autonomy model for bucket pick and place on Kubota U17 from just 6-7 hours of data collected during YC.

We’re just getting started. We have good amounts of variations in daylight, weather, tasks, and would be adding more hours of data and also converting to lerobot format soon. We’re doing this so people like you and me can try out training models on real world data which is very, very hard to get.

So please checkout the dataset here and feel free to download and use however you like. We would love for people to do things with it! I’ll be around in the thread and look forward to comments and feedback from the community!

Show context
seabrookmx ◴[24 Sep 25 19:04 UTC] No.45364639[source]▶
> The joysticks are connected to a pilot hydraulic circuit, which proportionally moves the cylinders in the main hydraulic circuit which ultimately moves the excavator joints

I've actually spent a decent amount of time running an excavator, as my Dad owns a construction / road building company. It was a great summer job!

An important note about the pilot hydraulics is that they _provide feedback to the operator_. I would encourage any system that moves these controls on behalf of a remote human operator or AI to add strain gauges or some other way to measure this force feedback so that this data isn't lost.

The handful of "drive by wire" pieces of equipment that my Dad or other skilled operators in my family have ran were universally panned, because the operators are isolated from this feedback and have a harder time telling when the machine is struggling or when their inputs are not sufficiently smooth. In the automotive world, skilled drivers have similar complaints about fully electronic steering or braking systems, as opposed to traditional vacuum or hydraulic boosting approaches where your foot still has a direct hydraulic connection to the brake pads.

replies(4): >>45365403 #>>45365849 #>>45366303 #>>45366966 #
opwieurposiu ◴[24 Sep 25 20:57 UTC] No.45365849[source]▶
Yes, and you also get feedback from your butt as the machine tips and wobbles, particularly on smaller machines. Hearing the engine straining helps also. Often you can not clearly see what you are digging, this feedback lets you know if you are running into a rock or something.

One big advantage would be cameras mounted on the boom and rear view cameras, as many machines have obstructed views.

replies(1): >>45366310 #
jashmota ◴[24 Sep 25 21:41 UTC] No.45366310[source]▶
We're indeed streaming the audio and have haptic feedback. My hypothesis is the seat vibration isn't as helpful. It's sub optimal and operators would be far more productive without it. We would do a paper when we have enough data on this. We're also putting more cameras but streaming a lot of them at once is tricky.
replies(2): >>45366735 #>>45377591 #
fragmede ◴[25 Sep 25 19:18 UTC] No.45377591[source]▶
The problem is how do you quantify it and metricize it and get some numbers to compare? Because it's absolutely invaluable after you get to a level of mastery with a machine that you're sitting in that the seat moves in. You just feel it, you can't explain it in words, you're just one with the machine. Your butt really needs to feel that in order for that to really happen. Can you do it without it? Absolutely. Does it make it worse to not have that? Also true. It's extra effort, it's extra cost, and it makes things better for the actual users of the product, but it'll never show up as something you can measure.

Do you want to sell Microsoft Teams to the executives or do you want to give joy to the people who actually use the product?

replies(1): >>45380862 #
1. jashmota ◴[25 Sep 25 23:56 UTC] No.45380862{3}[source]▶
You've touched one of my favorite things to think about. I think about this a lot. There will always be people who enjoy fully analog cars/planes/watches over alternatives. I think that doesn't define the whole market. How many of new joinees are we seeing in this industry? Would you rather have a functioning back because you operate these remotely or go to site and sit in one of these and have a broken back by 50s? This is leaving out the fact that remote operation could save lots of lives. In our launch video we did put together some of the news clips on it: https://www.ycombinator.com/launches/O8i-flywheel-ai-waymo-f...