Tesla is declaring victory on achieving its September milestones for its autonomy roadmap, but we see the most important milestone missing.
Earlier this month, Tesla released an AI product/self-driving roadmap that included milestones the team aimed to accomplish in the coming months.
With September coming to a close and Tesla starting the rollout of Supervised FSD for Cybertruck, Elon Musk sycophants are already celebrating Tesla accomplishing all the milestones:
Ashok Elluswamy, Tesla’s head of autonomous software, and Musk himself commented on the post in approval.
However, as Elluswamy points out, there’s already a small caveat that only some Cybertrucks are currently getting FSD, almost a year after the first deliveries, and the “end-to-end on highway”, which is the use of neural nets throughout the whole highway driving process, is only being shipped on Cybetrucks right now.
Therefore, the vast majority of Tesla vehicles with FSD won’t get it in September as planned.
With that said, there’s a much bigger problem with claiming victory on the September milestone than end-to-end on highways: interventions.
Arguably, the biggest claim in Tesla’s autonomy milestones in September is FSD v12.5.2 achieving a “~3x improvement miles between necessary interventions.”
As we reported when Tesla first released this roadmap, this is a problematic goal in the first place as Tesla has never released any official miles between interventions data, which brings the question: 3x from what?
Without official data from Tesla, we have to rely on crowdsourced data, which Musk himself has previously positively commented on.
We currently have just over 2,500 miles of FSD v12.5.2 crowdsourced data and it shows 35 miles between critical disengagement and 17 miles between disengagement:
Now again, Tesla never specified a 3x improvement from what?
If we go from an average of all the 12.5.1 versions, it shows that v12.5.2 is actually a. regression because over more than 14,000 miles those versions have averaged 174 miles between critical disengagements:
Just in case, we can also compare it with FSD v11, but that also expose v12.5.2 as a regression:
The only clear positive thing about FSD’s improvement that we can gather from this data is that Tesla went from an average critical disengagement every 109 miles with FSD v11 to one every 166 miles on average with all v12 versions combined.
But even that’s a 50% improvement – both “~150%” improvement that Tesla mentioned, which again was for v12.5.2 against who knows?
Electrek’s Take
This moving of the goalpost and constant praise from sycophants is sending the wrong message to Tesla.
I am not a hater. I can admit that Tesla has made some improvements in September. I certainly wouldn’t call shipping FSD on Cybertruck a win after almost a year following the first deliveries, but the new driver monitoring system on FSD v12.5.2 is a big improvement as well as auto-speed controls.
It makes the system more comfortable and useful. It feels less like a task of training Tesla’s self-driving system.
However, it doesn’t change the cold fact that Tesla is nowhere near where it needs to be in terms of miles between disengagement in order to see a path toward a true unsupervised self-driving system, which is what Tesla has been promising for years.
Tesla needs a 1,000x improvement in miles between critical disengagement, and the fact that it is currently celebrating a nonexistent “~3x” increase should be worrying to people.