Many corporations attempting to get into the massive market of self-driving vehicles. However, unlike other companies tackling the problem with LiDAR, Tesla is using a different approach: computer vision. In this video, I discuss why I believe this will ultimately cause Tesla to win the race for self-driving vehicles.
If you enjoyed this video, please hit the like button and subscribe! Also, please consider supporting me on Patreon at https://patreon.com/casgains
Music used in its respective order:
https://www.youtube.com/watch?v=e6nXweiNaeI
https://www.youtube.com/watch?v=RXLzvo6kvVQ
https://www.youtube.com/watch?v=kqh0r8lEo5M
https://www.youtube.com/watch?v=RTGEoh-vPIc
https://www.youtube.com/watch?v=Chh7IleTELM
https://www.youtube.com/watch?v=RTGEoh-vPIc

Unlike all the other car manufacturers and technology companies, Tesla operated self-driving software completely through computer vision, which is an approach where tesla uses different radars and cameras in order to detect and read the multitude of different signs and signals on the road. When us humans drive a vehicle, we use our visual cortex in order to spot what type of object we're looking at. For example, these variations of the letter e have totally different pixel locations, but our brain is able to figure out that that letter is an E with Tesla's computer vision technology. They use what's called neural networks, which is a set of code that is able to figure out what a stop sign is, despite the different variations and Lighting's angles and color the case.

On the other hand, almost every other company in the race for self-driving is attempting to use lidar, which stands for light detection and ranging essentially, what lidar does is shoot lasers in every possible direction and return data on how far away the nearest surface is. These tools are both expensive and unnatural. After all, as human drivers, we rely solely on our vision in order to drive safely. The different signs and signals on the road are built for humans to see it not lasers.

Elan lost has once said that anyone who uses lidar is doomed lidar is is a fool's errand and anyone luck, relying on a lidar is doomed doomed in this video. I'm gon na go into depth as to why this statement may be true or false. First we're gon na start with the shortcomings of lidar. Clearly, many companies have already built lighter power vehicles, but the issue is that these companies are not able to solve certain shortcomings of lidar technology and it's likely that they never will or will take far longer than Tesla to solve these problems.

Latta only works in ideal environments, one of the biggest advantages of Tesla is that it's computer vision approach is adoptable to different environments. So when we come to an intersection, we encounter it. Basically, for the first time we come to an intersection. How many lanes are there left right center? Which way should I turn what are the traffic lights, which lines didn't control all the just done just from camera feed a camera vision alone, and so this is a bit of a harder problem.

But of course, once you actually get this to work, we can also deploy up to the millions of cars that we have globally. Unlike the computer vision approach, vehicles that use lighter have to rely on high-definition 3d max every time a company like way mo releases a video displaying their technology. It's usually in a place where there are ideal conditions, for example, there's heavy reliance on GPS maps. The vehicle has already driven on the route with the real driver.

Sometimes there are items attached the traffic lights so that the vehicle can react to those. However, in a view-all scenario, it's simply just impossible to have an ideal condition every time. For example, if a construction crew did a road change in the GPS map is not updated, then the vehicle may go into an accident. Lidar fails to determine what an object actually is.
The lidar system is able to find out what type of sizes and shapes the nearby objects are through its laser shootings. However, it fails to determine what those objects actually are, which can be incredibly troublesome if a plastic bag was approaching a vehicle powered by lidar. The vehicle will believe that the plastic bag may be a hazard, no slam on the brakes. On the other hand, Tesla has plenty of data on the different angles and Lighting's of plastic bags and can determine that the plastic bag is harmless.

Lighter is far too expensive. Despite google, bringing down the lidar instruments, production price to $ 7,500 vitam technology is still far too expensive for customers to want the technology over Tesla's Hustlas full self-driving option currently costs $ 7,000, whereas Google's lidar instruments which we can't forget are designed, the profit would cost An estimated fifteen thousand dollars, if brought into production because of these shortcomings with fight art, we're constantly seeing automakers attempt to release their technology in geofence areas. The CEO of Ford, Jim Hackett, spoke for the company by stating that we overestimated the arrival of autonomous vehicles. It's applications will be narrow what we call geofence, because problem is too complex.

Overall, the lighter technology isn't horrible, but it just doesn't work for autonomous vehicles. In fact, you don't must even talk about how he used lighter technology and SpaceX for docking. I should point out that I don't actually super hit lidar as much as they sound, but at SpaceX it's physics, dragon uses lidar to navigate to the space station or dock, not only that we SpaceX developed its own lidar from scratch to do that, and I spearheaded That effort personally, because in that scenario, lidar makes sense and in cars, is friggin stupid. However, if lighter doesn't work for vehicles, why don't other companies just switch to computer vision? The problem is that these companies don't have that same fleet that Tesla has and as Tesla senior director of AI has said, it's very difficult, if not impossible, to generate a computer vision autonomous network without having a lot of data.

This is because of the way Tesla is able to overcome inaccuracies in the system. The three points that I really tried to drive home until now are to get neural networks to work. Well, you require these three essentials. You require a large data, set, a very data set and a real data set, and if you have those capabilities, you can actually train your networks and make them work very well, and so why is Tesla? Is such a unique and interesting position to really get all these three essentials right and the answer to that, of course, is the fleet we can really source data from it and make our neural network systems work extremely well.
So let me take you through a concrete example of, for example, making the object detector work better, to give you a sense of how we develop these in all that works, how we iterate on them and how we actually get them to work overtime, so object. Detection is something we care a lot about, we'd like to put bounding boxes around, say the cars and the objects here, because we need to track them and we to understand how they might move around. So again we might ask human annotators to give us some annotations for these and humans might go in, and my tell you that ok, those patterns over there are cars and bicycles and so on, and you can train your neural network on this. But if you're, not careful, the neural network hole will make miss predictions in some cases.

So as an example, if we stumble by a car like this, that has a bike on the back of it, then the neural network actually went when I joined would actually create two deductions. It would create a car deduction and a bicycle deduction and that's actually kind of correct, because I guess both of those objects actually exist, but for the purposes of the controller and a planner downstream. You really don't want to deal with the fact that this bicycle can go with the car. The truth is that that bike is attached to that car, so in terms of like just objects on the road, there's a single object, a single car, and so what you'd? Like to do now is you'd like to just potentially annotate lots of those images, as this is just a single car.

So the process that we that we go through internally in the team is that we take this image or a few images that show this pattern and we have a mechanism, a machine learning mechanism by which we can ask the fleet to source us examples that look Like that and the fleet might respond with images that contains those patterns, so as an example, these six images might come from the fleet, they all contain bikes on backs of cars and we would go in and we would annotate all those as just a single car And then the performance of that detector actually improves and the network internally understands that hey when the bike is just attached to the car, that's actually just a single car and it can learn that given enough examples and that's how we've sort of fixed that problem, I Will mention that I talked quite a bit about sourcing data from the fleet. I just want to make a quick point that we've designed this from the beginning with privacy in mind and all the data that we use for training is anonymized. Now the fleet doesn't just respond with bicycles on backs of cars. We look for all the thing.

We look for lots of things all the time. So, for example, we look for boats and the fleet can respond with boats. We look for construction sites and the fleet can send us lots of construction sites from across the world. We look for even slightly more rare cases.
So, for example, finding debris on the road is pretty important to us. So these are examples of images that have streamed to us from the fleet that show tires cones plastic bags and things like that. If we can source these at scale, we can annotate them correctly, and the neural network will learn how to deal with them in the world. Here's another example: animals, of course, also a very rare occurrence, an event, but we want the neural network to really understand.

What's going on here that these are animals, and we want to deal with that correctly. So, to summarize, the process by which we iterate on neural network predictions looked something like this. We start with a seed data set that was potentially sourced at random. We annotate that data set and then we train your lab works on that and put that in the car, and then we have mechanisms by which we notice inaccuracies in the car when this detector may be misbehaving.

So, for example, if we detect that the neural network might be uncertain or if we detect that or if there's a driver, intervention or any of those settings, we can create this trigger infrastructure that sends us data of those inaccuracies and so, for example, if we don't Perform very well on lane line detection on tunnels. Then we cannotice that there's a problem in tunnels that image would enter our unit tests, so we can verify that we've actually fixing the problem over time. But now what you do is to fix this inaccuracy. You need to source many more examples that look like that.

So we ask the fleet to please send us many more tunnels and then we label all those tunnels correctly. We incorporate that into the training set and we retrain the network, redeploy and iterate this cycle over and over again, and so we refer to this iterative process by which we improve these predictions as the data engine, so iteratively deploying something potentially in shadow mode, sourcing, inaccuracies And incorporating the training set over and over again - and we do this basically for all the predictions of these neural networks, let me know what you think about Tesla autonomy in the comment section below. If you enjoyed this video, please hit the like button and subscribe. Also, please consider supporting me on patreon in the description below.

I appreciate your support. You.

By Stock Chat

where the coffee is hot and so is the chat

33 thoughts on “Lidar vs computer vision – why tesla autonomy will win”
  1. Avataaar/Circle Created with python_avatars Joe Q says:

    That's not true that LiDAR always need HD map information.

  2. Avataaar/Circle Created with python_avatars Nash says:

    For anyone watching, this video is nonsense. Sorry kid but this goes a little bit past what is clearly a high school understanding of the topic.

  3. Avataaar/Circle Created with python_avatars Pierre Huguet says:

    Camera sensors are extremely cheap with high resolution, they don't send laser or radio signal and wait for it to bounce back. it is a one way trip to the camera sensor including the colors missing with lidar… (ok, sure, it is super fast anyway) I believe, the closer to human experience, the better the system will adapt to Human created situations and environments. Humans are building roads, not robots with lidars. Elon musk might change his mind with lidar as the laser measurements devices on new phones are getting popular and become a complement to cameras. in the meantime Tesla dataset, is closer and tighter to our personal experiences, without adding new senses we never used to drive or or build roads with intention of using it.

  4. Avataaar/Circle Created with python_avatars J A says:

    As a computer science student, this video has fundamental flaw. There is mention of sensor fusion, which combines data from multiple sources. Lidar, camera, etc. This allows to take advantage the pros of each modality.
    Just a reiteration of what Tesla is saying is not analysis.

  5. Avataaar/Circle Created with python_avatars Martin Ng says:

    Will tesla works in new countries where they have no data, or need to start from scratch?

  6. Avataaar/Circle Created with python_avatars Chris Keo says:

    Lidar isn't the sole vision being used. It's complimenting CV.

    This video is basically saying I don't want airbags in my car because I have seatbelts.

    All the tech is still in progress. Tesla has been promising autonomy and still hasn't delivered. Such a presumptuous video.

  7. Avataaar/Circle Created with python_avatars Ravi Teja KNTS says:

    I don't even understand how brainless this guy is. Most of the other companies use Camera Vision just like Tesla. They are having Lidar as an additional advantage. You might have just a pen which is great, but there is nothing to find fault at the one who is having both pen and pencil.

  8. Avataaar/Circle Created with python_avatars Paul says:

    The Tesla Model 3 has 21 sensors and only 8 are cameras. You're whole argument is based on the premise that Teslas only use cameras, which isn't true. LIDARs have the highest accuracy depth data and that data can be used with very little compute power. As they come down in price, they will provide very useful data to any autonomous platform, including Tesla's.

  9. Avataaar/Circle Created with python_avatars Frank Wiersma says:

    Lidar is the best way to develop an FSD system, because it provides reliable data. It is relatively easy to derive depth data from images instead to replace Lidar later on with other sensors. But that can be done in an optimization phase when the AI system has been developed.
    Tesla's cheap approach is mostly aimed at keeping stock holders happy, but will not allow them to develop full FSD. Their data is too messy and requires a lot of manpower to be labeled. The approach Waymo has taken to avoid this expensive labeling is to use mapped areas.
    So Tesla is basically trying to take a shortcut to develop FSD, something that is most likely prone to fail.

  10. Avataaar/Circle Created with python_avatars t3knoman00 says:

    lidar is essentially train running on virtual rails, and now i understand why AI is being so heavily invested as.

  11. Avataaar/Circle Created with python_avatars Suresh Chowta says:

    People are so dumb. Just quack quack what a richest man on earth speaks. That road construction issue is bullshit. Lidar has the same issue as a real camera or vision processing has in case of closed roads. Atleast waymo has maps integration which can easily identify a different route on the fly. Why do people think the lidar + AI + machine learning can't solve the issue. Lidar is nothing but a picture with depth. If the cost of lidar reduced to 10% in five years, it could be the same again in another five years and also who knows it could be too small in size.

  12. Avataaar/Circle Created with python_avatars Martin Day says:

    It's not Lidar v Camera – it is Lidar + Radar + Camera v Camera + Radar, Lidar adds to what Camera + Radar can do. The only negative to using Lidar in the mix is that it is currently expensive, but that is like saying the problem with EV cars is that batteries are expensive, they were prohibitively expensive, they are now expensive but in the future they will be affordable.

    All the examples shown are not negatives to using a system that includes Lidar because they have the option to use the best information available to match the problem.

    While testing Lidar they may use mapped areas, but they are used in the same way that stabilisers are used when people first learn to ride a bike, it allows them to advance their development without getting bogged down with limitations that they will eventually remove.

    Often driving in a familiar area will help a human driver, it would be easier to programme a autonomous system to recognise road works knowing that a road that is normally two lane has been reduced to one, otherwise the system has to guess.

    Reflections can cause a vision only system problems that cannot be solved through software. There are everyday occurrences where vision is temporarily blinded such as a low sun or a oncoming vehicle with high beam on, a secondary source of data is going to be essential in order to achieve Level 5, otherwise it is just a clever driver assists device.

    You can be a Musk fan boy, and yet still question what he says because sometimes he is not 100% right.

  13. Avataaar/Circle Created with python_avatars Jack Joe says:

    LIDAR is getting cheaper, just like batteries. And LIDAR sensors are very helpful if you combine them with cameras and radar. Tesla only doesn't use it because they want to be able to update their old cars.

  14. Avataaar/Circle Created with python_avatars T James says:

    And that is where the trillion dollar valuation is coming from ( or more if Tesla decides to start manufacturing autonomous weapons))

  15. Avataaar/Circle Created with python_avatars Bryant Whittaker says:

    So another words trying to come up with a perfect autonomous car would be virtually impossible and that the best driver is you and just like he was saying what may work good and one place might not in another which could potentially become life threatening

  16. Avataaar/Circle Created with python_avatars Lee Swee Huat says:

    I am wondering if Tesla can accelerate the vehicle / computer vision technology by involving 10s of millions of people in training their vehicle / computer vision system. People can be incentivised to train the vision system thru awarding random Tesla souvenirs to participants whenever a small but significant vehicle /computer vision or object recognition milestone is reached? A Tesla vehicle screen emulator can also help to draw people to participate in this vehicle / computer vision training, as well as train people on how to use Tesla vehicles and market the sales of Tesla vehicles?

  17. Avataaar/Circle Created with python_avatars Patrick Sokoloski says:

    I hope that Tesla full self driving will be complete by the end of this year and hope that Tesla will have fully autonomous cars by the end of this year. I want a level 5 fully autonomous Tesla, so that I can sleep, text and talk on my phone, be at my destination and so that my friends and I can go to a bar and drink a couple alcoholic beverages and get drunk together and not have to worry about drunk driving, since the self driving car will be the designated driver and drive us home safely.

  18. Avataaar/Circle Created with python_avatars V_Zlfkr says:

    1. Using LiDAR will make the car's design looks like a potato

  19. Avataaar/Circle Created with python_avatars M D says:

    Would all of you stop with TESLA competition?
    TESLA HAS NO COMPETITION AND IT NEVER WILL.
    GET OVER IT!!!!!!!!!!!!!!!!!!!!!!

  20. Avataaar/Circle Created with python_avatars Rongbing Mu says:

    I don't know how ridiculous you sound and honestly it doesn't matter, I work at Waymo and we have much better computer vision system than Tesla. Tesla = vision alone + no lidar + no hd map, Waymo = vision + lidar + HD Map. Tesla is being irresponsible pricks to its customers, it has fuckign 5 fatality on it autopilot!!!

  21. Avataaar/Circle Created with python_avatars Nukul Khadse says:

    Tesla uses neural networks to predict the distance between the object and the car. I know deep learning has advanced a lot in recent years but finding distance between objects accurately using just camera vision is too challenging at the moment as there are no solid results on the research out there. But tesla has very good chances to succeed as they have a superb team working on this under the guidance of andrej karpathy. And once this is successful many other applications will revolutionize where finding distance between objects is significant.

  22. Avataaar/Circle Created with python_avatars Nukul Khadse says:

    Which mechanisms do they use to detect inaccuracies in the model that is deployed in the car? According to me this is the most complex thing that they have developed. According to me, for detecting inaccuracies human intervention is necessary and it is difficult to rely on a machine to judge false positives.

  23. Avataaar/Circle Created with python_avatars Michael B says:

    Hate PayPal and patron. Please give a coffee or YouTube join alternative.

  24. Avataaar/Circle Created with python_avatars ateendra kumar says:

    Many of your claims are false. Those who use Lidar are also using computer vision

  25. Avataaar/Circle Created with python_avatars Nikolay Tonev says:

    Actually, both technologies are completely ready to take over all the driving, IF we were able to replace all human drivers at once. Both technologies will navigate the roads much safer and better than humans, but only if they have to deal just with other self-driving cars, and no human drivers.

  26. Avataaar/Circle Created with python_avatars Paweł Frydlewicz says:

    Sorry but tesla is a pis of trash, and nothing more their self driving system is worst then volkswagene i don't understand why the CEO of volkswagene sed that tesla is beterr propably he don't know whot volkswagen have or tesla and the gay Eleonor musk try to scared the CEO of Volkswagen, i am litlbit disposed of that situation but Eleonor musk will regret it for sure, ID will crush tesla in all parameters. No one should buy one of tesla shit car ever.

  27. Avataaar/Circle Created with python_avatars Paul Sunday says:

    I think this is in part authority bias in play. The fact that Elon says something is trash seems to be the nail in the coffin on that matter. I remember when he damned Hydrogen fuel cell for cars too.
    I don't know either LIDAR or computer vision so well, but I don't it think LIDAR is a failed tech. The people using it are not stupid. They know to an extent what they are doing & they know they can improve on with feedback. Tesla fan boys everywhere

  28. Avataaar/Circle Created with python_avatars Nine2Nine says:

    I don't know of a single self driving car company that isn't using cameras as well as lidar (except Tesla). This is why the issue has never been lidar vs camera. Instead the issue is whether we can achieve the same level of safety without lidar as with it. Of course we can't. Cameras will never be as accurate with distance as lidar. The whole idea of using mulitible sensor types is to overcome the shortcomings of each.

  29. Avataaar/Circle Created with python_avatars alex Lo says:

    So if Tesla want it self driving software more efficient, Lower their price make more people affordable. Then they will have much more data set and various training data to train their software intellectual.

  30. Avataaar/Circle Created with python_avatars bertobertoberto3 says:

    “Companies that use lidar have to rely on high def 3D maps” what are you talking about dude?! There’s no causal relationship there. And for your information Tesla also depends on maps, just not high definition. Actually they have a vast fleet of vehicles and using them to map global roads is such an obvious move they should have the best maps!

  31. Avataaar/Circle Created with python_avatars bertobertoberto3 says:

    Not convinced cameras are enough. In a recent presentation they said with cameras and computing they’re nearing lidar sensor capabilities. To me that sounds like lidar is the way to go. It’s quite a bit more expensive and that seems to be the major reason Tesla doesn’t want to use it. It’s not a technical challenge, after all it’s just a different set of sensors. Additional, other companies are using Lidar in ADDITION to cameras and radar. The only advantage Tesla has is data, which is a big advantage for sure, but a lot of problems can be solved with other algorithms not just learning and I think Lidar opens that door. Mobileye for example can use only the cameras to feed their driving policy. They’re one of the leaders in this field but I don’t hear much about them. When a Tesla can match their level of navigation I’ll be more convinced of Tesla’s approach. Anyways I’m still on team Tesla, but let’s be realistic here

  32. Avataaar/Circle Created with python_avatars Brandon Johnson says:

    Tesla uses a vision based system in conjunction with psuedo-LiDAR techniques to enable autonomy. The combination of CV and psuedo-LiDAR techniques will soon outperform LiDAR. Their approach to autonomy is focused on solving vision. IMO, their approach is the right approach. Time reveals all.

  33. Avataaar/Circle Created with python_avatars mike thek says:

    Tesla FSD uses forward pointing RADAR as well as cameras.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.