Adam Bry on The Robot Brains Season 2 Episode 18
Pieter Abbeel
When you are skiing down a mountain slope or riding a bike along a forest trail, it's hard enough just to concentrate on what you're doing, let alone try to capture video of the moment. But what if you could just take a drone out of your backpack, ask to follow along and film you while it effortlessly avoids any physical obstacles along the way? Today's guest has done exactly that, and in fact has enabled many more use cases for drones. Adam Bry is co-founder and CEO of the unicorn startup Skydio, which produces drones that use AI to take the pilot out of the equation. Adam, so great to have you here with us. Welcome to the show.
Adam Bry
Thank you, Peter. I'm super excited to be here.
Pieter Abbeel
Well, I'm so glad we get to talk about everything you're doing. Let's maybe dive right in, what does Skydio build? What is this Skydio drone and how is it different from other drones?
Adam Bry
So we make autonomous drones. The motivation for starting the company was looking out and seeing all the incredible things that people were starting to do with drones that they wanted to do with drones. You know, in the consumer world, capturing amazing video in the industrial world, there's all these kinds of inspection mapping tasks. In public safety and defense, you can use drones to get really useful information without putting people in harm's way. And then there's also the possibility of doing physical goods delivery. So, you know, the potential of what drones can do is incredible. But we felt like none of this stuff was really going to scale and work the way that it should. If you needed to have an expert pilot there flying the thing, and that's what motivated us to start the company. That's what we've been working on since we started in 2014 is giving the drone the ability to to fly itself with the skills of an expert pilot. And the goal is not to remove the human from the equation. Human input, human ingenuity is still really important for a lot of these, these missions and tasks, but the goal is to allow the person to work at a higher level, to think in terms of what they want to get done and let the drone automate a lot of the lower level sort of piloting, avoiding obstacles, automating data collection kinds of things. So at the most basic level, that's what we're all about as a company.
Pieter Abbeel
Now I've actually seen your drones, but for our audience, could you describe what they look like?
Adam Bry
Sure, at a high level, they look like a normal drone. They've got four motors, they've got a camera on front. But what makes them special is the autonomy system. So we have three 4K navigation cameras on top. These are fisheye cameras or each of them see sort of the whole top hemisphere. And then we've got three on bottom. So there's six total navigation cameras. It's about 45 megapixels total of navigation data. And then we've got an Nvidia TX2 computer embedded inside, which is running all of our AI and autonomy algorithms. And all of the hardware is really designed to enable the autonomy software, which is where we've made the biggest investments. So the things that really make them special, you can't see, you can't see it in the device itself, but you can see it when it flies and the kinds of behavior that it's capable of.
Pieter Abbeel
Now I've seen your drone in action, and I think one of the most amazing things about it is the way it deals with obstacles or humans. It somehow is able to see them, understand them. It doesn't have a map of the environment, it just goes into a new environment and it's able to navigate that. Can you say a bit about how that is possible? What's the technology behind building something like that?
Adam Bry
So when we started, we made a really big bet on computer vision. So my co-founders and I came from research backgrounds. We met as grad students at MIT working on basically the same stuff like our lab was building autonomous quadcopters and autonomous fixed wing vehicles going back to 2007. But when we started in 2014, we made a big bet on computer vision and this was at a time where it seemed much crazier than it does today. You know, there were a couple of research examples of people doing basic low speed flight using visual state estimation. So SLAM type techniques. I think no or almost no, even research examples of doing collision avoidance, using drones or using computer vision. So we basically felt like all the information you need to make good decisions is in the image data. The challenges algorithmically is extracting that. But the pace of progress on those algorithms in 2014, which is just continuing exponentially today, is really incredible. So there's a number of different components in play inside the software system and to people who have worked on any kind of autonomous system, a lot of them would be familiar. One of the ones that we've invested the most in is this three dimensional understanding using these fisheye cameras. And there's a lot of algorithmic challenges that go into that. We're using fisheye rolling shutter cameras. So anybody who's worked on computer vision, especially geometric computer vision, knows that rolling shutter makes things challenging, and it makes things especially challenging in a context where you have fisheye lenses. Because a lot of the linearity assumptions that might make it tractable just kind of aren't, aren't there and fall apart so at the core of that geometric perception system is a deep neural network that's computing essentially flow between the images and then using that flow. So for those of not familiar flow, it's basically estimating correspondents vectors between two images. So if you see something in the left image, in the right image you can find the vector that stitches them together. That vector is the key to figuring out how far away the thing is. So that's kind of one of the foundational elements to understanding the geometry around the drone, but then getting the intelligent behavior requires making predictions about what's going to happen. So that requires predicting how a person is going to move if you see a person. It requires having a fairly good understanding of the vehicle's dynamics in order to predict how different inputs might result in different, different physical movement in the world. And then, of course, the real magic and challenge is getting all these systems working in real time in a compute constrained environment reliably and robustly enough to give the drone its intelligent behavior.
Pieter Abbeel
Now on the podcast, we've had several guests who work in the self-driving space, including Andrej Karpathy from Tesla, Chris Urmson from Aurora and one of the big things there is whether to use cameras, just cameras that are expanded with lidar and or other sensors. And it sounds like with Skydio, you made a similar bet to the Tesla bet, which is that camera only can get the job done. Is that right?
Adam Bry
We did, yeah. The problems are very related, the self-driving car problem on the ground, the self-flying drone problem in the air. I think a lot of the ingredients from an algorithm standpoint and from a sensor standpoint can be the same. The recipe is different and the tradeoffs are different. You know, some things are much easier in the air. Some things are harder. The stakes are generally lower. Like, we don't have a person sitting in our drone and we're not flying, you know, with a bunch of other person agents. But thebig things that push you towards computer vision and a lot of context, I think is like general three dimensional, omnidirectional perception and then size, weight and cost constraints, which for a flying vehicle are pretty extreme. Much more so than you get in a car. So our view of if computer vision is the right answer for cars, which it may be, it's definitely the right answer for drones where you really care about size, weight, power and cost.
Pieter Abbeel
Now, as I talk about the vision for the drones, you talk about rolling shutter cameras, fisheye lenses. Can you say a bit more about how are they, maybe different from the cameras we usually use? And, you know, when we're just doing a web video call or something? And why did you make those choices? Because you're saying they make things hard and yet you went for them, right?
Adam Bry
So actually, the choices that we made take the sensors that we're using much closer to the sensors and cameras that we use in our phones. And that was an intentional choice because there's so much investment going in there. So it's worth sort of rewinding a little bit to think about the alternatives. So classically when in a research context, in a lot of industrial machine vision context, people use these very specialized computer vision cameras. And one of the dominant features of almost all of those is what's called a global shutter, which essentially means that every pixel on the image sensor is exposed at exactly the same instant. Which essentially requires storage for every pixel to live on the sensor itself, which means that some of the surface area of the sensor is taken up. You know, in the silicon manufacturing process is taken up by storage, not taken up by photosensitive pixel area. And so you make a tradeoff there. You get, it effectively as if you have a smaller sensor for the same physical surface area if you want it to be a global shutter. By contrast, almost every single sensor that we're used to using in our phones, even in DSLR and other things, are typically rolling shutter sensors where every line is exposed one at a time very quickly in succession. So, you know, if you have a 30 millisecond exposure time, every line will be exposed for 30 milliseconds. Then the camera just rolls down them very quickly, but not instantaneously. And what that means is that you don't need storage on the sensor, but so you can get more surface area. You get like a higher performing, lower cost sensor for the same surface area, but you then have this problem where not everything is exposed at the same time. So if you take your cell phone and you wave it around while you take an image, you'll actually see this this kind of warping effect, which is maybe slightly annoying for taking pictures that you're going to look at, but becomes incredibly complex for computer vision because you can't make the assumption that every every pixel was exposed at the same moment in time. So the reason why we did that is because basically, you can just get higher performing lower cost sensors, in the sense that you get better data, better dynamic range than with a conventional global shutter. And we, you know, we push the complexity of that into the math and the algorithms to extract the information we care about. But you know, in general, we tend to make the tradeoff of like we would rather have the signal that we care about be present in the data, even if it's hard to extract then than not.
Pieter Abbeel
So I like the way you explain this Adam, and that makes me wonder about the AI side of things, of course, because you're saying effectively that as long as the information is there, it's okay if it's there in a complicated way because the AI will somehow handle it later. So how do you do that? How do you make sure the AI handles the rolling shutter fisheye lenses in a way that it gives you the results you want?
Adam Bry
Yeah, it's a great question. I'm laughing a little bit because this is kind of a joke that I have going back and forth with the team working on this, where they will sometimes complain that like, oh, you know, if there's something about our sensor configuration that is making it really nasty to deal with the thing. You know, there's like this weird pattern in the data that makes it hard to get the information we care about. And my retort to this, which has become a joke, is like, well, if there's this weird pattern in the data that makes it hard to get, we care about that means it's observable. That means we should be able to calibrate it out. The worst, the effect it has on the thing that we actually care about, the more inherently observable it is, and the more we should be able to come up with some algorithm to calibrate it and deal with it. So they're very sick of hearing this for me now, but it's, you know, at a first principles level, it's at least true in theory. So this particular rolling shutter fisheye was one of the big transitions from our first product, Skydio R1to our second product, product families Skydio 2 and X2, which we're selling now. So on R1, we had 12 conventional computer vision cameras arranged in stereo pairs. So we had basically like one stereo pair facing on every side of the cube. And we were using more classical computer vision stereo algorithms where you basically have a left image, right image, you rectify them, which means that you do like a calibration alignment. And then that turns the stereo problem into a straight linear search along an epipolar line, which is just a straight horizontal line in the other image. And there's a lot of research into how you do that with conventional algorithms over the last 20-30 years. With Skydio 2, we went to rolling shutter fisheye and all the assumptions that make the conventional stereo work just basically fall over. And that was a big bet and a big transition from using conventional computer vision to machine learning to do this. And you know, this was again, sort of like you could look at the academic research and glimpse and see some results that maybe made you think this would be possible. But the team really did a phenomenal job of coming up with the right network, set up, the right training methodologies and the right kind of input output framework with the cameras that we have to calibrate and format the data in a way that makes it all work. So, you know, there's no single magic answer in there. But in my experience, the best sort of deep learning algorithms tend to be built by folks who really have a deep first principles understanding of the problem. You know, it's not just sort of like pulling it off the shelf network and throwing some data at it and hoping for the best. It's really thinking critically about the structure of the problem that's there, how you can set up the network in the right way to leverage that structure and in particular, how you set up the training data and and the cost functions that you're optimizing again to really get the signals that you care about. So there's some secret sauce in there, but that's basically high level what we're what we're doing.
Pieter Abbeel
Now, I'm curious because whenever a neural network is trained, the most canonical setup is that there is an input and output that's supposed to be predicted and a training time. Humans provide the outputs also. And so there are labels. And so for Skydio drone, doing what it's doing, I mean, what are the right labels? Is it even set up that way, the training?
Adam Bry
Yeah. So that's another great question. I think one of the most exciting things about drones is that it's kind of like a wide, open creative space in terms of designing the algorithms and designing the behaviors of the drone. In the self-driving car world, it's pretty clear. The ultimate goal of self-driving cars is basically just to follow the rules of the road. And it's pretty clear what those are. Now getting it to do that is incredibly hard. But it's like a very constrained problem. With drones it's like it's pretty wide open. There's examples of people flying drones today in manual ways, and you can look at what they do. But there's all these sorts of new applications and new use cases that can really be enabled and invented through autonomy. So I'm not directly addressing your question here, but it just sort of pointing to the fact that it's this unconstrained problem which sometimes makes it less obvious what the label is or what the obvious training methodology might be for behaviors. In the case of the depth perception algorithms which would basically basically come back to optical flow. That is a more constrained problem, but it's much harder to get the labels because the sort of ground truth labels for that from actual data would be the correct depth and the true 3D geometry. So there's research examples of people using unsupervised techniques and enforcing consistency between motion and geometry and photometric appearance. And you know, I would say that we're reading all the research and doing some of our own research. The thing that is really key for us is the blend between synthetic and real data. And I think one of the sort of pleasantly surprising things for us and I think we're probably not alone in this is how much you can get done with synthetic data if you're really careful and thoughtful about how you create it and how you train on it. And to me, this is like, you know, frankly, it's still kind of a miracle how well this works, how we can train on synthetic data and then be exposed to real world data that the thing is never seen before and it performs extremely, extremely well. And you know, the first time this happened, it was kind of mind boggling and in development. But of course, if it didn't work, we wouldn't have the capabilities that we have now.
Pieter Abbeel
And it's so interesting, of course that synthetic data, at least for things like geometry, the labels can come for free. They're built into your simulator. So now there's nobody involved in the labeling. It's just a simulation engine providing it. It's amazing how all that can work. Especially thinking about the videos. I mean, when I watch the videos of the Skydio drones, I mean, I was watching a bunch of them again last night, and I see the Skydio drone following somebody who was on a mountain bike going through the woods. I mean, the drone is not just above everything. The drone is with the biker on there, some of the foliage of the trees that are popping up. It just seems like it really understands 3-D, even when it's that detailed. It's not walls or something like that. It's a very fine detail 3-D.
Adam Bry
It is. And it's, you know, it's coming out of very careful design of the whole pipeline of training and getting some feedback from real world data. And then the other piece of it, which is really key is the motion planning system, which is constantly predicting into the future, predicting where a person is going to go and then figuring out what the drone should do. And that's one where we have less deep learning today. Although we have actually done some research on this, we wrote a blog post about something closer to an end to end deep learning system, which I think may be somewhere out there in our future. But the motion planning system is basically optimizing against a bunch of different objectives continually in real time to figure out what the drone should do. And in order to develop that, there's a lot of real world testing, but there's a ton of infrastructure that has a lot of different scenarios that we care about where we're constantly developing against those things. And it's, you know, it's really not an accident that when you go out and fly, the thing that it has this sort of like intelligent predictive behavior where it seems to be making the right tradeoffs in the right situations of should it go through the gap or should it go around or should it go above all, these things are coming out of very carefully designed cost functions and balancing in different scenarios that we care about.
Pieter Abbeel
It's really impressive. Let's say a consumer buys a Skydio drone and receives the drone, maybe they open the box. What can they do with it? I mean, we've talked a lot about the AI side of things, but I imagine the consumer is not directly exposed to that, and it's more like a regular drone to them, but with extreme capabilities.
Adam Bry
Yeah. So the thing that makes it really unique and powerful is the ability to follow and film moving subjects and capture amazing video. So we started developing this product in 2014. The vision for it is we want it to be as if you had a professional film crew there with you, wherever you are, whatever you're doing. And that is basically distilled down through a bunch of AI and autonomy into a very simple user experience where you can hold the drone in your hand. You can connect to it from your phone, fly it through an app and tell it to take off. It will take off from your hand, turn around automatically, start tracking you. And from there, if you want, you can put your phone in your pocket and you can go running or hiking or biking or skiing or whatever. And the drone world will follow you and film you and you can choose if you want it to be in front or to the side or above. You know you get some of that level of control. There's different skills, we call them cinematic skills. You can tell us, do a droney, you will pull back and then and then come back towards you, which creates a really nice cinematic effect. So that's kind of the core autonomy capability. But you can also, if you want, fly it manually or semi manually with what we think of as AI pilot assistance, where you can buy an accessory, our controller, you can fly it with joysticks. And you're giving it a command to fly forward, fly sideways, turn around and it will follow that command. But it's using the same autonomy system to look at the 3D structure of the world around it and then predict into the future and basically reinterpret your commands to keep itself safe. So, for example, if you go up to a forest and just jam with the joystick forward, if it can, it'll find a way through all the trees. It’ll weave left and right and up and down to the extent that it needs to. So it's sort of like flying with a magic safety bubble around it, which enables you again to create really fantastic footage that you wouldn't wouldn't get any other way. And then some people also just like flying drones, you know, regardless of whether you're taking video, it's fun to have this tool that sort of at your control, at your disposal. And you know, it's especially one of the things that we see with our customers. I think it is really exciting that people will let their young kids fly the drone in a way that you never would with something that didn't have this technology because they know that it's inherently much safer. So that's kind of the consumer experience. There's also an incredibly exciting wide range of industrial, commercial, sort of professional user applications, which is something that we've really aggressively expanded into over the last couple of years with autonomy capabilities geared specifically for those.
Pieter Abbeel
Yeah, I love the consumer experience which I've seen. One of our former guests Anca Dragan on the podcast, we were at her wedding and you had your drone there and do photography for the wedding, which was amazing. But I am curious about the non consumer side. Absolutely. What are some of the more industrial or other commercial use cases you're seeing?
Adam Bry
We've learned a lot about this over the last few years, some of this was in our sights when we started the company. But as we've gotten customers in these markets, there's across all these sorts of infrastructure industries. So energy, utilities, telecommunications, rail, construction, transportation infrastructure with bridges and roads. There's just an enormous amount of inspection and maintenance work that goes on. And it's kind of interesting because most of it happens out of sight. Most people don't even realize that this is happening. But if you just think about the scale of an energy utility with tens of thousands of miles of power lines and, you know, enormous power plants, just to pick one example. Basically, this stuff is just constantly experiencing issues. It's like nobody's fault, but it's just very complex, sensitive physical infrastructure where things break down, things rust out, things get damaged by wind. And it's different in different industries. But the common themes are basically like a lot of manual effort, typically a lot of risk to the people involved in it and a lot of heavy machinery. So for energy utilities, they might be flying through helicopters, which cost hundreds or up to a thousand dollars an hour fully loaded to go along and inspect this stuff. Or they might have people climbing up in every transmission tower to look for rust and other damage, and to make sure that the insulators and the cables and everything are still safe and secure. And so the really big opportunity is to make it just frictionless to have a full digital picture of critical physical infrastructure. And drones, manual drones are starting to be used for some of these tasks, where organizations will either pay like a drone service provider. So a network of drone pilots come in and do this stuff, or they'll try to train some of their own operators to do it. And it's exciting because it's sort of working today. People are delivering value and getting compelling results with manual drones. But it's still just, in our view, at least just a fraction of the percent deployed compared to what it could be and will be. And if you could make it frictionless to collect the data, you unlock this entirely new, exciting world where you just have a digital picture of your infrastructure. Then you can start to think about running automatic damage detection and change detection and basically like proactive notification for issue areas. So that's the big theme, and that applies to a bunch of different industries. And that's what our products are being used for today. So people are flying the drones either semi manually or they're using automation with a product that we call 3D scan where you basically just tell the drone, here's the structure of the scene that I care about. And then it will adaptively map the whole thing in real time and then use that map to guarantee high quality photo coverage of the whole structure seen from every angle. So the output of that is basically, you know, a full visual record of the asset that you can turn into a 3D model. People call us like a 3D digital twin, or you can just look at the individual photos as sort of like a visual record of what's there. So that's something that we're incredibly excited about, where there's a lot of applications today. But I think there's just tremendous growth opportunities as these things become easier to use. And you know, this is one of the things that we talk about is we want the drone to adapt itself to the user in the application rather than the other way around. One of our big goals is to make it so that, for example, a construction company doesn't need to have specialist drone pilots. They can give drones to the people who are part of their normal engineering workforce, and this just becomes another tool for them to use.
Pieter Abbeel
Now, as you think of the full potential of drones, naturally you're building the platform enabling any applications behind whatever somebody might want to do. But are there specific applications that you are personally excited about and hope to see in the future?
Adam Bry
The inspection work that we talked about, I'm super excited about, you know, I think that spending time on the ground with customers in this industry, it's just impossible not to be excited and optimistic. Both about what they're doing today, but really like what can be enabled over time as the products get better and that applies in a lot of different industries. You know, I think energy utilities are one of the really compelling ones because, you know, it's the core infrastructure that we all depend on. The stakes are really high. You know, there's examples, especially in California, things going wrong and causing fires and and it being incredibly dangerous and expensive. And I think that's something where drones can really help a lot. So that's one that I'm super excited about. Another one that I'm personally very excited about, which I think is more controversial and somewhat polarizing, but I don't think it should be. And it's something that we've been proactive on this front is public safety use of drones. So we have a number of public safety agencies using our drones today in what we call the drone in the trunk model where you have a drone in the police vehicle and on a case by case basis, you can use it for all kinds of things. So there's a lot of search and rescue work, like if there's a missing person and you know, you need to search a wide area, drones can be a very useful tool. The capabilities that we have enabling you to do search. There was an example where a car went off the road and rolled down the side of a hill and was down in an area that would be very difficult for a person to get to. Not impossible, but difficult and time consuming. And the officer was able to fly one of our drones down there very quickly to see if there was still a person in the car and if they needed help. So there's search and rescue work. There's also kind of this tactical stuff where, you might have somebody who's say hanging out in someone's backyard and is potentially armed and you don't know if they're armed. You don't know what, the officer doesn't know exactly what the situation is, and one possibility is for them to go back there themselves. But that means potentially going into a dangerous situation where they're at very least going to have their hand on the gun. They might have their gun drawn, or you can fly a drone back there and see what's actually going on and calibrate the response appropriately. So even in these tense tactical situations, I think drones have enormous benefits. And then the future possibility here, when you think about drones and docks, is having drones basically automatically or not necessarily automatically, but with very simple operator input responding to 9-1-1 calls. Where if a call comes in, you know, it might take five or 10 minutes for the responding officer to get there. But you can have a drone there in a minute. Some situations, maybe even less, which I think at maturity is really a transformative technology for keeping people safe or you can imagine if there's like a violent crime in progress, being able to have a drone there on scene could change dramatically the way that situation unfolds. It could serve as a deterrent. It could interrupt whatever is happening and at the very least, it could provide real time sort of evidence to officers to figure out who's guilty and who's not and inform their actions afterwards. So there's examples of agencies sort of making this work today with manual drones, but I think it's something that we can really make happen in a scalable way with autonomous drones. The flip side of this is that we don't want to live in a world where we've got police drones flying overhead, 24-7 spying on us. And I think that that's a concern that I have, that we have as a company. And so we've been proactive in developing a set of principles, which we call the five Cs, which are sort of five core ideas for public safety use of drones. Which were proactive with customers and customers in that space and regulators, and I think that if we do it right, as not just us, as a company, but as an industry, there is an opportunity for dramatic positive impact. But it's going to take both thoughtfulness from a product standpoint and thoughtfulness from sort of an adoption and a regulatory and policy standpoint.
Pieter Abbeel
So Adam, you mentioned how Skydio wants to responsibly engage drones and you have this principle of the five Cs. Can you say a bit more about that?
Adam Bry
Sure. Yeah. So they're basically five core ideas that we think are a useful framework for industry and agencies to set up scalable drone programs. So they cover things like community engagement and transparency, civil liberties, cyber security, common operating procedures and then clear oversight and accountability. So each of these has sort of downstream concept work associated with them. But, you know, to pick one community engagement and transparency. I think this is really like a super critical thing. And the agencies that have had the most success with their drone programs have been very forward leaning and reaching out to community organizations, sometimes like the local chapter of the ACLU to talk about what they're doing and get feedback on how they implement it. And also doing a lot of just sort of public demonstrations of here's what we're doing, here's how it works, here's why we use it. And I think that's really key. And the more powerful the technology becomes, the more that you think about, so the concept that I talked about before having drones respond to 9-1-1 calls. We call that drone as a first responder. If you want that to work, you really need and you really want the community to have a clear understanding of what the program is and how it works, why you're doing it and what you're doing with it and what you're not doing with it.
Pieter Abbeel
Now, of course, when you sell a drone, it's not in your hands anymore. How do you retain control over what people might do with the drones that you sell?
Adam Bry
Yeah, that's a great question. Most of our drones are internet connected, even if they're not flying out of a dock, you know you're flying them from a mobile device, which is typically connected to the internet. So we get data and telemetry back, which we're using for, you know, customer support and to debug issues and whatnot. I think that in public safety contexts, a few things, like I don't think that it's healthy for a company to be the final arbiter of what people are doing and what they're not doing. It's not to say that we don't have a role to play there, but I think by far the most leverage comes in kind of upfront work in the training that we provide, the principles that we've developed, the things that are built into the product that make it easy to do some things or not other things. And so I think that's where you want to put the most emphasis. Having said that, if we were working with the customer we saw repeated patterns of abuse and misuse. There are things that we could do. One just like deactivating features through the cloud, being able, you know, cutting off warranty support, typical kind of like customer provider relationship stuff. So there are a few of those tools available. But you know, this is not a conversation that's unique to drones, either. This is kind of like a tech conversation. I don't think in this situation, you want a company to be sort of like the final arbiter of what is happening in public safety. I think that we have a voice and a role to play, and we've been very proactive on that front and we'll continue to be. So far, we haven't had a situation where there was abuse or misuse. It's an abuse or misuse issue that causes us to step in and do more. But it's not to say that it couldn’t or won't happen in the future as we get to larger and larger scale.
Pieter Abbeel
Now, Adam, I'd like to take a step back and learn a bit more about how did you decide to start Skydio? And how do you even, as a kid, when did you get excited about engineering, science and from there, drones? And from there then founded Skydio? What's your journey that led up to where you are today?
Adam Bry
So I grew up flying radio controlled airplanes, which were kind of the predecessors to drones. I mean, they really were the predecessors to drones. So these were, you know, some people may be familiar with this stuff, but these are typically like balsa wood constructed small flying things where you'd like, build them out of wood and then cover them with either just plastic wrap stuff or tissue paper. And I just got super into this as a kid and really probably unhealthily obsessed with it. So I spent a disproportionate amount of my childhood in my basement building these things, and then I started flying in these aerobatic competitions where you basically get given a set sequence of maneuvers and you fly them as precisely as you can. It's a little bit like figure skating, actually, because you're sort of like doing this routine and you're getting judged on how well you do it. So I was super into this as a kid. I was fortunate to have my dad who loved it as well. So we did it together and traveled all over the country flying these aerobatic competitions. That's really what got me interested in engineering and flight. And I think, you know, looking back on it, I certainly didn't do it because I thought it was going to become a career. But yeah, it was fortunate to develop a kind of intuitive understanding for flight and what's possible and how these flying systems really work. And then going through engineering education, couple that with kind of like the rigorous technical mathematical formalisms that describe all these things. And so I think that that gave me like a useful combination of skills, and I was fortunate to be starting grad school at MIT in Nick Roy's lab, the Robust Robotics Group, around the time when people were first taking basically RC airplanes and putting computers and sensors on them and then writing software to get them to do smart stuff. And that was just incredibly and still is incredibly exciting to me. This idea that you can try to write software to replicate a lot of the things that an expert pilot would do. So that's where I met Abe Bachrach, who is one of my co-founders and our CTO. So we worked together for three years at MIT. And then Nick, our advisor, Abe and myself got the opportunity to go to Google to help start Project Wing, which was their drone delivery program. So that was 2012. We moved out to Silicon Valley to do that and sort of did a lot of the initial work on that project. And then the motivation for starting Skydio, as I said before, was at the outset of people starting to see all the cool things that you could do with drones. And there were a lot of startups going after various pieces of it. And we just felt like it's really exciting the potential here, but we feel like most folks are skipping over this really important product enabling piece, which we think is going to be foundational. And it's stuff that we knew a lot about and and loved working on. And that was the thrust that got us going.
Pieter Abbeel
And Project Wing at Google, I haven't heard too much from it, from it lately, but it used to be focused on deliveries. And I remember watching a delivery by Google drone in Australia, Australian farms getting deliveries from drones. Is that project still around, do you know?
Adam Bry
Oh yeah, it's still going. They've, I think, gotten up to a reasonable scale and they have a few trial commercial deployments.
Pieter Abbeel
But your focus, of course, has been very different to put the drones into consumer hands or professionals who just can use it for whatever they are doing as opposed to automating delivery specifically. Now when I watch one of your recent talks Adam, one of the things that's really interesting to me is this sketch that you have, the vision for the company that you call the Arc of Autonomy. Can you explain the Arc of Autonomy?
Adam Bry
Sure. So maybe before I get to that, it's kind of related to the Arc of Autonomy. I think it is interesting to talk about this, the application space and drone delivery versus what we're doing, which is basically like a flying sensor platform. And I think that there's clearly a lot of exciting value, utility to be provided by drone delivery. And I think Zipline, like you had Keenan on, Zipline to me is really the most exciting pioneering company there. And I think one of the things that they've done a brilliant job of is finding the applications and the regions where basically the ROI is very high. Delivering critical medical supplies in areas where you don't have a lot of infrastructure is a very high value proposition. But the urban, the thing delivers a burrito or it delivers like your Amazon package. There is some value there, for sure. But if you think of it like, what's the upper limit on how much value per flight there? I mean, we're probably talking about like $5, maybe $10. If you just think about conventional delivery costs, whereas you know, a single flight inspecting a bridge or a wind turbine, you know these can be worth the thing that you're competing against there oftentimes costs thousands of dollars and involves real risk to the people involved. And you can do it with a much smaller, lighter, inherently safer system. So I very strongly believe and you can just see this today that the flying camera platforms are going to reach a much larger scale much sooner because the just risk value proposition is much stronger there. Which is not to say delivery isn't going to happen. I think it will, but I think it's going to take much longer because it ties together a lot of the hard problems and it does so in sort of like a lower marginal value framework, at least for most of what people want to do. So that very much informs our strategy. Coming back to the Arc of Autonomy. This is basically like a five stage framework that we think about where, you know, on at the beginning, you go with an operator on the ground, basically flying the drone, benefitting from collision avoidance and visual navigation to then adding workflow automation with products like 3D scan. But then the really exciting transition, and I think this is a coming theme for the drone industry, is going from needing an operator on the ground, flying the thing to how having the drone be able to fly itself remotely over the internet. So this is a product we call Dock, which we're developing, where the drone lives in this internet connected charging base station and can basically be flown anytime, anywhere, either scheduled or on demand without a person needing to be there to control it. And I think that's just an incredibly exciting paradigm for a lot of the infrastructure inspection work that I'm talking about. This basically means that you can have these things installed and flying themselves as needed. But then the future possibility which we have is sort of stage five on the Arc of Autonomy is drones basically becoming like a networked service where you've got docked drones installed all over the place that can be available on demand or the people who use them never have to physically touch them in most cases and may not even need to to own them. This is futuristic stuff, but I think there's kind of an analogy to cloud servers where when you're using a cloud server today, you don't, you know, it's available, you use it sort of on an as needed basis. You can timeshare it with other people that are using it, and it just does the work that you need it to do and you don't have to actually ever go there and mess with it. And so I think that that possibility very much exists with drones. There's a lot of technology and product things that need to happen. There's some regulatory things that need to happen to unlock that. But I think that's what we see ourselves kind of incrementally building up towards.
Pieter Abbeel
That's so intriguing because in some sense, right now, you're selling drones, but in that ultimate vision, you might not even be selling drones that people wouldn't even know, necessarily that drones are involved. They would just want an image of something or a video of something and request it. And then it would show up.
Adam Bry
Yeah, exactly. And you know, I think that this is true in a lot of different robotics categories, but it's important to understand how early we are in the category. You know, it's like you can look at the evolution of like the computer or the personal computer or the phone and get some sense of if you sort of line that up with the drone industry like, I think we're still, you know, we're like in the the sort of pre true PC Windows era. And there's just a lot of stuff yet to come to make the products really live into their full potential.
Pieter Abbeel
Now talk about public facing communication and education. I saw that recently there was a push on Twitter by you and Skydio for Marcus Brownlee to use Skydio. You're clearly a big fan of his work? Did you get him interested?
Adam Bry
Too soon to say, I guess. I don't know if he watches your podcast. He should. So we're constantly working on some new stuff, on new stuff, and we've got some products coming out that we think would be especially exciting and compelling for, both you know him to review, but also kind of a lot of what he does of video creation. So the area where we've gotten most sophisticated with the autonomy to date is in the following and filming cinematic applications. But I think there are other kinds of cinematic things that don't necessarily involve following a subject where autonomy can do some really special things. So this is something that we've been working on really for the last 18 months or so, if not more. And we have some exciting stuff coming there and we're interested to get it in the hands of people that we think will have interesting use cases for it and might find it valuable.
Pieter Abbeel
One of the things that's quite unique about Skydio is that I believe roughly 10 percent of the employees come from Olin College of Engineering. Is that right?
Adam Bry
I don't know if that's, we've grown a lot over the last 18 months. That may not be true today. Although it's probably true for the engineering team are close to it.
Pieter Abbeel
Now when I think about the journey of Skydio from the outside, of course, not knowing exactly what's what's been happening on the inside. And I look at the history of drones, especially drones used by consumers. It feels like there was a stretch, maybe four or five years ago where DJI, the Chinese drone company, seemed like on a path to essentially take a monopoly on this entire space. And I'm curious, how did you experience that period at Skydio and how did you come out so strong the way you're here now?
Adam Bry
The drone industry has been a fascinating landscape really forever, but certainly in the last six, seven years. And you're exactly right. I think that DJI is a very impressive hardware company that makes really well integrated, you know, drone hardware devices. And I think that they've executed extremely well. And there was a period, kind of like 2015 to 2018, maybe 2019. Some people still hold this view today that, like DJI, is just totally dominant. They've won the market. The whole thing's over. This is just what a drone is and how it's going to be. And you know, as much respect as we have for a lot of what they built. We never really believed that to be true because we just knew how primitive the products were compared to what was possible and how important autonomy was going to be to making drones really scale the way that they should. So these things play out in different ways. You know, some investors were very spooked by DJI. I think that there were a number of Silicon Valley startups that had business models that weren't really compatible with what DJI was doing. So there are companies that were trying to build these very bespoke, modular like way more expensive drones that just looked silly next to like a well integrated consumer flavor drone. So they're certainly like a wake of destruction that got left behind DJI. But in general, I think we've kind of been fortunate the way this has played out because that in a lot of ways it sort of, I feel like we should have way more competition today than we do, given the size of the opportunity and the market dynamics. And I think one of the reasons we don't is because people were so afraid of DJI that the space was very underinvested from a venture capital standpoint and from just sort of an entrepreneur, people trying to start things, standpoint. And DJI, they're still a force to be reckoned with. They still have dominant market share. But I think that grip is loosening because the product landscape is evolving. And then there's also this just kind of external force, geopolitically where these things that started off looking like consumer toys have now become really critical tools for all kinds of national security critical applications. And I think for a lot of people, for a lot of reasons, people realize that it's not healthy, probably to be fully dependent on a Chinese company, both from just sort of a supply chain standpoint and from a data integrity standpoint. And so that's another, that's another big theme in the market that really cuts across every segment, certainly like public sector customers, but enterprise customers and even consumer customers to varying degrees, think about this stuff.
Pieter Abbeel
Well, Adam, as you look ahead to the next few years at Skydio and beyond, what are some of the plans that are most in your mind? And maybe even looking to get new people involved for those plans?
Adam Bry
Yes. I mean, I've talked about this, but I really think that we're very near the beginning of what's possible. And we've built a foundation for trustworthy autonomous flight. We're seeing our products now adopted in all these exciting industries with really cool applications, but there's just an incredibly exciting roadmap of stuff to be built to fully realize that opportunity. There's a lot of very general purpose kind of platform stuff that applies everywhere. And then there's kind of the opportunity for more and more specializations, for different kinds of tasks. So the stuff that I enjoy most is, one, spending time in the field with customers and seeing what they're doing and seeing what works and seeing what doesn't and figuring out how we can deliver them more capability and then spending time with the engineering team. You know, we're fortunate just to have an incredible group of folks across every discipline in hardware and software and autonomy and all the infrastructure to support this stuff to really bring these crazy AI futuristic concepts to life and make them useful in the real world. So, you know, I'm certainly not neutral on this, but I'm fortunate and I love the job that I have, but I constantly feel like I wish I could spend more time just actually hands on engineering because the problems there are just so exciting and the results are so immediate. We have this grand vision for what's possible, but we also have a real business today with real paying customers and every incremental step we take in developing products like 3D scan, for example, and making it work better, just very quickly shows up in customers hands and has a real world impact. And I think that feedback loop of developing like cutting edge AI robotics and shipping it and seeing it do useful stuff for people is just super, super fun to to be a part of. And the only thing that makes it possible is just having a world class team across all these different disciplines. So we're constantly looking for folks that are interested in all aspects of that. So if you go to our website, it's pretty easy to get in touch through our careers page. But anybody out there that thinks robotics and drones or really just robotics in general, or AI are exciting or particularly passionate about drones and wants to be in a place where you can see immediate real world impact for the stuff that you build. I think that, I'm not neutral, but I think we've got the best set up in the world for for that.
Pieter Abbeel
What is the URL you would send them to?
Adam Bry
Skydio.com and then go to the careers page. Yeah.
Pieter Abbeel
Great. Well, Adam, this has been such a wonderful conversation. Thanks for joining us.
Adam Bry
Thank you, Pieter. This has been great.