Benedict Evans on the Season 2 Episode 4 of The Robot Brains
Transcript edited for clarity
Pieter Abbeel: Today we're speaking with Benedict Evans. And if you haven't heard of him, you've likely read one of his opinions or analysis. He's the voice of reason on the internet, trying to sort out the technology issues of the day. Today, Benedict is an independent analyst, having formerly been a partner at the VC firm Andreessen Horowitz. He's one of the world's leading experts in mobile media and technology trends. Most of our guests are people building new types of AI and robots. They're usually very much focused on enhancing or creating brand new areas in the field of AI and robotics. Benedict, however, has a bird's eye view of the entire world of AI. As a writer, and analyst, he sees all the hype, hope and reality of what's going on in modern technology. He publishes a popular weekly newsletter on the most important happenings in tech. And if you aren't a subscriber yet, I strongly urge you to sign up now. Welcome to the show. Benedict, we're so happy to have you on.
Benedict Evans: Thank you.
Pieter Abbeel: I spent last night watching several videos of yours and going through your presentations and essays. And I must say I'm just stunned by the amount of information you're able to gather, process, and synthesize. How do you do this?
Benedict Evans: Oh, well, I just find a thing:
Pieter Abbeel: Now where do you find the things you read? Because what you present is very data-driven. And kind of curious, where do you find all that? Now, when I look at your background, what I see is you started at Cambridge where you graduated with first class honors. When you were studying as a history major, what did you think of the world back then?
Benedict Evans: History is to only extend to trial and error, because there are infinite effects. The deployment model is to try and understand what it meant and what could have happened differently and why it happened the way it did, and what were the sort of fundamental trends that are driving this. And so you can apply that process to thinking about the Napoleonic Wars or the Crusades, or why Britain became the dominant power in India. Or you can apply it to trying to work out, “well, what does it mean to say that this is open or closed? What does it mean to us? Why would that platform work? Why are consumers doing this? What is underlying the importance of this trend?
Pieter Abbeel: Now, in your time, at Cambridge, was technology part of the coursework or something you brought into your life after that?
Benedict Evans: Well, I was studying history. I wasn't studying computer science. But I suppose the general point is that technology was quite a small story until quite recently…I think in my college, you had a PC and you use it to code. And in my third year, the personal PC came out. What would you do with a PC? Well, you know, you'd write documents with it or be associated with it, maybe play games. There's no real reason for a normal person to have a PC except because you wanted to have a PC. And that changed for the internet. More than anything else. I think that's been the transition in the last 25 years is that tech went from being one of many industries to the sort of biggest industry that shapes everything else.
Pieter Abbeel: Feeding off that, every year, you publish a big presentation, digging into the macro strategic trends in the tech industry. This year was called “The Great Unbundling” and there's a great quote in there from Jorge Paulo Lemann saying, “I'm a terrified dinosaur. I've been living in this cozy world of big brands, big volumes, nothing changing very much, you could just focus on being very efficient, and you'd be okay. And all of a sudden, we're being disrupted in all ways.”
And of course, it's referring to tech effectively enabling a whole new set of paradigms of how to sell things. And I'm curious, you know, what's your thinking on this? How do you explain the great unbundling to our listeners?
Benedict Evans: Our industry structures are breaking apart. And all the costs are thrown up in the air, and no one really knows where they're going to settle. A third of us pay TV subscriptions and it has gone away in the last five years. Over half of all advertising in the US, globally is now the internet. And it's driven by data being driven by creative.
And you know, most of the growth in CPG in the last decade has come from the brands even before the explosion of e-commerce. And so you have this sort of breaking apart in which it used to be this fairly straightforward value structure in that, you know, that Proctor and Gamble would make the product and they would have an advertising campaign and then they would ship products of that product to retailers. And retailers also have advertising campaigns and the advertisers have to sell the product. And now there is the app and the advertising we may see on TV or in print and TV is going to decline rapidly and printers have gone away. And those apps are those which help us now have many new kinds of retailers and very often from outlets competing with people who don't sell through those retailers that sell direct or start by selling garbage or Velcro brands and the new retail is functioning in completely different ways.
So all of the presumptions about the way that you go to market, the way you build a brand, the way you ship a brand change, the same thing on the retail side, it used to be that, you know, you're constrained by real estate, and by the distances that people would travel, but now you have a whole other set of constraints and another set of parameters. I think, in many ways, either sort of core observation might be that, you know, as a retailer, or marketer, or brand or an advertiser, there's sort of two fundamental purposes you're trying to serve, and logistics and discovery. On the one hand, how does the customer physically get their hands on the product? And how do they know it exists? And how do you persuade them to like it? And in the past, you know, you have a budget and a marketing budget and retail budget, and a shipping budget.
And now those budgets all kind of become interchangeable. Because you can say, if we open a store in that city, do we decrease Amazon budget? If we open a store, what happens to our returns? What happens to our Instagram budget? Should we shift from search advertising to free returns? And you know, before the internet, those were not choices open to you. You couldn’t say, ‘Should we open a store in that state? Or should we advertise in that state?” That was not a meaningful question, now it is a meaningful question. And so you might have a trillion dollars just in the USA for spending but it becomes interchangeable. And everyone tried to work out well, what does that mean? What are the new balances? How would you go to market in this environment?
Pieter Abbeel: Now, one of the things that plays a big role there, of course, is AI, at least in my mind, and I'm curious about your take, I'm very curious about your thoughts on on that direction, the role of AI in this grid unbundling and and beyond and our everyday lives.
Benedict Evans: So I mean, I think there's a kind of a useful sort of coolness in AI research and that is anything that doesn't work yet. So as soon as it works, people say, well, that's not AI, that's voice recognition. That's pattern recognition. That's just a database. And if you go back to the 1970s, AI research was just a database. They could do trend language translation with database lookups, you would match this word to that map that word and you could translate it. And that was AI, people call that AI then now you say no, that's just a database, and it didn't work anyway. So AI is anything that doesn't work yet. And it's sort of more useful, I think, to talk about specific techniques. And so the technique at the moment is now machine learning. And machine learning is essentially pattern recognition. And it unlocks a class of computer science problems that we couldn't really help at all. And the general way of describing this is that there's a class of problems that are easy for people to do, but hard for people to explain to computers. And so you have problems that are hard for people to do, but easy to explain. For example, calculate 10,000 mortgages. It's quite hard to do that in your head, but it's really easy to tell a computer how to calculate 10,000 insurance premiums. On the other hand, the opposite kind of problem is things are very easy for us to do, but very hard for us to explain why.
So like how to tell the difference between a cat and dog. Okay, it's very easy to do but now if I’m explaining, “Well, why exactly is that a cat or dog? Why is that?” It gets hard. I mean, if you remember the story about Plato describing a man, he says it's a featherless biped. And Socrates holds up the fox and says, “Here's your man.” And so you had this kind of whole class of a problem where you would do it with rules. How would you recognize it? Oh, well look for something with edges and look for something with a texture that looks like a feather and make something find legs and look for ears and look for pointed ears and look for tail. And it will always sort of work but it would never actually be able to tell the difference between dogs and cats. And the same thing with translation. The same thing with speech recognition. And so it turns it into a pattern practical problem. But it's not like a step to Hal 9000 anymore. And so the other side of this conversation is we're in 1910, and somebody looks at an airplane and says, “Wow, these are getting faster and faster and higher and higher, like they might accidentally go to space.” And the aircraft engineers said, “Well, number one, we're not quite sure if it's theoretically possible to go to space, but if it is, it won't be with a 10 horsepower engine and canvas wings.” And I think where we are now with all of this is, you know, there were some great, great people trying to do machine learning in the 1980s. And it didn't work because we didn't have enough data, enough computing power, then in about 2012-2013, people worked out, “Oh, actually, it's working now.”
And now we're at the stage of trying to work out well, what does that do on a one axis? How far can you take this? And what other kinds of problems? Can you turn into pattern recognition problems? And on the other side, what are the formats? Can you build with that? And what problems can you solve with that? Because it's not just image recognition, and speech recognition. There's lots and lots of other things. And so we now do interviews of people building companies using this and existing companies, kind of creating new products with it, or making their products better with it.
So as I said I hate the term AI because it conceals much more than it reveals, like machine learning. In 2000, we started seeing companies that would say, “Well, I'm an academic, here's my CV, give me $20 million, I'll register a domain name.” Then the next step is you see people saying, Well, I'm going to make an image recognition platform, I'm going to build the best image recognition and other people can build image recognition products using my image recognition platform. And then the next step after that was, something that comes to mind is we invested in a company called People.ai, which does natural language processing on text coming in and out of Salesforce to work out which of your sales pipelines is going wrong. At that point, that's not really a machine learning company. That's a sales process optimization company that's selling enterprise software to people with Salesforce. And that at that point, you'd become a product. And there's always a sort of a progression with any new piece of primary science which is that it goes from science to a sort of physics then becomes a technology and then it becomes a product in companies. And then eventually it just disappears, and it's inside everything and no one notices anymore. And that's sort of the progression that machine learning went through.
Pieter Abbeel: Absolutely, I couldn't agree more. I get the sense that the systems we're currently building actually could get us really, really far. We could build amazing AI systems five years from now, maybe with just scaling up the ideas from today, even without needing to make a lot of changes. And that's a trend that we've seen for several years now, especially at a place like OpenAI where they really emphasize the scale of things they work on. And in contrast, you're saying, well, it's nice to have pattern recognition. And it allows us to build so many great applications. It's going to influence the world in many ways, but it's not going to get us closer to a more general AI system. And I'm kind of curious how?
Benedict Evans: So that's pretty much what the vast majority of people working on machine learning research think. And, you know, this, yes, this may be a building block, but you're not going to get HAL 1000 with more ML anymore than you've got it with more database, or more expert system. Now, of course, you know, it's sort of theoretically possible that if you added a million times more compute, then general AI might pop out the other end. But that's something that's very difficult to sort of know deterministically. And in the same way, it's sort of theoretically possible that if you just put more engines you keep going higher and higher and higher.
At some point, in the future, we may have something that you could describe as general intelligence. The challenge, of course, is depth. General apologies, this is actually just a sort of a one linear axis. That, you know, a dog has general intelligence. And, you know, on one hand sort of say that a dog is on the linear axis between us and say, a mouse or something. But an octopus clearly does appear to be about as intelligent as a dog, he doesn't seem to be on the same axis. And you know, a hawk has a very specialized kind of general intelligence. So it's a non general general intelligence, so to speak. And so it's sort of there's all sorts of kind of theoretical and philosophical problems, even with the concept of general intelligence, we can start to say that, well, we think that we have something that dogs don't have, and we can see a capability that we can work out that they can't force, it's no particular reason to believe that we are the endpoint of that spectrum. And they might, you know, just as we can sort of do a calibration test on a dog or horse and say, Well, look, they can work out this, but they can't work out that it's hardly possible that there's some alien species. So people in a box somewhere, doing talented tests, they really don't have general intelligence. See, they can only work out there's a reason to presume that our intelligence is somehow binary. It's very easy theoretically, to say that some alien intelligence might have an average IQ of 300. It's more interesting to say, Well, I know there might actually be some other step change beyond where we are for some other form. Think about just as we looked at dog walkers and say we've got some corrugation but allow progression. So you kind of know that people have these questions all the time in class. Most people actually working on machine learning, not particularly, do not vote. Most people do not think that we are in a predictable way face growing, whatever that might even mean.
Pieter Abbeel: I personally definitely think it's very unpredictable but the unpredictability part of that is that it could always be closer than you think. Could be further away, but possibly closer than you think.
Benedict Evans: Most people do not think that it's going to come from just Moore's law, and more data and more heat. We have no idea what that is. And it might not be that other thing. It might be two or three or five other things arriving at a determinant point in the next 10 years. 50 years. 500 years. We have no idea.
Pieter Abbeel: Yeah, a big part is of how you think about what it means to have more of what we have. And you kind of alluded to that, right? You said we'll need more breakthroughs. And one way to think of more of what we have is to say we have, you know, 50,000 people or more, maybe around 20 to 50,000 people working on AI research. And so we have in some sense a dynamical ecosystem where there are people who are constantly trying to come up with, you know, new ideas that are different from we have today. And a big one, of course, happened in 2012 with the ImageNet moment and neural nets really taking off. But I think among those 20 to 50,000 people, I gotta imagine one of them somewhere or a few of them together somewhere will have a new breakthrough at some point. And if there's so many people pushing, it's very unpredictable when it will happen.
Benedict Evans: This isn't like saying people who work in steam engines don't think, well, this is the same economy of that.
Pieter Abbeel: Well, it's one of those things where I think a lot of people are getting paid to, to do exactly that to try to advance AI. That's actually what they're getting paid for giving them a good amount of time to spend on it. And a good amount of resources definitely helped by all the commercialization.
Benedict Evans: We have no idea what you're required by saying, Well, you know, we've got plenty of people working on it, if you just said, Well, you can't live with that. You can get off completely. They didn't come up with a name for one completely different from all the people working on a logic fallacy.
Pieter Abbeel: So what do you think about non-edge AI directive breakthroughs? The kind of stuff that's happening in the industry…do you see more lower hanging fruit there and things that will take off?
Benedict Evans: Things that are interesting include image recognition. So for the future or some implementation of something else, everything will have image recognition. But what does that mean? A while ago, I looked at a company that is doing the benefits of the ground rules and workflow. Like you've got an FDA and then taking photographs or video the department and then you turn the image sensor into a kind of a generalized universal input. But gathering the data means there are an awful lot of places in which you can own machine learning or some other mechanism and allow you to automate something that you couldn't automate before.
Pieter Abbeel: They definitely become a lot easier to use every year. I mean, 10 years ago with initial breakthroughs, it required very special purpose programming these days. Well, it's easy to build on top of the frameworks that Google and Facebook are providing like PyTorch, TensorFlow and so forth. And you can even use it as an application without even building something new. The direction I'm personally really excited about, and I know you spend a lot of time on too is logistics. Because I mean, at my company, Covariant, we build AI for robotics. So robots can help out in warehouses, ecommerce, fulfillment centers, and so forth. And so you've been spending a lot of time in that space more generally, looking at how robots can help. So I'm curious about your thoughts on the upcoming trends of what's possible in logistics and ecommerce in the next few years?
Benedict Evans: [Unintelligible]
Pieter Abbeel: Now you like to categorize e-commerce logistics effectively into the way it gets delivered, like you talked about the bike delivery, the truck, the refrigerated truck, and postal mail, and so forth. And when I think about logistics, and I look at a warehouse and what goes around in a warehouse, it's essentially one category of three categories.
What happens mostly in warehouses is things that fit in boxes or little baggies, and they're kind of nicely get one click and get shipped to you through the mail. Even there, when I look at it, it's partially automated, a lot of is not automated, yet. A lot of it could be much more automated. And we're seeing trends of, of this becoming more automated faster, and so forth. And I'm curious, how important do you see that as part of the whole picture that the ability to to make these things more efficient, the operations themselves, beyond expanding them into new regimes like, like, like restaurants, and so forth?
Benedict Evans: Well so physical logistics. And some of that is really just, you know, remember that a lot of ways anyway, going on the model, specifically efficiency models, market models, you can just re engineer your company a little bit, you can get oil and water in three weeks, and there's a new world order, which is out to be Asia. And, you know, ticket prices. And this is what it's all about. That's the idea.
Pieter Abbeel: You know, now, one of the other things that you talk about quite a bit in your 2021 report is how China is becoming more and more capable in AI next to the US and Europe. And I'm curious to hear your perspective. What role will China play in AI and how would that affect us in the rest of the world?
Benedict Evans: Because AI is driven, as you said, or machine learning, as you like to call it, you know, what's currently, of course, most most effective in AI. It's largely driven by compute and data, right? And the ultimate capability are driven capabilities are driven by the amount of compute and data available, as well as innovation, of course. And it seems like part of what a lot of people think about is if, if somebody can have drastically more compute, or drastically more data on a certain domain, that that could really give some advantages compared to entities that don't have the same resources. So maybe it's not so much AI then well, you know, resources would be the right word who has more resources to build what they want to build which might materialize itself and better machine learning. There's a lot of data there's a reason they can't use the system they can't even use this system because we are all learning challenges and it's actually extremely difficult even though very narrow domain like scanning because this is a solution for hospital systems. And so then what do we do with that? [Unintelligible]
Pieter Abbeel: Absolutely, yeah. Now talking about countries, one of the things you've actually talked about quite a bit, and that's, that's, I would say, at least in the machine learning AI space kind of new is to talk about legislation and regulation as becoming more important as we can build applications that influence everybody's lives. And that there is a real role for regulation to ensure our lives are better thanks to all this technology. Then at the same time, of course, you see, things come out, like the European legislation demanded only one type of charger connection for the iPhone, right? Just USB C, and that's it, that's what needs to be there. And so I'm kind of curious how you see kind of the, in some sense, the need for regulation, but also then the ability to regulate correctly and how to get in the right spot in that regard.
Benedict Evans: So you know, gender identity is is that every industry, every company in general, accounting, or health health says that you and but then every industry, if you work for the oil company, and you murdered, somebody will call you. back if you have a specific quite technical issues and relationships, oil and air, shipping and in fishing, and banking and medicine. And architecture, is like if you fuck out, the building falls down. So there's like specific rules about engineering. And you're not allowed to call yourself an architect unless you know the rules about who can call an architect. And this technology will kind of become saying that there will be a bunch of sort of industry specific laws because there are big important problems that aren't covered by federal legislation. So that's one answer. I think the second is that it is though Okay, fine. But so we regulate cars, and we regulate banks, but actually we don't that's 50 different things to regulate finance, okay, but the rules are you can have a credit card have got nothing to do with how the Fed manages the money supply, which has nothing to do with insider trading law or capital adequacy. Well, you know, these are different questions. The same thing, you know, for car, which is maybe a better analogy, you know, the regular but, you know, there's a song it is and as I was driving, you no will go away. And even they may make you say you're gonna find them to build her while problem. And I think that, you know, it's a fact. Well, that's, you can have that. So, there are a few questions here. And I'm thinking oh, well, yeah, but what is the result? And then, you know, the fundamentals of public policy are a complicated before the trade off and where people are able to say, Oh, you don't get it. tends to come across as peoples that are wanting special treatment but what generally are actually saying is no technology policy is just as complicated as education policy or transport policy or health policy you know these are all complicated fields of policy and as always also trade offs and you can't have everything that you want all at the same time you know, you're going to have to choose which which you want you know, if you want more employment protection or more demand dynamic economy Well, you can't have both you've got to pick one and the same thing for you know, how you want content moderation to work, you know, he's an Uber driver and employee labour law question. So these are all kind of complicated questions and they're not generally questions that have easy answers and almost none of them are questions that would get solved by splitting up Facebook which is kind of like saying you know, let's call love and pollution by breaking up fraud well yeah you could make up for it but you will probably think that's going to solve
Pieter Abbeel: Yeah I mean hopefully the the the car pollution will be solved soon ish with with electric cars which is something you've also written about that the trend towards electric and you've also commented on self driving and capabilities there I'm kind of curious generally in in the transportation space what gets you excited there right now and it's I mean, you've recently commented on self driving being a bit of a missing ball fully self driving not being the perfect nomenklatur for what for example Tesla currently offers and it's
Benedict Evans: I think Indiana we have a steering wheel all the way to picnic and when we get to the UI for our school if this ad manager is labeled without doing so level level it seems to me almost definitely not that level because all situations make me break. Does that sound right? You know, Does that sound right through me? It is wrong.
Pieter Abbeel: So when I when I listen to you, I mean you look at a lot of things from an economic lens that trends economically Do you see a lot of value created with no self driving technology even if it's not fully self driving in the next few years because you're talking about quite a few applications their garbage trucks golf carts. How about deliveries for example, with
Benedict Evans: Little Lego blocks on things and experimenting with some level of density in which Rain Rain delivery makes sense. [Unintelligible]