Mike Fisher on The Robot Brains Season 2 Episode 2
Transcript edited for clarity
Pieter Abbeel; We all remember the massive shortages of 2020. No one was spared. First it was toilet paper than hand sanitizer, and finally face masks. Some retailers chose to profit from the panic by marking up the cost of masks. Other retailers took a different, more socially conscious approach, like Etsy. Etsy show just how you need its AI powered ecommerce model is by adjusting in 24 hours to become a top supplier of masks at reasonable prices. Once known as a digital flea market that sold collector's items to cat lovers, Etsy has matured into an AI driven, customer obsessed major retail player. The mask scenario is just one example of how the side can adjust on the fly. Our guest today, Etsy CTO Mike Fisher is here to share how Etsy has transformed itself in the last few years how AI has played a central role, and its big plans for this year's holiday season. Mike, thanks for joining us. So great to have you here with us.
Mike Fisher: Yeah, thanks for having me here. My pleasure.
Pieter Abbeel: Well, right now you're the CTO at Etsy. But actually, you have quite the illustrious career leading up to this. You were a captain and pilot in the US Army. You were VP at PayPal, you had various leadership roles at General Electric. How did that all come together? Working in the army and now being CTO of Etsy?
Mike Fisher: Yeah, I've definitely had what I consider just a really interesting career. I've been very, very lucky in that. I think it all started with, you know, in high school, I sort of discovered computers and programming, I taught myself how to programme. And then I attended West Point, the United States Military Academy, and majored in computer science, so always had this, this love of computer science. But you know, attending West Point, I had a commitment. And so I was commissioned to the army as an aviation officer, and they taught me how to fly I flew helicopters for about seven years on active duty. And I love my time in the military, I love my service. But I also always had this love of computer science. And I even did my masters while I was in the military. And so I had a big a big decision between staying in and, and remaining as an aviation officer or getting out and ultimately decided that I wanted to pursue a software engineering career. And so that's why I left and joined General Electric as a software engineer. And I've, I've continued to love programming, and really thought I just wanted to be an engineer, and then maybe someday, if I was lucky to be an architect. And I thought that would be my career path. And then one of my mentors, reached out and said, you know, you really should look into and consider injury management as a career. And she convinced me to do that. And then just, you know, really helped me my first couple of years in injury management, learning how to manage software engineers, I had a lot of leadership experience from the military that carried over, but really managing software engineers and projects and so forth was different. But that started the jump started my career into sort of leadership and management within the industry. As you mentioned, I then joined PayPal as VP of Engineering and Architecture, and got to see this enormous amount of growth, both in the organization, I think, when I started, there was about 30 engineers, and we had several 100, when I left, as well as in the transactional volume, and scaling the site and keeping the site highly available. And then I left there for ad tech startup, as you mentioned, we go in New York, was an ad tech startup startup that I joined and became their CTO. And also got to see the massive amount of scaling from producing ads around the internet. And that was acquired by AOL. And I left that and said, I want to apply all this scaling knowledge into the industry. And so I started with a couple of friends of mine, a consultancy, called aka partners, and we are sort of specialty was around scaling. And so we wrote a couple books about it. I ended up and continue back for my doctorate in management at that time, and really wanted to be this practitioner scholar that taking the sort of knowledge that researchers were producing and bringing it into the end industry and helping people with their organization and technology scale. And I did that and for almost a decade, and one of the companies that I met during that time when they were just a start up in 2008, was Etsy. And I got to know the company. And always I just love the mission around, you know, giving this marketplace for creative sellers to produce and find their, you know, their audience. I loved it, I got to lead a company, I came back several times for either speaking engagements or to consult with. And then in 2017, when Josh Silverman took over as CEO, he brought me back in as a consultant and kind of one thing led to another and he gave me opportunity to stay. And I said, kind of only, only, I only thought for it. You know, I just love the mission. I love the culture. It's such a strong engineering culture, that's just got amazing people. And it's now been four half years. And nothing has changed my mind about that amazing mission, amazing culture, or just super strong, visionary talents that I really enjoyed to get to know.
Pieter Abbeel: And Audible has been some the most successful and amazing years for Etsy as a company this these last few years. Now, for our listeners that are not familiar with Etsy, can you describe it?
Mike Fisher: Yeah, so Etsy is a global online marketplace, for buyers and sellers. And we have just to give sort of a scale and mentioned that, sort of in this scale, we have about 100 million items on the marketplace. And to give a sort of context of how large that is, you know, the, I think the average grocery store has like 40,000 items. So that's like having 2500 grocery stores, and massive amounts of of items. Don't don't have skews, you know, the creators, the sellers. You know, it's all handmade eventually, items, they get to make and describe it how they want, because some of its very unique. And we have about 100 million active buyers, and seven half million sellers. And we we bring these together. And our mission is to keep commerce human. So we bring them together one buyer, one seller at a time. And as a marketplace, we want to find those buyers and get on their buyer journey, and find the right product that they're after. And it might be still it could be something quirky, you know, a gift for your cat, but it might be something just amazing and beautiful. That you know, that's handmade, like a dining room table, or you know, or amazing art. It's almost anything you can think of you can all find on Etsy, and somebody around the world is making it. And it's just beautiful and amazing. I'm always so inspired by our sellers and what they can produce.
Pieter Abbeel 07:55
Now, you wouldn't tell from from my background here, probably because it's pretty empty. But if we were actually set up in my my wife's office here at home, you would see a lot of pieces acquired on Etsy that are hanging up against the wall. She's a huge fan and has bought so many so many pieces from there that are absolutely beautiful. And it's it's just any time she buys something from there, it's it's great. It always improves her setup. So now, you then effectively have two audiences as a company, right? You have both the buyers and the sellers at the same time to cater to a make sure that they can do what they want to do. Can you say a bit about that? And what what is your focus?
Mike Fisher: Yes, absolutely, we have you're exactly right, as the marketplace, we sit in the middle. And we bring those two parties together. And we have groups of product engineers that are focused on the buyer as well as the seller, although there's of course, a significant overlap. And, you know, for the seller side, we know that most of these individuals are creators, artists, craftspeople, who want to do that they don't want to run the business. So we try to make the run of the business side as easy as possible. Because that's way secondary to them. They'd rather be in their workshop on their kitchen table making something that's what draw drew them to the marketplace. And so our goal is to make it easy. And also bring them that audience I mentioned like, you know, of course they you know, they want lots of things but like getting someone who loves their items as much as they do is really fulfilling. And so finding that item through our search and through you know, through ads is really really important for them. And so we've got to do a good job for both. And then on the buyers the same thing. Everyone's busy and in this world, you know, sort of 24/7 but seize the place for special and people do come there to be inspired and find something that you can't find anywhere else. But we also know that you don't have all day. And so we've got to get that 100 million items down into a manageable list for you to look at. Some of it might be something you're not even thinking of. Or you might be on a very, very specific journey, like I saw this in a friend's house, and I want to get something very similar. And so we've got to figure all those different buyer journeys out. And you mentioned in the intro, that we are a heavy, heavy, Big Data machine learning company, I describe, that's the kind of an iceberg, where what you see up top above the waterline is the marketplace, what you don't see is that we're really a big data machine learning company. And again, just to give you a sense of the scale, on the data side, we process over 6 billion events of data a day. And we use all of that in machine learning algorithms to power everything from recommendations to search to your, to all of these, you know, sort of parts of the buyer journey. And, you know, that's what we try to do to keep commerce human we power it with, you know, this real, really large data.
Pieter Abbeel: Now, when you joined back in in 2017, what was your first order of business? What did you want to get done? Right away?
Mike Fisher: Yeah, you know, and I was actually brought in to look at it analyse for Josh, the infrastructure versus product development, and to see if we could be able to invest less in supporting the infrastructure and more in the product development, right? The people, you know, the buyers and sellers both really want more features. They want that functionality, and the better search and all of that, what they don't necessarily care about is, you know, how the site is hosted as long as it's available. And so when I looked at that, I decided that one of the first things that companies should do is migrate out of the data centres, they want three data centres into the cloud. And, you know, so that was our sort of first order of business was a really massive migration. And we did that in under two years. And we migrated not only the the marketplace, but all of the supporting infrastructure around that the processes, all that big data, machine learning, and got that to the cloud. And, you know, the reason for that was really about three of more of our engineers to be able to work on stuff that I think about as higher in the stack, I say that we're we're moving engineers closer to the customers. And that's what we've been able to do is focus more and more of our engineers up the stack closer to the customers to make what they really, really care about.
Pieter Abbeel: And can say a bit more, what is it that the engineers now get to focus on them?
Mike Fisher: Yeah, we certainly still have infrastructure and SRE engineers who support the site and keep it highly available. I also tend to say that availability is our number one feature, right, that, you know, we first and foremost, we keep the site available, you know, but we can also focus on performance, and we can focus on as we grow the Protestant jury team, keeping them efficient, which is another aspect of scale as you grow this organization. You know, one point right after I joined, we're, we're down to about 270 engineers. And we're now up over 600 plus engineers. And as we scale that organization and build more features, we need to keep those engineers efficient. So the engineers can work on stuff that helps each other. And then you know, those products engineers, we can expand that we can work on things like recommendations and search and redoing our, you know, our, our mobile app to sell an Etsy app and, you know, continue to invest in the buyer on Etsy, the buyer app, and all of these features or functionality that we've been able to work on. And so we've been able to apply more and more of our percentage of engineers into product development.
Pieter Abbeel: Now, one thing that when we're reading up on SES recent accomplishments stood out, of course, is the ability to adapt so quickly, in case with a pandemic, to start offering a very large number of masks. Right. How did that come about? How did you first even realise this was possible and how I mean, the sellers need to change what they're doing, I imagine otherwise you can't offer it.
Mike Fisher: That's right. You know, we're in March of 2020. You know, right as the pandemic in the US started, if you had come to Etsy and search for face masks, you would have either found a Halloween mask, which we have amazing Halloween costumes that make or sells for like a facial cleanser mask, because again, you know, some some sellers make items beauty products. What you wouldn't find is a protective mask that You would stick up today. And so we immediately when we saw that this was becoming a trend that people were coming out, see, to look for this, we started working on our search. And we basically trained, we took humans, we trained several 100 on these new mask, Trader algorithms, and then fed that back in. And overnight, we're able to retrain the algorithms by jumpstarted, with some human called human in the loop. Some humans annotated with these mask are, retrain that so that you would see this. And then, as you mentioned, on the sort of supply side, we reached out to sellers, and said, Look, there's this massive demand. And we think that this could be an opportunity for it to do good in the world. You know, we do lots of good in the world. But this is a another particular time. And so we reached out to sellers and said, you know, as the pandemic is hitting, you know, many of you have access, or our seamstress, and you know, access to, you know, sewing machines or machinery that can make this, think about switching and making this an overnight we have people that might be sewing wedding dresses, start making asked and able to, you know, to the switches, and this is a great again, the great thing, not only do we have an engineering culture that can overnight pivot to this, I'll talk about some more of the things we have to do. We had to add banners to make sure people were safe, and then even add functionality for sellers not to get overwhelmed, that were weren't used to seller selling 1000s of items a day. And so we had to, you know, from our Member Services team and our trust and safety teams, we had to put in a new process, the engineering teams had to make ways to, to, you know, to sort of shuffle people into search so that they didn't get overwhelmed from a seller. But the sellers can pivot so quickly that they don't have these massive supply chains, like most ecommerce or retail did. And that's why in the early days, so many people couldn't find mass, but you could find them on Etsy. And I think you mentioned, you know, or if you've seen the data, but like, yeah, 1.7% of our gross merchandise sales in 2020 with a mass. So a massive amount of that year.
Pieter Abbeel: Yeah. 7%. That's, that's absolutely amazing. That's quite the switch up there. Yeah. Now, one thing I'm really intrigued by, as you said, you know, you have human in the loop, a human in the loop process, to essentially reteach the AI system powering the search. Before we get into the human in the loop part, I'm really curious about the AI systems overall, that you're deploying at Etsy, where are they? And what are they doing?
Mike Fisher: Yeah, I mean, you know, I have to probably start with search, because that's been, you know, just a massive focus of ours over the past four years. Yeah, I think the team has really taken us from probably, you know, early, early sort of stage search into the modern search era over this past four years. So I think, you know, they've jumpstart us with that. And the way we think about search is really sort of these three stages, there's the information retrieval layer, which is basically, you know, grabbing the very, very basic, it's something like solar that's grabbing terms, you know, in a TF IDF sort of way. And then second layer is around, you know, this candidate selection, and we call that kind of a first pass. And then the second pass is a re ranking. And so, in all of these, we've been applying machine learning and the information retrieval layer, we introduced something called neural information retrieval, and IR. And this uses deep neural networks that are trained on user behaviour data and language patterns to encode search queries and listings via dense representation. And when we applied that, we were able to see a 15% decrease in no result queries. So you might type something super specific, and not see anything if you're just using the basic, you know, TF IDF results. But by applying this, you know, we, you know, which, you know, conducted this user behaviour and language patterns, were able to reduce that significantly. And what this addresses are things that, you know, that traditional term base retrieval just doesn't do things like, difficult to understand antonyms hyperdub, synonyms, things like that sensitivity to spelling errors for different word formats. And then, you know, the fragility of this morphological variants like woman and woman. And, you know, this, this is very, very data hungry. I mentioned, we process a lot of data every day. But, you know, it was super, super useful, and it wasn't meant to replace this exact term matching and we actually merged them together. And that was one of the first things we did recently this year, we've introduced what we call x walk. And the whole goal here is we call it, you know, bridging or reducing the semantic gap, the term that you put into the term that the sellers use is the semantic gap. And I think about this and things like, you know, you know, blazer and sports coat, like as a seller, I might put my listing as a blazer, but you might come along and search for sports coat, and we need to build a bridge, that semantic gap, it could be an eye touch, just one example. You know, it could be anything from colour to many other things. And so one thing that we introduced this year was, again, not to replace this information retrieval, but to supplement it with what we call x walk. And x Walk is a large scale, real time graph retrieval engine. And it provides more relevant searches by blending some of the listing facts with the buyer intent. And we did go public with this metric. So I could talk about it that this, you know, large scale graph retrieval engine, was able to provide incremental 50 million annualised GMRS. So 15 million in more sales for our sellers by this, but again, very, very data hungry X walk allows us to use an 11 times increase in data leverage. So we went from pulling in about 150 million data points to 1.6 billion for a given query. So massive amount of, of data that's processed. And so that's all at that like information retrieval layer. At that first past was helps with candidate selection. Over the years, we've introduced this, what we call the semantic candidate set. And we did something with a VPC, G or vector complicated clique graphs. And we embedded machine learning models into the candidate selection. And we built these factors off the data that we had around the around users clicking, that's why it's the VPC G. And that was able to so when you get the information from the IR layer, you get this massive candidate set, and we reduce it with the first pass, we did not with these epcg. And then we pass into the second pass. And this is about rewriting. We've done a couple things here. You know, one of those, we introduced a lambda mark, which is a pairwise Learning to Rank algorithm. And this, this has really helped us, right, because, as I mentioned, the buyer, they come to us to be inspired, but they don't have all day, we'd love them just live all day, but they don't have all day necessarily. So you know, we still have 1000s of items in this character. And we need to put at the very top, you know, because like most search engines, people click on in our case, purchase from mostly the first or second page. So we rewrite them to get the most likely result. And so we use this landmark algorithm to do that. And then most recently, we've began introducing the real time features for model inference. And what this allows is to have in session personalization, so prior to the, you know, we would take yesterday's data and process it, and then use that to represent groups to you. So if you search for a blue sports coat, and clicked on certain things are purchased items, I can use that data so that when I come in and search for it, I get the benefit of everything that you looked at. And you know, for my personalization, but now we can do it in session. So if I type something, I might type leather, or click on leather goods. And then if I just type in wallet, my search, I can now use this idea that oh, but this person has, you know, has enjoyed looking or searching for leather items that put leather wallets at the top of the search ranking. And so all of this is enabled just better and better candidate selection, heavy use of machine learning. And this personalization, you're in real time.
Pieter Abbeel: Now, that's amazing. It's really interesting, the way you lay this out, Mike with I mean, the clicks, the user clicks are essentially guiding what the positives are right? And what the right connections are. I am curious, though, about how to get Do you need any negatives are there you know how to get because if everything is positive at some point, you don't learn anything either. Right? So how do you bring the negatives for the training data?
Mike Fisher: Yeah, that's right. And we use like in DCG, as a way to analyse. And so when we train and replay different algorithms offline, we look at items that were very high in the list, but didn't get clicked, or favorited or added to cart or any of this. And that is a negative signature to us that someone passed over that item. And so you're exactly right, we use the positives. We also use the negatives to see, maybe this item didn't belong in the first or second row. Maybe we should be a little bit lower.
Pieter Abbeel: In the first part information retrieval part and neural information retrieval system that you described? I'm curious, is that trained in an unsupervised way? Or is that also trained, supervised on on clicks?
Mike Fisher: Yeah, that is trained in a supervised way with a clicks and with, you know, basically taking more of that user behaviour. And, and using that in to train the models.
Pieter Abbeel: Now, everything you described so far makes me think of text, the way you describe it. I'm not sure if I'm correct about that. But Etsy, obviously, also very visual when I visit the site, even more, look at pictures, and then read things. Right. So how do you bring in the the visual part?
Mike Fisher: Yeah, that's a great question. So you're right, much of our work in the early days was around text. So it was about the text of the the, the sellers wrote for the listings, it was about tax for tags, it was about the search query, you know, the text in that. But what we've done over the past about 18 months is focused on computer vision. And so, you know, we've done, we have we ongoing, lots of projects around this one of this is really building a classification model, based on, we use ResNet, 50, and one on one and things like that as the basis but then train it on Article items. That, again, we're doing things with, like the human in the loop. So we're having humans help us with, you know, identify, in this case, over 100,000 Human annotated images, for further training of those models. So you know, we take something that's pre trained, and, you know, in sort of more generic items, and then we specifically trained it on ours. And so what we're doing is we're trying to identify this listing image, so that we can feed that as a feature into our models. So we don't rely on just you know, what the seller describes it as we can actually use what we think the images, we're also using another application that we use in computer vision, is mature content. And so we're using image content cluster prediction models that can help us flag things that should be only for mature content. Another interesting way that we're using this is home decor, and trying to identify style. And you know, whether your style is farmhouse, or, you know, modern or something else, we're trying to identify style. And, you know, our current candidate said is reaching, you know, above 70% precision, to identify these types of styles. And that can be used, if we notice that you're clicking on again, this real time clicking on or interested in a particular style, we can start feeding more of those in that because maybe this particular buyer journey, you're looking for a particular style for you or a friend or family or gift or something. So style is a really interesting one. And then the last sort of one I can talk about, I guess, is colour. So as you can imagine, colour is used with it's, it's, you know, these colours and attribute is applicable to like, 90% of our listings. So almost everything has a colour, but only like 56% of them have actually colour labels on them. And you know, the most used variant, which is that drop down to see like, Oh, can I get this in different sizes or something, but the most uses colour. And so we're building computer vision that takes the primary colour of that listing, and provides it as a tag on the image. And that way, whether you use it in search for recommendations, or any other, you know, advertisements, or wherever you want to use it, you can identify the primary colour of this item, and help shoppers buyers search that way as well. So, you're right, we're moving into, you know, way, you're kind of away from just a pure text based and heavily heavily into computer vision. Now,
Pieter Abbeel: The examples you give the things you do with computer vision, they're they're quite, I would say, disconnects us from the typical academic computer vision data set classification level. So it seems like you probably have to develop a lot of your in house research to see what what neural net architecture is, as well as what kind of data annotation schemes are most effective for what you need for your buyers, your sellers. And I'm curious, are there any any lessons learned or any kind of interesting technical learnings about your kind of process of the whole computer vision stack that you have?
Mike Fisher: You're right, we spent a lot of time taking, you know, sort of the academic based, you know, algorithms and data sets and so forth and then having to apply something specific for outsi. You know, I think for us, one of them is certainly around the human and loop I mentioned, that we need Humans that it's not It can't just be pure computer classified or identified that our items are typically, you know, so unique, that it's not something you can find in a general dataset that gets recognised because you get this interesting world at Etsy where you can cross categories very often, something might be a lamp, but also a piece of art, or one of our favourites. Another feature that we've been working on is the gifting feature. So if you're looking for a gift, you can put in ideas that will recommend stuff. And you know, something that the team has been demonstrated as a interesting sort of what you know, probably the salts are things like pizza earrings, you wouldn't think of you know, like, but some people who loves pizza and might want to wear it as earrings. Of course, some seller on Etsy, many, probably many sellers make pizza earrings that look like pizza. And this is something that's kind of unique, you're not going to find that in a lot of datasets. And so having someone that human in the loop to be able to identify some of these more unique things where we cross categories, has been really important for us that it can't be just 100%, sort of computer driven, driven.
Pieter Abbeel: Now, one of the things that I have noticed, IKEA has an app where you can effectively show your house and see how furniture would fit in your house, which is of course complicated, because you need to understand the sizing and everything. I'm curious if you think about the computer vision side is Dre, is there a buyer side also where a buyer can take pictures of their house or the room they're trying to decorate. And instead of typing a search query, they would just put in a picture and say improve my room. And something would come back out.
Mike Fisher: Yeah, I love it. And I think the answer is definitely yes. You know, we have in our mobile app for buyers, the buyer on Etsy app, the ability for AR and so you can do exactly that with a piece of art or something, you can see it on your wall and see the dimensions. I've used that recently to see how the how large of the print I should buy from my wall. We also have done some, some work with third parties around. It's called the Etsy house, which is this sort of 3d virtual walkthrough of a house that's fully furnished, and everybody who with Etsy products, everyone who's seen it says basically Can I just buy that entire house with all the furnishings in it, because it's it's so beautiful and amazing. But yeah, I love the idea we've talked about like, eventually, should a buyer be able to take a picture of a lamp or, or a couch or a piece of furniture, and a spine that style and something that would match? And I think that is absolutely where we're going with computer vision and style. And, and all of that, that you'll be able to recommend for maybe for those of us like myself, who aren't necessarily the greatest designers or interior designers like I would love something to be say this really complements. So yes, I think that's absolutely, you know, in the future.
Pieter Abbeel: Nothing that has really stood out in the last year, I would say for me, you know, AI, Twitter is the ability for people to create art, with AI systems helping with the art creation. So where you can have you just type something you say, you know, I know that one of the famous examples of courses, you know, you want maybe an avocado chair and you know, somehow generates a picture of an avocado chair. But that doesn't necessarily really exist. But somehow it knows how to generate that picture. And more generally a visual ability to create visual scenery based on text, right in all kinds of styles. And that makes me curious, kind of, more generally on what do you see as the future for AI in terms of enabling your creators, your sellers?
Mike Fisher: Yeah, I do think, you know, when I talk to my teams that I think we are going to use AI and machine learning on almost every part of the site marketplace all the way from the start of if you think about the sellers journey, when she starts to write the listing, that we could use AI and machine learning to help them accurately describe or more visually describe their items to make sure the tags are correct. We could look at the quality of the photos. You know, all of this we could talk about, you know, the categories that these items are in, like you said it maybe even earlier in the creative process. We could give them ideas on on projects based on what we've seen with their, you know, their sort of current listings. And then all the way through Of course, heavily on the buyer side, you know with a search in the recommendations and The gifting and all of these, you know, there's almost no part of the site or the sort of marketplace journey that I don't think we could, you know, you wouldn't be helped by more machine learning. I mean, we're using I mentioned mature content, we're using machine learning. And we have been for years on the trust and safety side, to make sure that the marketplace is secure. And, you know, and you know, as much intellectual property is protected, you know, all of this is heavily used by machine learning models, and we're just continuing to invest in there. So, yeah, as I mentioned, the mission is about keeping commerce human. And it's just one buyer, one seller at a time. But it's powered by such strong engineering data science machine learning Big Data. Yeah.
Pieter Abbeel: Yeah, that kind of makes me wonder if you think further ahead. What do you see as the bigger picture future of online marketplaces?
Mike Fisher: I do think, you know, there's people, people are short on time and attention span. And so there's only a few places that they can sort of think about where to go. And I think more and more, that is becoming one of those places that comes first to mind when people say, I need to purchase something. And, you know, I think there's a couple of reasons for that one, you know, certainly the drive for facemask, as we've talked about was the sort of impetus that helped people sort of remember at sea a little bit. But what we've done since then, and now that, you know, the mask are much, much smaller portion is we brought them back. And we've shown them through, you know, not only these amazing items that the sellers have, but this all this technology that we've brought to bear on the marketplace, that this is a place you can go every day. And in this world where you know, there is so much automation and you know, you it's dropped off in your doorstep, that's great. But I do think people, people still want something that's special. They want to talk to sellers, and they want to customise it, they want to personalise it, you know, they give the gift of themselves maybe or they give a gift to a loved one, you know, a family member or friend, someone like they weren't any special. And like that's the is the place to come for special. You think about like, you know, someone had a wedding or a birthday or a baby. And if you say you got a Nazi people know, it's special, they know that you've taken the time, you probably talked to the seller, you know, there's a story behind it. It's a small business, there's so much positive. And I think that is the futures there's just fewer and fewer places that people are going to keep in their minds to go. Certainly, there's the mass of retailers that like all of us need for commodity items. But then there's places that just bring joy, and special and I think that's just one of them. And so I think that's one of the things that we're gonna see in these marketplaces is there's just fewer places that people will move to go.
Pieter Abbeel: See joy, getting economic getting special gifts, we're heading into the holiday season, obviously, is there something that Etsy does, specifically to prepare for that?
Mike Fisher: Yeah, we start in well, really, the entire year, is built around our product roadmaps and feature development for the holidays, but from a pure infrastructure. We start in around July, and we start working on plans for you know, this period. In the holidays, I mentioned, availability is our kind of most important feature. And so it's really, really critical that we have high availability and performance. And so we have a team dedicated to this, starting about July, we work with our partner to host the cloud, we're on Google, we work with them, we play on we give them all of our sort of predictions on our size. And we even run, you know, these sort of, you know, game days in which we load up traffic, and you know, artificially, and we ramp it to a much higher percentage that we think we're going to even need, and it shows us where the cracks are in the system. And then we get time weeded out early enough that we have the teams can focus on this. And this might be you know, something the caching layer, database layer, it could be a service, a micro service, that whatever. And we really kind of push that all out, or or ramp that up. So we can find that and find it in time to fix it. And we've done all that. And then coming into the actual season. We of course monitor everything that's been known historically for monitor everything we do monitor an awful lot. And we watched the graphs and we were constantly sort of very vigilantly watching and monitoring for for things, and then we're very good at reacting. Should there be an issue you or the team is just really, really good at reacting. everybody pitches in it's part of I mentioned that wonderful culture about the is we have this blameless culture, you know, it stems from our blameless post mortems, when there will be incidents, and there will be issues. And we take away from there and do a post mortem that we call blameless post mortem, because we're not looking for who to blame. We are looking for why something happened, what surprised us? What can we learn from this? And by having that amazing sort of open culture, if something happens, people jump in, actively say, Well, I just did this, could it be that, you know, I just, I just made this change, and we are continuous delivery, or continuous deployment. So we're pushing code, you know, a lot, you know, up to 50 times a day. And so any of these changes might might happen. And so, but nobody sort of hides and says, you know, could have been my change, it was always I made this, what can I do to help and everyone pitches in? And because of that amazing culture, you know, we respond very quickly. So lots and lots of months and months of prep, for the holidays, so that our buyers and sellers have a wonderful holiday season. You know, that's, that's what we're here for.
Pieter Abbeel: Well, I love this blameless postmortem culture and sounds amazing. That also makes me wonder back to actually your earlier days, I mean, as a captain serving the Army as a as a pilot, helicopter pilot. I wonder if Are there any kind of lessons that you've brought in from that into what you're doing today?
Mike Fisher: You know, I mentioned leadership, and I think, certainly from the military, from West Point, to my time, active duty and National Guard, that they taught me an awful lot about leadership and I that sort of is applicable, almost anywhere. And, you know, the other thing, specifically, as a pilot, I think, breaking down the situation into a simple, you know, sort of, you know, the simplest parts of it, and trying to stay calm. I mean, there's very stressful times, you know, within technology, just like, you know, within a pilot's, you know, sort of career, but trying to stay calm, trying to, you know, to stay focused on what the problem is. And again, often debugging, you know, it's very similar in aircraft as it is on the site you're trying to debug, and sometimes the signals that you're getting aren't from the actual cause of it. And that can be the aircraft as well, you might have a master caution, but have to actually try to dig in and figure out like, Okay, what's really happening? You know, is it just about sensor? Or is there something really serious going on? So a lot of that, trying to stay calm in this situation, and thinking things through and try not to jump? Oftentimes, unfortunately, the aircraft's it's not the actual issue with the system. It's not the human side of it, that, you know, they do something reactive that they didn't maybe even need to whether to do something incorrectly. And so, you know, trying to remember that and incur those lessons over I think has been helpful in my career.
Pieter Abbeel: Well, it seems if you're, if you're used to debugging things in the air where your life is at stake, it must feel pretty relaxed to debug server issues.
Mike Fisher: Sometimes there's a lot of stress. And I know there's a lot of people out there. But the team does a great job. They're just really proud of that culture. The people that support the site, the site. Just how much of a big data machine learning company and how important machine learning is to Etsy we're like I said, you see it and you think about our mission you might not think that we are such a technically advanced company, but we really are so I appreciate the opportunity to sort of talk about that