## Intro [00:00] AI coding tools. I'm interested how much do you use the current ones? What do you use? What is working? I think Apple's built-in Xcode one is perfectly fine. I use it a tremendous amount. Like I feel like chat GPT has officially replaced my knowledge of Git. I can never remember how to use Git invocations properly. So I just pop open a window and be like, "Hey, can you please re-educate me on this?" I use a lot for that kind of like ad hoc things where I don't have an idea of where to start. I've noticed that they are starting to do a lot more when you ask it a question it can generate a lot more complicated like it'll give you a integrated kind of web project or something like that. I'm excited to look back into that. But yeah, on the mobile side I've asked it a couple of specific wrinkly questions that I actually have to solve on a day-to-day basis. I've never once gotten good code for those, but it's a good rubber duck if you're trying to like break down a problem and carve it up into chunks and ask it focus questions. You might get decent code. So in personal projects, I've used Gemini's built-in Android Studio. It's honestly pretty good. I don't use it to ask questions and have it write code for me. Honestly, I still don't believe that that's going to work very well, but as like a very fancy, efficient autocomplete, it's fantastic. Reddit completely reroll it iOS and Android apps starting in 2021 and did so without most of us noticing any of it. But why did they do it and how? Today, we reveal details by talking with three members of Reddit's mobile platform team, Lauren Darcy, Brandon Kobalinsky, and Eric We cover the sheer size of Reddit, how they employ more than 200 mobile engineers, have around 2 and a half million lines of code for both iOS and Android, more than 500 screens, and hundreds of modules. How Reddit changed their API from REST to GraphQL, and why this was more tricky to do than most assumed it would be. Why Android chose Jetack Compose when Compose was still in alpha, and why iOS decided not to use Swift UI at the time, and why the iOS team is reversing course now, and many more details. If you're interested in how a fully remote mobile engineering team operates and makes decisions and want to understand the challenges of building mobile apps used by tens of millions of users, this episode is for you. If you enjoy the show, please subscribe to the podcast on any podcast platform and on YouTube. Welcome to the podcast all of you. Thank you. Thank ## The scale of the Android code base [02:04] you. Happy to be here. So to start off, can we get a sense of what is the scale and shape of the Reddit codebase? And you right now we're talking about the mobile codebase of course. Uh yeah, I guess I can start with Android. Um there's some like standard scale metrics uh I don't know two and a half or so million lines of code about 800 modules. Um but the thing that really surprises people is we have about 580 screens. Um you know you you think of Reddit and you think of like the feed and the post detail screen. Uh maybe a handful of others but to this day I do not understand how we have this many screens or where they all go but we have a lot. I don't know Brandon you want to give ## The scale of the iOS code base [02:42] the iOS take on that? The iOS site is actually I think a pretty similar size. We're behind on having like a more specific concept of screen that like I would trust. But my favorite answer to this question is like we're we're large enough that if you ask me like what scale we are, I have to ask you clarifying questions of like, well, do you care about generated code? Do you care about specific kinds of targets? Because like we've got a thousand basil targets, but I would have to take 15 minutes to categorize them or ask Brenley uh uh to figure it out. But yeah, it's it's big enough that we don't have to prove that it's too big. And I guess one one question for this type size of code bases for those who are not as familiar with let's say mobile is the compile time on on back end and on web it's usually not a big deal although on typescript it could be ## What the compile time is for both Android and iOS [03:26] but on mobile usually is you know these type of code bases a million line plus often would compile 10 minutes 15 sometimes even longer than that what is your compile time like from fresh again I this question causes me to make a face because if you end up being a Reddit iOS developer that ends up being like, hey, my build times, because I think if you do the entire app, everything, it's probably about 30 minutes. Um, but that'll cache a bunch of things and after that it's going to be way faster. Um, but uh if you are exposed to a 30-minute build time, we're probably going to see you on a chart and be like, "Hi, why are you building the whole app? Would you like to be more productive? Can you try a couple of these different techniques?" Uh, because it's really hard to make that any faster. Yeah. On the Android side, um, incremental builds are are not bad at all. I think our clean builds are somewhere in the neighborhood of 8 to 9 minutes. Um, but that's with, you know, remote build caching and a lot of things that we have to to kind of help things out. Yeah. But, but just to clarify, as as you said, random as well, you wouldn't really do a clean build except maybe on your first day or, you know, when you join or if you've been away from, I don't know, a long time or or you do something. So, it's just a rare event, right? Like most devs on the day-to-day, you're going to do like small incremental builds that we're talking are not beyond minutes and probably even faster. Yeah. And essentially we we try to get people into uh uh like selected focus targets or like I think we we have a couple of names for this because we've had it for a long time. We've had varying versions of this tech like our playgrounds. We want you to take like a selection of those targets and do like a focus mode. I think other places call it. Um, and in those builds like you'll vary from if you're in the feed area where you have a tremendous amount of dependencies, maybe that'll take 10 minutes, but the small libraries that I might be working on, it's like 30 seconds to a minute. You like it doesn't take any time at all. If you're outside of the main project and as as context, you mentioned, you know, number the lines of code, which we know is not a perfect metric, but above a certain size, it does give you a sense, right? Like Uber has a similar size for example and again a lot of Starbucks will have a lot lot lot a lot smaller. What is the size of a team that ## The size of the mobile platform teams [05:33] maintains uh all of this right all of Reddit's native apps? It's a great question. Uh we have two dedicated platform teams for Android and iOS. Uh they're about 10 to 11 engineers a piece on a good day. Uh and they support about 200 mobile engineers across the company on different teams. Wow. Uh I I want to say we have about 20 feature teams and they are set up like a fairly typical feature team with some Android, some iOS, some web engineers, maybe some dedicated backend engineers, uh an EM, a PM, a part designer, that sort of situation. Uh but what's special about the platform teams is they tend to be homogeneous. So everyone on the iOS platform team is an iOS developer. They're all together. Oh wow. Um, so they develop a little bit more subject matter expertise to support all of the feature iOS developers in their spaces. Yeah. Because at most places the mobile platform team would have a mix of like 50% Android, 50% iOS. Usually the one the ones I've seen. Yep. Uh we we work more like sister teams instead of having one team that is both platforms. We also have a web platform team and a UI platform team that owns the design system. This episode is brought to you by Graphite, the developer productivity platform that helps developers create, review, and merge smaller code changes, stay unblocked, and ship faster. Code review is a huge time sync for engineering teams. Most developers spend about a day per week or more reviewing code or blocked waiting for a review. It doesn't have to be this way. Graphite brings stack pull requests, the workflow at the heart of the best-in-class internal code review tools at companies like Meta and Google to every solver company on GitHub. Graphite also leverages high signal codebase aware AI to give developers immediate actionable feedback on their poll requests, allowing teams to cut down on review cycles. Tens of thousands of developers at top companies like Asana, Ramp, Tecton, and Verscell rely on Graphite every day. Start stacking with graphite today for free and reduce your time to merge from days to hours. Get started at gt.dev/pragmatic. That is g for graphite t for technology.dev/pragmatic. This episode is brought to you by Sentry. Buggy lines of code and long API calls are impossible to debug and random app crashes are things no software engineer is a fan of. This is why over 4 million developers use Sentry to fix errors and crashes and solve hidden or tricky performance issues. Centrica's debugging time in half. No more soul crushing lock sifting or vague user reports like it broke, fix it. Get the context you need to know what happened, when it happened, and the impact down to the device, browser, and even a replay of what the user did before the error. Central will alert the right dev on your team with the exact broken line of code so they can push a fix fast. Or let Autofix handle the repetitive fixes so your team can focus on the real problems. Sentry helpmonday.com reduce their errors by 60% and sped up time to resolution for next door by 45 minutes per dev per issue. Get your whole team on sentry in seconds by heading to centry.io/pragmatic. That is s nt r y.io/pragmatic or use the code pragmatic on sign up for 3 months on the team plan and 50,000 errors per month for free. I I have to ask this question because ## Why Reddit has so many mobile engineers [09:00] people are going to already asking it in their heads. Why does Reddit have so many mobile engineers? And by the way, I we I did get this when I was at Yuber as well, but and you've you've gotten this so many times before, right? But again, Eric, you already mentioned the 580 screens, which some people will just not really believe, but we we've all mentioned now more than like 100 mobile engineers, which is a lot. Like this this will make you one of the largest native mobile engineering organizations globally. pro maybe not the single biggest one but probably top 10. It's really scary when you say it that way in in in one team that is right like you will have I don't know Google if you if you add up they'll have a lot more combined but but not in one app. Yeah. So so we talked about the screens we have a lot of feature surfaces. Um, but there's also a lot of uh mobile developers working in areas like building our safety features and building our eventing and our logging and our experimentation platform. Um, so not everyone is focused on like and our ads platform also is an entire organ to itself. Um, so while a lot of the bigger teams are teams like the feed and the post detail experience, the things that many Redditors know and love about Reddit, there are there's an entire team just devoted to supporting mods. Um, so when once you get into all of the different sorts of areas plus an entire dev platform that is trying to extend the experience to third party developers to actually build on Reddit, um, it gets big fast. The other big investment we saw, I want to say like a year and a half to two years ago, is we have a lot more uh, we we don't call them estats, but we actually have a lot more automation folks. So a good portion of our actual builds are being are part of our test infra investment. Um which a couple years ago we didn't have much test infra and now we have a wide variety of test infra. Um so they account for quite a few of the builds that we see running today too. And then but by test infra and automation is this end to end test integration test some some fancy AI stuff like what what what kind of mix? I'm assuming it's going to be a mix right? I I was about to say yes until you added the AI part. Uh Brandon, Eric, ## The different types of testing done in the mobile platform [11:28] you want to talk about the different kinds of tests we have in our test pyramid? I guess the other part of this is we did have some hackathon projects that did some AI testing stuff. They're just not something we use in tooling a lot. Um but yeah, testing wise, we have actually I I think we pulled metrics for what our code coverage is, but basically we uh uh ratchet to enforce like um unit test coverage in various areas. We have definitely a lot of integration tests and we also use a lot of um E2 stuff or like blackbox tech blackbox testing for like access accessibility assertions and that kind of stuff. There's a lot that we actually do with it. We also do weird stuff like I've been working for a while on using the UI testing infrastructure to like exercise the app and then try to connect it to leaks to do memory leak analysis inside of our endto-end testing. Um, so yes, the answer to your question is we have all of those in a pretty good mix. And is this simulators or do you actually have a device lab where like these the things are deployed and and you're holding your hat so probably I completely well yeah I completely forgot that we also have uh we partner with a vendor to do that as well. Yes. So every kind of test I think it we've come a long way. uh when I joined I we didn't have coverage metrics uh but I want to say we're like maybe 2% unit test coverage um and nothing beyond that. Um, so this is something that's like really grown within the last, I don't know, two, three years. Um, where I think our our testing story is quite a bit better than it used to be, but I think we still have a long ways to go, honestly. Um, I think we have the pyramid established, but actually kind of filling in all the bricks on the pyramid has some ways to go. And like you've done way more testing than most engineering teams will do, and especially, you know, startups starting out will not even dream of doing. So what has been the the benefit? You know, you put in a bunch of work, right? Like cuz I I do hear engineers ## The benefits and drawbacks of testing [13:20] asking like why bother, right? Like it's it's going to be a lot of effort. They're hard to maintain. We know end to end tests especially on mobile are are slow, etc. What have you gotten out of this? So I if we go back a couple years like Eric was saying, we had almost 0% test coverage and we were having a lot of incidents in prod that were easily preventable. Um, and so there was a whole shift left approach of trying to find those things before our users were telling us about them at scale. Um, and so I think that our test info became a part of that. But it was actually really hard to stand up in the first place and that was because we were in a tech debt situation that had really really long build times with no test infra in place yet. So we didn't have a lot of capacity that we could say, hey, we're willing to increase our build times to add tests in to be more protective. So uh you have a kind of perfect storm of rapid growth in our engineering orgs. So lots more people running builds. our builds are already really really long and we want to introduce all this test coverage and we're constantly having incidents with the fact that we don't have test coverage in place and we were mostly relying on manual QA from an overseas uh vendor with like a 12-hour turnaround. So fixing things ouch really really difficult. Uh so I think trunkbased development so shifting where we're testing and what artifacts we're testing introducing automation ideally in places that we felt had value first. So if you're in a postmortem and you notice that a test could have resolved this in a way that uh we would never have this problem again. Those are the tests that we really tried to evangelize doing first. Tests like that issue or similar to that issue. um it's a lot easier to convince people those are tests that pull their weight and are worth running. Um but yeah, we did have ratchets in place to just kind of increase testing across the board and a lot of those kind of contracting firms we brought in to actually help us build tests and build a like culture of testing. And I will say though that the the downsides you brought up are very real. Uh we've had lots of issues with flaky tests. you know, people, like I said, we started with virtually no coverage. And so now all these engineers who maybe have never written a test in their life. Um are now trying to test their code that was not written to be very testable. Um so we we ended up with a lot of bad tests and a lot of flaky tests and a lot of pain around it. Um so it is very real. Yeah. I tech that exists anywhere regardless of whether it's in your app or your test code. One of the thing or one of the uh little tidbits that was really helpful to me is we had someone who uh joined Reddit after we had kind of go gone through this process and we were talking about some of our frustrations with the testing and he's like this is the first place I've worked at that like added a ratchet and like it worked like yes we have flaky tests. If you have tests you'll always have some flick in there. That will just be true. But like you also have mobile yeah you also have a lot of tests. And when Eric was referring to like some of these iffy tests, one of my favorite like side effects of trying to build tests is you're at least adding code paths to like inject your dependencies and like you can at least reach that code in test which means even if you can write a bad one today, you can probably write a good one tomorrow much simpler than starting from zero. There's a there side effects have absolutely been worth it. So you you're doing a lot of modern engineering practices. One of the things that is the most modern and you know ## How Eric, Brandon, and Lauren use AI in their workflows [17:00] maybe the future. So some some you know there's a lot of hype around it is AI coding tools. I'm interested how much do you use the current ones? What do you use? What is working or or how much did it help you or even if it helps you? And I'm asking this because especially with native mobile you know this these tools are not nearly as cutting edge as does as they are let's say on web or or or some of the the kind of JavaScript frameworks. I'm just interested in like like today, you know, like like just honestly like like what are you getting out of it and and what is working? What is uh not really working? YouTube, this is a little bit of a tricky one to answer. Um so in personal projects I've used, you know, Gemini is built in Android Studio. It's honestly pretty good. I don't you I don't use it to ask questions and have it write code for me. Honestly, I still don't believe that that's going to work very well. Um, but as like a very fancy, efficient autocomplete, it's fantastic. Yeah, I think Apple's built-in Xcode one is is perfectly fine. I use it a tremendous amount. Like I feel like chat GPT has officially replaced like my knowledge of git. I can never remember how to use git like invocations properly. So I just pop open a window and be like, "Hey, can you please re-educate me on this?" Um, I use a lot for that kind of like ad hoc things where I don't have an idea of where to start. Um, I've noticed that they are starting to do a lot more like when you ask it a question, it can generate a lot more complicated like it'll give you a integrated kind of web project or something like that. Um, I'm excited to look back into that. But yeah, on the mobile side, I've asked it a couple of like specific wrinkly questions that I actually have to solve on a day-to-day basis. I've never once gotten good code for those, but it it's a good rubber duck if you're trying to like break down a problem and and carve it up into chunks and ask it focus questions. You might get decent code. You're always always going to have to revisit it. But yeah, I think most of just the personal project bug standard ones. I I I do wonder if these tools we'll see a similar thing with uh what we're seeing with native mobile development. You know, native mobile development tooling is different than than web than backend. we've always had, you know, if you iOS, I mean, you're doing it right. Xcode has has different functionality. Android Studio or or Jet Brains ID has and the challenges are just different. So, I I feel there what what we might see, you know, certain teams using it on other domains might work better and, you know, maybe mobile either catch up or be slower or who knows, maybe it's a different fit. I I think we are using it in a couple of ways though. So, so I agree with Eric that rapid prototyping with the Android Studio Gemini features are pretty cool. That is a lot of fun to do on the side and for validating ideas before you actually make them official Reddit ideas. So, is is it fair for me to rephrase that you know for AI coding tools specifically for mobile it somewhat useful like you know some people use it here and there but sounds like the bigger use case is you actually using this technology LLMs and elsewhere to add to your to your workflows. you know, and eventually maybe even your dev tools, but sounds like that's kind of a little bit more more promising and exciting path. Did I sense this right? Yeah, I think that's right. I think our test insights are also somewhat powered by AI now, but we're not really using them as much as we could. Um, I would love to see them more like we had a lot of investment in test infrastructure, but there's a lot of duplication in it. And when a team goes and looks at how many tests they actually need and how many are P zeros, it's usually a very small subset of what is currently sitting there. Um, being able to actually figure out what tests are pulling their weight post Ratchet is a great place I think that we might see AI in use. Yeah. And let's talk about the the interesting topic that we wanted to talk about. you ## Why Reddit grew its mobile teams in 2021 [20:50] did a really big tech mobile modernization starting from maybe 2021. Can you tell me what was the state? Why did you decide to suddenly make make a big change which as I understood also led to hiring more more native uh engineers and how did it go? Uh it that's a big topic for us that we really like to talk about. Um let's see uh let me try and set the stage. So, uh, 2021, um, Reddit is, uh, growing as a company. They are trying to establish themselves more as an international company instead of a US domestic company. Um, and uh, they're seeing the world shift from a like web- based space to a mobile based space, especially overseas. uh that led to a whole bunch of new strategy which led to a whole bunch of hiring across many many teams especially in the mobile space. Um and so we start seeing a a team that was probably in the like 40 engineers 50 engineers across both platform size and within like two years were around 200. Uh and that is the space in which uh you start from a place of having really really poor build times like two and a half hours type of a thing with no test infrastructure but all of these big goals. Um and it's also around that time that platform teams really started to form officially. Brandon, you can correct me if I'm wrong here. It was within that year or so where we kind of transitioned from a company like many companies that kind of beg and borrow platform support from everybody to keep things up and going to saying we need proper platform teams that are going to actually help us scale this way. I think that time frame is about right. Oh no, actually that that's definitely right because I was like I this is my brief dip as an EM in this thing and I got to hire a couple people. Um, but we were kind of, at least from my perspective on iOS, we had always been about three people supporting however big the org was. So like the iOS platform team was three from the time that we had 10 or 15 engineers until until about the time frame Lori is describing. And yeah, it was uh it was a lot. It was a tremendous amount of work. I can remember specific incidents about this, but yeah, it was really a time where we could finally start developing at least the start of like specific areas of expertise instead of uh uh platform engineers just being experts in everything and then having to pretend that it wasn't stressful and context switchy. So, so one of the things that happened though is we're hiring all this great talent because we really want to invest in mobile and they're showing up and onboarding is rough. They have never seen build times anything like this. If you ask them what the tech stack is, we're like, "What team are you joining?" It depends on what team you're joining. And then we'll tell you what your tech stack is, maybe if there is one. Um, and uh, so we're getting a ton of feedback from our existing developers and the new people showing up that Debex is not bringing them any joy. feature teams have these big ambitious new goals and the feature team uh hiring like they they join together they form a team and they can't execute because our codebase is really slow and they can't figure out how to how to actually like deploy safely. All of the things are kind of coming up at once. Um, yep. Yeah, I was gonna say one of my favorite um, actually least favorite I guess uh, stories about this was when I joined. Um, so I guess I'll say I'm generally all for people on any team uh, getting to know you know the finer points of their tooling and their tech stack. Uh, but there's one fact I strongly believe that not a single feature engineer should ever know and that is at what point their CI provider decides that it's more likely that a process is stuck in an infinite loop than that you're actually still trying to build. Uh but when I joined we weren't so lucky. Um I regularly had to retry PRs after they get timed out on CI after you know two and a half hours. Um and you know like a lot of these were from decisions that made a lot of sense when we were small team small code base and we just didn't have the the staffing to readress them as we grew. Um so yeah we we kind of reached a point where something had to be done. So it wasn't just Devx though. This was also bleeding out into the users and their experience. So, uh this is around the time where we probably had our all-time worst stats on mobile um across both platforms. In terms of ratings or or feedback, all of the above. Uh I started in in 2021 and the week I started our our Android startup time was 13 seconds at P90. Uh I think Brendan our iOS was around 7 to 8 seconds P90. It was the worst of before Bolt. Um uh we I I was pulled out of my onboarding the first week for a feed incident that had no observability. We couldn't figure out what was going on even that we were not showing feeds. Um so we we were basically blind to the problems that we had. We were just constantly reacting to problems at that point. Uh, and so I think that that all of this together became a really good like pivot point for the company to say these are all becoming really big liabilities for us. We need to actually have a plan uh uh for how we're going to resolve our Devex issues so that we can actually build what we want to build. but also to dig ourselves out of the hole that we were in from a from a like existing tech perspective. It needed to be a like ## Reddit’s modern tech stack, Corestack [26:50] pretty comprehensive plan. Um so that kind of sets the stage for how we bring together a group of people to define our tech stack and where we go from there. And then what was your journey in the modernization? So what what were the the things that you put in place? We previously me talked about before the recording about a monorreo which is an interesting one. Early on executive leadership asked us for a prescription for what was going to be our kind of like big solve and ideally could we have a kind of umbrella effort that that kind of trademarked the entire approach and that's how we ended up with a branded name for our tech stack which is core stack. So this is for the new stack right? Yes. This is this is our answer to everything. Um and I'll very quickly TLDDR it. Uh at the bottom of it, it is how are we going to organize our code? We we agree that we are going to organize it in a monor repo for each platform modularized primarily by feature. Y uh above that we're going to use a modern programming language. we're going to commit to using the modern program programming language for the given platform. Uh we were already uh one thing that was helpful was that a lot of these particular choices were already well underway. They just needed to be blessed and pushed into the like primary way we wanted people to work which made it actually pretty easy to go down most for most of these choices. Um but uh we were in the middle of a GraphQL transition from a REST company. So we committed to being a GraphQL first client and then a GraphQL only client. Um above that we see a little bit of divergence across the two uh platforms but we settled on MVVM of some ## Why Reddit shifted from MVP architecture to MVVM [28:48] flavor. Do you want to explain why we chose MVVM Eric Brendan? Uh sure. Um I mean a lot of it was kind of forced by we haven't talked about this yet but our UI our UI framework choice of compose um so we were previously an MVP model view presenter uh architecture pattern which was even when I joined felt very out ofd and a little odd um but it was fine right like it's it's better to have consistency than to have a few things here and a few things there so we we just kept going with it um despite it not really being anyone's favorite um But the issue was kind of forced when we chose Jetack compose because it is a reactive framework. So we can no longer use imperative architecture patterns to drive a reactive framework. So we kind of had to um settle on something else which you know uh the the industry has more or less settled on MVVM or MVI or some flavor of that. Um so and so sorry just for those who who don't know so MVVM model V view model right and MVP model view presenter. Yes. Yes correct. And what is the difference between the the the two, right? They sound pretty similar. You have the model, you have the view, and then you have a third thing, but they're different. Yeah. So, the really what it boils down to is imperative or reactive. Um there there are several other differences, but that was the main driver for us. Um using a a reactive UI framework, we were kind of forced to stop using imperative uh logic to drive it. Brandon, how about the iOS side? we have ## The architecture on the iOS side [30:22] a so iOS uh that this was a little bit more complicated. So we at that point we were evaluating Swift UI and kind of like where we were at. I think our back deployment version on that was still iOS 14. So Swift UI was I think technically possible. I can never remember when it was actually 13 at the time. Oh. But uh Swift UI introduced some problems from that perspective. And it also was just like it was still pretty new cuz we're talking about 21. It was three years old at that point and it it's still a little still pretty fresh. Um, so we opted for a layer on top of basically collection view we called slice kit. I'm pretty sure we've had some blog posts about this. I can't remember exactly. We talked about that uh uh for a while. It had a lot of benefits like there there's some cool things that if you kind of if you imagine uh wrapping a screen in a collection view behind the scenes, there's some really nice things like dynamic type was trivial to just like get that stuff working. Um, I think Conrad's pretty happy with how accessibility can work uh in some of those areas. Um, but ultimately we started running into issues because writing a declarative wrapper around UI kit is tremendously complicated. And it turns out even Apple uh is, you know, not perfect at doing it. Swift UI, I would argue, is what we wanted at that point, at least from the the code we wanted to write wasn't just quite ready yet. So, we're revisiting some of that, but I think we'll talk about that in a little bit. Yeah, it was a it was a pretty legit journey though. Yeah. So, we choose MVVM for both of our platforms with a slightly different answer for what our UI framework is going to be. We choose compose on the Android side. We choose Slicekit on the iOS side. One is an industry standard and one is an internal uh choice let's say. Yeah. Uh, and then ## The new design system [32:08] on top of that, we build our new design system. Um, which is ideally going to solve for a lot of the feature delivery pain around people con like constantly reinventing the wheel. Um, I think that at one point there were like 15 spinner controls in each client and there was a guy on Reddit that just like continuously found new ones and posted about them. Uh, I love that guy. Um but yes, so it wasn't particularly uh revolutionary except in small parts. A lot of it had already been started but and committed to but some of these projects had been going for like 3 to 6 years already and not finishing which had left us in a place of uh the worst of both worlds. When I got here, if you had an incident, um, our client was half planted in the old REST world and half planted in the new GraphQL world, and you had to know which team and which project and which endpoint you were talking about before you could solve for even where to look at the observability for that element. Um, and so, uh, you could vastly reduce, um, and simplify for everyone. Then a lot of things would get easy. And our promise wait our promise uh for core stack was if you actually followed our our simple prescription platform teams would be here to build easy mode tools to provide golden paths and to generally make it easy for feature teams to focus on building the best features instead of struggling with the tools and the architecture. So starting from the bottom uh in terms of the the API you mentioned there was there was a transition from REST to GraphQL why did it happen and what did that mean for for the mobile clients especially that you know as we know for native mobile you can't just like ship a new version and forget about the the old versions you you either need to have backward support or you need to have some sort of kill switch which forces you to update you know WhatsApp is I think for people using it they they will say saying oh this this version of the app cannot be used. You need to download a new one. But uh until until things like that are built, you can't really do that. So like what what what was the impact and you know how long did that take and how do you how do you respond? Oh my gosh. I I it would be funny if I actually did the math on how long it took. Like I think right now it's true that we're not in an active GraphQL migration, but for most years of me being at Reddit, there's been some kind of of that migration. like the uh we were tackling it early on. Um one of the shout outs to uh Aiden who was in charge of our like original iOS GraphQL implementation is like that system is actually still in the codebase today. So we got 5 years out of that thing at least. Um but it provided a lot of just benefits in terms of contracts and like knowing what the mobile app is actually doing and which like which client is even doing a request. One of the happy accidents I actually like is that because we have to name our GraphQL requests in a specific way. Um there are a couple of incidents that got solved in 15 minutes because we were like, "Oh wait, well that call is only deployed on Android." So this is an Android specific issue. So the like max thing is here. So like iOS on call people, you can take off like we don't have to worry about this. But it was a it was a lot. Um but a lot of cool benefits. We and we got to build like sort of our own Apollo internally. Uh that uh finally proved to me that we definitely should just use Apollo instead of trying to roll our own. It turns out it's a complicated thing to build. And then then just just to hammer home the difference between REST and GraphQL. REST is it's it's it's APIs and then JSON, right? And then you need to do all your validation. With with GraphQL, you you get more more type safety. Do you? Yeah. Like Oh, definitely. So the nice thing is you can go and read the GraphQL spec. It is actually like this somewhere on the internet. I actually had to read that a couple weeks ago. Yeah, there's an explicit contract. You're going to get types. Um there's going to be a schema that enforces those types. Um and if you think about a older website like uh Reddit, you can still probably find some of our old school like API documentation. A lot of those endpoints are like older than some of the people who listen to this podcast. like Reddit is 20 some years old and there Eric kind of alluded to this before. There were a bunch of reasonable decisions that were made when we like shipped those like after 10 years some of those contracts have to be reworked. So GraphQL was a great opportunity to like apply a layer on top of our like old services that established a clear contract and like allowed people to actually understand what clients want and what you know the backend needs to surface. I would never go back. Go ahead, Eric. I will say though like um I I think we took a little bit of a naive approach. We definitely have some learnings out of the migration. Um really what we did at least to start out was we took our REST API and did a 1:1 migration to GraphQL which is not how GraphQL is supposed to work, right? Like we take these very flat models um and we pull them over to GraphQL and then we we don't get that type safety really. We get some nullability safety but that's about it. Um, so we still ended up with just massive amounts of client side logic that didn't belong there. And we still have a lot of these um these issues today around too much logic in the app or things having to be redesigned in kind of weird ways or even just overfetching. Some a lot of times we we fetch massive amounts of data that we never even touch just because it was in the REST API so we probably need it. Um, so we made some mistakes. we have some learnings and you know it's the kind of thing we're still improving every day. And then for for this a API migration, you know, the API, the back end API changing from REST to GraphQL, how did it go in terms of like was it the mobile teams telling the backend teams, hey, this is what we need. This is what the app uses. And obviously there's a web team as well. Or was it more the other way around where the backend team is like, okay, we're exposing this things. Here's what you're getting. because it's always a question kind of who's who's ## How the backend drove the GraphQL migration and why it was worth the pain [38:20] leading in terms of you know you have the the the teams who are closer to the users but there's also the backend teams which have a lot of considerations for performance functionality maintainability that kind of stuff that's one of my favorite things about GraphQL actually is as long as you have some amount of collaboration you can usually come up with something that both people are happy about um or both sides are happy about I guess um you can I mean you you have a query and you can query only for the very specific fields you need and you can kind of form them into the data models the way you like them. Um, so I it the direct answer to your question is it depends on what team you're on. Sometimes the backend developers are leading it, sometimes the client side are, sometimes there's a lot of collaboration, but usually both sides can be happy just due to the flexibility of GraphQL. I think I think initially the migration like the web team probably started it. Um, but I I feel like they pretty immediately pulled in like platform people at least. I'm sure Aiden was in there like immediately as soon as Alex had the idea. Like, so I think it started from there, but it pretty quickly like we were we were uh in the room being able to ask questions and provide feedback. For sure. Go ahead, Lori. Oh, I was actually going to take a different answer on this one. I my entire experience at Reddit has been around a back-end GraphQL team having been established and really helping drive the migration away from REST and our old infrastructure to a GraphQL and then a federated GraphQL model that they have been evangelizing and like they've been excellent partners to the client teams but uh they have definitely driven a lot of that adoption and helped us like build our actual GraphQL well experience uh as as client engineers. And some of that is because when we go out into the the wider talent pool, most people come in the door at Reddit not knowing GraphQL up front. That has changed over time, but it was a lot more unusual in the time that we were growing. Most people came in with a REST background. They had maybe even written a Reddit app themselves before they joined the company. Um, and so, so I think that like our journey here involves us learning how to write idiomatic GraphQL over time, learning what happens when you don't. Um, and uh, one of the other things I think that uh, had an impact on this particular project is our initial GraphQL implementation was slower than our old REST infrastructure. Oh. uh it had a latency tax at the like we got the flexibility and all the good parts we wanted but we had to take it with a a latency tax up front for a while. Uh and so when we transitioned the the main Reddit clients to GraphQL um they were noticeably slower on the same features we had been running on our REST infrastructure before. and we had to decide if that was a trade-off we wanted to lean into and commit to or uh and work through or if we were going to rethink our approach. Um ultimately we took longer to actually move over to our GraphQL infrastructure because we worked with the GraphQL backend team to reduce that latency and basically make it uh much more performant. And so by the time our clients moved over, it was in a much better place. Uh but there was a point in time where it was frustrating for feature teams to be migrating in the direction of uh with a latency hit. Um especially when we were really sensitive to to our app being slower for users at the time. Yeah, this is the the that migration. Uh there's a comic about migrations at Google. Uh Manukornet the who drew it. Uh it it's it's like there's two paths to the systems. there's a path that doesn't that doesn't work and the one that's being deprecated and it's the usual store and he he's got a request to like whenever he went to like translate it because it's usually what happens with migrations, right? Nope. But but I asked both Brandon and Eric last week if they uh feel like GraphQL was the right solution for our size at the time. And what did you say? I don't know if it was the right size of the time, but I I feel like this is such a great example of like a one of my favorite things about Reddit and b why you should just hire tech experts and like let them cook is like yes, it was slower at the initial time, but now like because we have that, we can do a bunch of codegen. We can try to solve like a bunch of different tech problems and there's a bunch of optimizations that we can do because we have one thing. Uh we only have one kind of like networking layer to maintain. Well, that's probably not true. We probably have too many still, but at least we can optimize the GraphQL one. But because we, you know, took that or made that bet early and let people figure out how to make it more performant, I'm pretty happy with it. I'd do it again. Yeah. Nice. And then how did the architecture choices play ## Why the iOS team is replacing SliceKit with SwiftUI [43:17] out? So, you mentioned that on on iOS, you decided to go with an in-house uh UI framework at first. Where where how did that work out and where are you right now? um as as ever we are in a period of transition. So I I would still say that well actually no I would I would fight somebody about it. It was absolutely worth it for one very big reason like a big chunk of the actually let's talk about a few things. uh one the feed code uh uh in our iOS app is like a very very central component and it spreads its sort of uh design choices I think pretty so the feed code the feed is when you open it you see your feed or like feed of different Reddits right or different subreddits when you think about either uh just popping open the app and seeing what we would call your home feed like the automatically like the automatically populated list of posts that we think you're you're really going to like that infrastructure is actually shared between well a lot of it is shared between actually the subreddit feed if you look at a specific community and historically there was a pretty sort of nasty uh cluster of of like type hierarchies in profiles as well. Basically anything where you would show a vertical scrolling list of posts is probably the same infrastructure. uh before this that was based in texture and if you're listening to this you probably don't remember that framework or uh uh what it was but it was uh something Facebook started and then Pinterest uh shepherded for a long time that allowed you to do asynchronous and multi-threaded UI. So that created a tremendous number of complexity if you were like remembering Lor's examples of like somebody who comes in who doesn't know a particular kind of like pattern at Reddit. Nobody who we onboarded knew texture. So, uh, we wanted to get out of that. The first commitment to SliceKit was a massive win in that we just are back in Apple's nice main thread UI land. Um, however, we ran into a lot of issues where it's really hard to reuse the components. Uh, there's some boilerplate that we just can't quite crack. So our what we've been working on for a while and how we've been kind of going about this is I really strongly believe that if you're building infrastructure the new thing should at least be able to be hosted in the old thing. So what we're doing is we're saying hey if you're using slice kit we're going to figure out ways that you can put individual swift UI based cells in there and maybe you can experime and you can easily AB test on those individual cells so you can figure out is it performant like is it accessible like does it meet all of your needs um and you can kind of roll out at your own pace and then we're essentially adapting that principle so that um if you imagine like our vertical breakdown of course stack we are replacing eventually slice kit with swift Swift UI as we prove out areas of this thing and uh make sure that it for example SwiftUI list is a little bit perf suspect right now because it's really complicated but uh essentially we're replacing it on a manyear time scale but we're doing that by making sure that the teams who are currently leveraging it have support for everything that they shipped in prod today. Smart. I think one of the interesting things about this is that when we gave our kind of core stack prescription uh a lot of us intended for it to be uh a starting place that we could make uh on once you separate the concerns out a bit and you have an established pattern, we can build a road map away from that pattern if we decide that there's something better out there. But a lot of people kind of took it as oh when we have introduced all the new stuff we'll be good forever and we will never have to change our minds. But a lot of the people who were trying to uh evangelize and build out this idea were trying to plan for change for the long term because we knew that any anything that we chose in the past we chose with a reason. We had reached a point where those particular choices were not holding up over time and we needed to change them. But we understood that we probably would be having the same conversations ideally 3 to 5 years later about whatever we were doing now. So uh Slicekit and Compose were bets on both sides. Um they had different uh like risk profiles associated with them. Um but we wanted to reserve the right to not have a like oneway door there where we couldn't change our minds later. And even the slicekit choice had to like intentionally revisit this choice on a regular basis um for the proper time to consider Swift UI. Uh and then how did it work on an Android? Because you you took a bet on Compose and just like Swift UI, Compose back in 2021 was pretty new. Like right now everyone it's it's pretty accepted. Uh it's it's it's a great way to build things. In fact, I'm not even sure there's really a competition of how to ## Why the Android team took a bet on Compose [48:08] structure like a moderately. But back then, this was not the case. It was like here's a new new framework. You know, we think it'll be good, but you don't know. So, so I want Eric to answer this question, but one thing I want to to preface it with is when things are quite bad, you have a lot more opportunity to just take bigger bets and get away with them if they pay off. So, Eric, why did we take the bet on Compose as early as we did given how big? Yeah, I I don't know how proud of this we should be. Probably not super proud. Um, but when we started working uh on our first compose feature, it was still in alpha. We shipped while it was still in beta. Wow. Um, yeah. Uh, the reason we were able to do this, like Lorie said, was that we knew we needed change and this looked like the next big thing. At the time I was still on a feature team and I was working on a brand new feature like completely green field. Perfect fit for trying something brand new. Um you know so I took a bet. I I built the whole thing in compose. Uh I used MVVM instead of MVP. Um kind of introduced a whole lot of new patterns that seemed like the way to go. And the reason that I was comfortable doing this was because I was also willing to go back and rewrite it if the bet didn't pay out. Um, you know, I was I was the sole developer on this feature. I completely owned it. I if if it didn't work out, if it was a terrible decision, I was willing to own that and, you know, make it right. Um, and then ju just to confirm because yeah, I think what people might miss is like Reddit, you know, like it's it's a massive app like large amount of users are using it. This is not just like your, you know, your startup with 100 users. And this feature that you did, was it like, so it was like a nice to have feature or not something critical or it was critical, but you were just like, okay, I'm going to make it work. It was I wouldn't call it critical. Um so it's it's now gone but it was called Reddit talk. Um I don't know how many people are familiar with that but it was the like live chat room like voice chat uh feature club right? Yes. Exactly. Yes. Everyone was copying that for a while. Um so yeah though it was just kind of the perfect opportunity. So it it wasn't uh it it was it was launched to a very small audience. I think it was only like two subreddits had the feature at first and then we were, you know, slowly scaling it as we proved it out. Um, so it it wasn't a super risky bet, but there was some risk there. Yeah. Well, reputation, right? Still, it still only took like less than a month to adapt talk as it first came about into the golden example of what we ultimately decided was our very fairly straightforward set of tech stack choices. And so, uh, it was easy to adapt. So, it basically proved out that green field work as a starting point for our our, uh, new prescription was going to work just fine. um adapting some of our more legacy code, our larger critical path features came with like they needed more than even our core stack prescription was going to give. So they extended it in all sorts of interesting ways. ## How teams experiment with server-driven UI—when it worked, and when it did not [51:25] Now, one thing that you mentioned that you're doing, which is very interesting to hear, server driven UI, and this is interesting because we we talked about this with Lori, but a lot of companies above a certain size, especially when there's native mobile, they they will come up with something that might feel familiar to, let's say, either React Native or or or or just some something else with the goal of, you know, being able to drive some of the the UI with backend changes. What was your approach here? Why did you do it? How did you build it? And how do you feel about it? So while it's not an official part of our core stack, we definitely part of the core stack idea is that people can extend or experiment with anything they think is going to work well on their surfaces because they're not all the same. We have massive feed surfaces and we have chat surfaces and they and we have profiles and they all really need different levels of like complexity and uh capabilities. So we definitely had some teams try out server-driven UI. Uh, I think Brandon, you might have a good example where it worked out well. Yeah, I think for a lot of those tremendously complicated, but one of the things that I think has been shipped for a real long time is our reporter flow has a like our content reporting. if you find something that you don't think should be on Reddit and you want to report it. Um, there's I would say that has some server driven UI in it, but since you're like driving reporting reasons where you're really just trying to ship like a list of strings, like I think they chose a a reasonable level of complexity where it makes sense for that team so it works really well. But other cases, not so much. Eric, did you have an example of the nasty? Yeah. So the the first place we tried out server driven UI was our biggest most important service which is maybe not where you want to start with experimentation generally. Um on the surface though it made so it made a lot of sense. Like we had tons of logic in each client that went into determining how to display posts within a feed. Um you know overly smart clients aren't a good thing. Uh and the logic didn't even match between clients. Like there were times where you hold an Android and an iPhone next to each other and like the feeds would look different. not intentionally, just because we got the the logic wrong on one or the other and nobody knew which one was right. Um, but despite all the the good reasons that went into this, um, it was a decision that we've kind of ended up regretting to an extent uh, and that we're currently uh, working to walk back from, um, it turns out that even if the feed itself doesn't need, you know, the full post models to be able to display correctly, um, everything it links to still does. So, you know, if we make the feed server-driven UI and it no longer has any knowledge of the the backing models, um, we just end up having to do double fetching. Uh, we we have to fetch the server-driven UI definitions, then we have to fetch the actual models. And this just introduces double the opportunity for errors. Um, and it has led to lots of bugs that users see in in production. Um, I don't know if you've ever used the app and you've tried to tap on a post and nothing happens. Uh that's because one or the other of the fetches failed. Um so we're working on it. Yeah, it's interesting because ser ## Why server-driven UI isn’t taking off, and why Lauren still thinks it could work [54:30] server driven UI is is something that everyone eventually goes to because of the native mobile apps because the nature of mobile is if if you only have client side logic. You need to ship that code, maybe put it put it behind the feature flag, but it just takes, you know, if you want to add new business logic, it will take you at least a week or two weeks or however long it takes. So eventually everyone comes to a realization what if you know a what if we just had a web page wrapped into an app and that doesn't work but the next best thing is back in driven UI what what if we just have the business logic as as a JSON or or something that we download from the the website and it should work but I've seen so many times where like again like most companies don't talk about this publicly but it it becomes sour like it just doesn't really work out because I think eventually what I've seen realize is like you know we need to be backwards compatible. We need versioning. Uh we need to support older clients. We're now running into, oh, we want this feature that we didn't think about back then and now we could add it, but our older version doesn't have it. What what is your what is your take on why is this not taking off in mobile land in in general, right? Like it's it seems there's just you come around and learn a lot of these things. I I actually think that we shouldn't give up on the idea. All right. I I think it's like any idea that we have had and failed at or whiffed. Um you can say people you can say agile and it means different things to everybody. You can you can say modularization and you can name just as many times that has failed to deliver on its promises as it has worked. And so so I don't actually think that there's a real problem with the idea. I think there's a problem with an implementation that is that like works for the shape of the problem and we don't spend enough time making sure it is like like in this case I don't think we took enough time to make sure it was going to solve the Reddit shaped problem as opposed to serverdriven UI makes sense but don't actually like apply it to the actual reality of what our codebase and infrastructure looks like. Uh there were parts of our infrastructure that were not ready for it. we did not have an endto-end uh like commitment from everyone to work on the exact same plan at the exact same time in order to support it. Uh but I've also seen that happen with trunkbased development um and other things where it was like the fifth time you try you get it right and and then everyone is really happy with it. Um so I don't think it's necessarily the prescription that is the entire problem. I think it's a like underdeveloped implementation and like plan in place to make sure it can work in the cases that you want it to work. I want to I want to double click on on what Lori is saying with a specific example. So just in case any any product manager is listening who wants server-driven UI like the reason that managing that complexity is so important is that we know server-driven UI is possible and it can be done really well because web browsers exist. But that's the upper limit of this complexity and I don't have enough time to build Chrome really really well. So like if you want serverdriven UI and you want the full feature that you're specifying in a web browser, well all you have to do is spend several billion dollars. Uh if we if we control complexity well however we can we can definitely solve this problem. Just please no web browsers. But you also see that like web view implementations in mobile are often not great experiences for mobile developers. So like that there like there are ways in which we've tried to like bridge that at different times and sometimes it really works and sometimes it does not work. Um but I don't think we should stop trying on those like on these ideas because uh I think we all agree that our clients being simpler, having less logic in them and being more nimble is like those are all capabilities we want. We just haven't figured out how to deliver them well. Um, and in the meantime, we have weekly, sometimes more than once a week releases in order to keep the ability to change production. Um, no, nowhere near what our web our web platform can do in their continuous delivery pattern, but it's pretty much the fastest you can deploy on mobile. Oh, yeah. There's also like I I feel like for jetack or for excuse me for compose and for swift UI because the atoms are different we might be in a case where it's it's easier to try some of the things using those modern like uh lang excuse me the modern framework works that are closer to DSLs because if you think about a server driven UI thing it's probably really hard to do that in UI kit UI kit is not declarative since UI is declarative if you're shipping a declared spec it probably will change a lot and so you you've done a bunch of work over the last like three or four years starting from this this modernization ## The ways that Reddit’s modernization has paid off, both in DevX and UX [59:25] you know you moved to to compose on on Android swift you well moving to swift UI on on iOS added in the the mono repo um we we didn't talk about it but but you you added code ownership and and you just did a bunch bunch of things that should that made things better in the end was and of course you onboarded so many new mobile engineers as well was this effort worth it like how can you tell how this this big modernization the core stack effort played off. I think we have really good answers to this on the DevX side. I think we have really great answers for this on the user side. Uh I know that our sentiment has improved from our developer sentiment and we are serving so many more developers than we were before. Um, and pretty much every stability and performance metric has benefited from the changes that we've made. Uh, but who wants to go first, Android or iOS? Sure, I'll go. Um, yeah, I mean, like Lori said, our our developer surveys have shown that people are just far happier than they used to be. Uh, those developer surveys were rough for a while. Um, and now they're overall like they're they're pretty positive. Um, but a lot of things have gotten easier. Onboarding is easier. We don't have to coach people into the, you know, myriad of of patterns that we have. Um, when I used to interview people, you know, they were all excited like, yeah, Reddit sounds so cool. I really want to work here. What's your tech stack look like? And I would tell them and I could just see their face like, uh, that sounds terrible. Um, and that's not the case anymore. Like people are excited. Uh, people like want to work in this kind of codebase. So, we we've seen a lot of really good stuff come out of it. um even the the types of engineers that we can hire. We can hire more junior engineers on the feature teams because you don't need to know all this, you know, legacy knowledge that there's no reason to know anymore uh for the most part. Um but and yeah, there's there's obviously all the the runtime userfacing benefits as well, but I don't know. I'm a developer. I care about developer experience. Brandon cares about texture crashes. I think uh iOS is a little lagging behind in that, but I think well emphatically it was worth it. Like there's if we wouldn't have done that, I can't imagine how the heck anybody would be productive uh uh with the the expectations that we would have had, but emphatically worth it. And I think even in cases where I feel like the tech needs to get better, it allows us to have a sort of specific path and a specific focus for people working on this. Like I love that we now have like iOS people who are the de facto GraphQL experts. That was not always the case. And now as they like iterate on that tooling and on those technologies, like everybody gets benefits. So we can progress in that area independently while I'm, you know, trying to figure out how we're going to get Swift UI into this thing. Um, and structured concurrency also introduces a lot of wrinkles on this. We have paths to like activate these projects and um, I would much rather be in this universe than the alternative. So, so I can give you some concrete examples of this. Uh, we used to see Slack messages in our like shared guild channels and you would have to figure out what somebody was doing at a really detailed level because it could be anything before you could help them. And so it was actually really hard to mentor people, especially in a remote first company. Um, you had to get a whole bunch of context before you could answer a question about why their build was broken or what they were trying to achieve. We have a much higher level of shared understanding when the question gets asked at this point because most people are following goldenish paths most of the time. So there's a lot less frustration burn a lot less burnout on the people who are both the subject matter experts and um the people who are trying to become them. Uh we did a whole bunch of uh analysis of our Devex improvements to try and prove out um not just by surveys but uh with some sort of quantitative analysis if they were worth doing. I really wanted to be able to kind of take the product approach of could we defend this sort of effort like we would defend a a product uh initiative. And we definitely saw that breaking up the monoliths and modularizing and giving people stronger ownership of their code areas improved a ton of productivity uh signals. Um people were building new features like Reddit recap with half the amount of people they did the previous year. Uh they were able to do they were able to finish their features sooner. So they would add these like extra bells and whistles to their actual like deliveries which had unintended consequences like they would introduce more uh uh animations and we'd be like where did those come from? Oh no, our performance. Um uh but but no all of those had like real business benefits from a developer efficiency and productivity uh perspective while increasing developer joy at the same time which was the kind of developer productivity we were after as platform team. The feature uh from a user side uh pretty much every one of our metrics that was bugging us at the beginning of this effort is in a much better and stable place and it very rarely sees regressions. our crashy rates benefited from going to uh Swift and Cotlin. Just from the like nullability perspective alone, we see very few NPE problems in production hot fixes associated with obvious avoidable problems like that. Um I think we we were up like 1.5 1.5 percentage points on crash rate, which is a ridiculously great improvement on Android. Um, iOS was always a little bit more stable because they don't have quite as many devices in the wild. Uh, but they also saw I think almost a 1% improvement over that sort of time period. I I never remember the percentages. I only I only remember when the problem completely goes away, right? And so so we are at a point of diminishing returns on investing in that space and we can focus on uh we then focused on startup times. those are down under like 3 to 4 seconds even in the P98 plus sort of range and they've stayed there for years and people have noticed um which allows us to basically move on to another problem we want to solve like scroll performance or video performance and working with the video team and stuff like that. So by kind of making these improvements and then putting uh policies into place to keep them relatively stable in those places without big regressions allows everyone to go redirect their time to what is the like most pressing thing that is uh in front of us either as a company or bothering our users the most and try and focus on improving that space. Um, and we can do that in part because everyone is kind of a little bit more similarly shaped, which allows us to make a huge amount of assumptions. We can assume that we can build a tool that will work for most of them most of the time, which we now do. We build a lot more tools and scripts and things to make things easy. Um, and we can build observability such that when we actually go looking at what the problem space looks like, we can actually build a map of what it looks like as opposed to it looks like a very strange world map with really strange boundary lines where you can't actually see the whole problem because everyone ## The overall modernization philosophy; fixing pain points [01:07:15] is doing something different. I want to double click on one of the things Lori said in case there are any prospective like uh uh modernization uh enthusiasts in the audience. Like we're not doing this because like we think we have a better solution than the feature teams. Like I feel like there's a bunch of boxes at Reddit where I don't I don't want you to have to think about how to interact with our GraphQL service. I just want you to like think about your feature and I want you to be able to be creative in the ways that she was describing. I love the idea that like you're adding animations and you're like you're actually bringing your creativity as an engineer to like the features you're building. The reason we're doing this is not like uh uh uh to take uh flexibility away from people. It's to eliminate a bunch of boring things so that you can do the actual important work. Uh if you're trying to do modernization, do it in service to your stakeholders. Do it in service to the teams that you support. Not because you think Swift is a cool language, not because you think Swift UI is a cool thing, because it has a benefit to the teams you support. That's a huge reason for us behind this. I I agree with that. I think that's one of the reasons why we focused on some of the places that people do not gravitate to like dependency injection. Like, can we make that less painful for people? Oh, yeah. Because nobody loves dependency injection with the possible exception of Drew until he was in charge of it. Um, and now can never not be anvil guy. Um, but we picked the places with pain points for our feature teams, not the ones that we were trying to like constrain them. To Brendan's point, um, and then you're in the enviable position of being a mobile platform team that is actually iOS and Android, so two pure native mobile platforms is quite rare in the industry. and al working with a large native mobile engineering team which obviously you you need a larger team to for a platform team to make sense. My question is what does it ## What the mobile platforms team looks for in a new engineering hire [01:09:10] take for an iOS or Android engineer to work at a platform team like this in in terms of when you're hiring? What are the traits that you're looking for? Uh is is it is it a lot of internal transfers? I'm asking this because for a lot of mobile engineers this is a little bit of a dream to get there at some point if they're lucky enough to have either their organization or you know to look for the select few few companies that do have this this thing. What are ways that that mobile engineers who are doing either native or crossplatform can work towards skill sets or experience to increase their chances to later join a team like this? I'll take this first. Um so the people who have joined the teams in the last couple of years are a great mix of internal transfers and going out into the the hiring pool and hiring externally. Uh we often find people grow in their feature orgs to be a top performer and they know everything about their scope in that space but they have frustrations that they would really like to get resolved by platform solutions that would also impact other teams. They are great recruits and we have gotten a some really great people come into the platform teams who then just immediately turned around and helped solve real problems at Reddit with us and helped us understand them and scale them to everybody else. So I can think of a couple people who have joined our teams that absolutely meet that sort of like pattern. We've also gotten some really great talent out from other companies who have brought in fresh ideas from other companies at scale as well as from startups who are scrappy and Reddit is kind of messy and we often have zero to one problems that it does not take at scale experience to solve. It takes somebody looking at the problem and going I know exactly how to make this 100% better and just going and doing it. Um, so taking ownership over something and being practical and like having first principles is like extremely important. Uh, and then the last thing that I'm always looking for is uh, so it's a little different between the platform teams here, but I have a really robust rotation of DevX. Everyone on my team doesn't necessarily want to be Devx all the time, but they want to give back on the DevX side, at least part of their job, like at least 25% for everybody. Some people want to completely specialize on on the DevX side, but a whole bunch of people actually really enjoy helping other engineers work through their problems and and and such. So, I'm looking for like that interest in giving back to other internal engineers as well as thinking of end users at the same time. Um, but if they do join, they get to work with awesome people like Brandon and Eric who probably have a different perspective on what it what they're looking for on their platform teams. I my joke answer is that I don't know why someone would choose to do this. I just sort of kind of like stumbled into it. Um, it's very stressful and it's very hard. Um, having said that, uh, I would not choose any other position. I I want to be you will drag me from this team kicking and screaming. If I think Lori provided a good perspective. I want to I want to talk about like what I think is the most important thing for an IC who's who was trying to have a job on a platform team. You need to sit in the consequences of your decisions. Like you need to, in my opinion, you should try to work at a tech company for a year or two and actually see what happens after you ship a system and then the assumptions change and you have to figure out how to keep this thing going. There are systems that we have that have no joke been in our codebase for 5 years and the platform team has been like evaluating whether to replace them or how to keep them going. That's incredibly hard and I think that's uh we have very talented software engineers at um across like our iOS or right but I think the big difference between folks who are uh working in features and working this is it's more reasonable for a feature team to like take their code and after the product doesn't work they just kind of delete it and go on. Eric mentioned like the hawk is not in our codebase anymore. Most things that I've had to write at Reddit are still in the codebase. I wish they weren't. But you have to understand and I think the what you get out of that is you get a bunch of software design intuition. Um because you have to like re-evaluate your assumptions for an incredibly long time. Um if you can do that, you're probably ready for platform stuff. Um but take your time. Uh I don't want to burn you out before you're ready because this is hard. I I want to interject really quickly there. Uh, one of the things that Brandon mentioned that I think is really important and it definitely comes up in all of our interviews for platform roles. Uh, what we're not looking for is somebody that just wants to go introduce platform work that they are not going to use themselves or dog food and assume they can solve something without understanding the ground. And so that is one of the reasons why we like internal transfers or even feature team to platform team transfers or growth simply because if you take product thinking in here, we're just looking at a different set of users. Uh and if you do not take the feedback cycles from your developers, you will absolutely miss on what you try and deliver to them as a platform team. And that is a problem that we're always looking for like are we open to that feedback? Are we actually solving the problem that is the real problem at Reddit? Eric, what about Yeah, huge plus one to that. I don't think platform work is a great first job. Um although we have someone on our team who's doing great at as at it as their first job. Um but I don't think that there's really a blueprint for becoming a platform engineer. Like some people are jack of all trades because you need that to some extent. you have to know how the build system works, how to make a screen more performant, um just you know how how the compiler works. There's there's just so much knowledge that you need. And then others are just specialists. We have you know build experts. We have performance experts. We have people who um just go really deep in a certain area. So I don't know if there's a certain thing to do to become a platform engineer other than having experience across a lot of different things and just having an interest in it. I guess I do think that remember that you're talking to a couple of people at I would entertain the argument that when I joined Reddit it was probably not an air quotes startup but like it was a startup kind of environment but like both Eric and I like started in uh at ## Why startups may be the best place to get experience [01:16:00] Reddit when the teams were small enough that it was easier to get to that impact. So, one of the like things, you know, if you're trying to get onto a platform team, it's probably easier to go to a startup and get a bunch of that experience than it is to go to like a scaled uh uh platform because like our our bar of what we expect our engineers to do has gotten way higher. Like my my job now as a staff compared to my job like I think I was a staff four or so years ago, completely different completely different expectations. So, I think like being in a smaller org where you can get exposed to a lot of those uh different things that Eric is talking about is probably an easier path, but you should still apply. You might not uh you might not get accepted this round, but uh I'd still love to talk to you about how we do this. The last thing there is that like I want to push back on platforms being elite teams a little bit. We are not we're no smarter than any goals. I mostly from the perspective of I'm I'm always worried we're going to ## Why platform teams need to feel safe to fail [01:17:00] introduce a something that's going to reduce the the psychological safety of this group. They need to be able to break things in order to to make big improvements. And so having the ability to say you don't know an answer to something and not having any sort of problems with that and being able to just like really grow through blameless postmortem culture is extremely important to platform teams. Otherwise you always play it safe and you usually just continue to introduce friction to teams instead of actually solving their real problems to help them like reach their maximum potential as feature teams. It's too easy to be like, "We'll just add another lint rule so they avoid that foot gun and suddenly you have a gazillion of them and everything is slowing down because of all of their checks." Um, so, uh, I think there's also just the the brilliant a-hole sort of platform archetype that we specifically are not interested in. We like the brilliant uh, humble really wants to work with other people and help them solve their their problems, too. It is it is definitionally like a position of service. Like my job is to help engineers be more efficient. It's not to uh ivory tower designs and then pretend that I'm smarter than anyone else. I'm really not. No, this is nice to hear and I think it's just a good reminder as as companies grow like there's more opportunities. So like if you're lucky enough to work at a company which is hard to predict but if you know there are trajectories and like when companies live up to expectations they they will grow. It's just a lot of paths open up. May that be going to a platform team, may that be changing stacks, may that be going to a leadership position or you know transferring teams working on a lot of them. It's just a lot easier like and on the other hand you know companies that either shrink or or say stay kind of flat everything will be a lot more challenging and difficult there. I I would say that like befriend your platform team is the best way to become a platform engineer someday. If you have a platform team, befriend them during hackathon weeks or do some mentorship and side projects. Almost everyone has done something like that before they end up joining our team. Uh but also we really like having really good partners on the on the feature teams themselves because they are very honest with us about what their real problems are and they are the best source of that. So there are also people who just never want to join a platform team because they're really user focused and that's the like the end user is the user they really really want to focus on with with their career and they are also some of our very best partners because they tell us what they're trying to achieve and we try and help them figure out how to do it. Um, they get they get early access to tech that's not ready yet. Like uh uh if we we have to find people who want to use Swift UI so we can deploy it at scale so we can verify all these issues. Like if you're getting buddy buddy with us, you're 100% going to be the first person in my head of like, oh well, search is a great partner for us on the iOS platform of like, yeah, we'll get feedback from them and see what Alex thinks of this. Yeah. Meanwhile, ads is a really great litmus test because they have to monetize every feature whether it's in legacy or modern and therefore they are very sensitive to slow migrations. Yep. So yeah, we go to different teams for different reasons and have like there's lots of ways to be part of the platform answer even if you're not on a platform team or if you ## Rapid fire round [01:20:30] choose to specialize in it. Nice. So as closing, let's just do some rapid questions where I'll I'll shoot the question and then you tell me what what pops up. So what's a framework that you really like for for its usability or elegance? If we're choosing usability and elegance, I have to pick Swift UI. But I was thinking about this question. I think my answer is actually lib GraphQL.js because that's the last framework that I uh interacted with that taught me a lot. Uh would recommend should read. I think for me uh it's compose. Um you know I love how maintainable and testable it is. It's just so much less boilerplate than anything I've used before. Um just really solid, you know, unidirectional design. Events work their way up, state works their way its way down. Um and you know, all these principles are are appealing for far beyond just the UI layer. Um you know, I know when you have a hammer, everything starts to look like a nail. Uh but you know, if I'm being honest, a lot of things have been looking like very appealing nails lately. Um, and you know at Reddit we we do use it for both our UI and our presentations layer, our presentation layer and it's been really great for us. I do also like compose the more I use it. Um, I was out of the actual IC world for a while but while but I have side projects that I've been really enjoying it again. Um, I picked a framework that is not a technology framework. Um I am often thinking about the western typology of organizational culture. It's in the accelerate book and I think about it a lot. Um it basically says that all cultures will eventually like settle into a pathological power oriented culture, a bureaucratic rule oriented culture or a generative performanceoriented culture. And when I first read about it, uh, I thought the whole answer was to keep moving people away from power cultures through bureaucratic into generative and that the best place for high performing teams was in generative. But now I have kind of come around to you have to choose the right tool for the job. And there's a reason why, for example, a QA team or um a like a deployment situation might actually be more rulesoriented and have a lot more process around it um than say how certain other teams could work. Um so how do you actually work together when you are like you have different cultures within teams? Um how do you get anything done when you have different incentives? So that's that's something that is I think of every uh people problem as a system design problem and so my answer is there are three types and how like what are what is the inter between them and then as closing what's a book that you would recommend and why I thought about this forever and I have to choose the KN&RC book um like I feel like it melted my brain a little bit when I really really understood that like object-oriented programing is like just C structures with function pointers inside of it. Like go through that book, type out every uh code example they have in it and I promise at the end of it you will be a confused programmer but you will be better. Can you repeat the book again? Oh the sorry the KNRC book the like classic Kernnegan and Richie uh Cbook. Oh did I get those names right? Carnean Richie. You did you did I have it in eight different forms. Is it like 35 40 years old? And the book's probably older than me. Okay. Still still works. It was a It was a Bible for a reason. It is a good book. So good. Uh my So I read a lot. Uh I'm I'm a really big fan of taking a first principle and trying to get a whole bunch of people to try it at scale in order to like change one small thing. Um so I decided my my recommendation right now is Kent Beck's Tidy First. We read it as a we read it as a team last year and had a whole multi-day conversation about how we could incorporate Yeah. The little cat book. It's also extremely skinny, so you're not investing too much, but it has a lot of great ideas in it. It's it's skinny, but lots of good ideas. Um, so yeah, like I would suggest like pick up the book and then just you can open it up to any part and just pick one thing and try it for a few weeks and see if it changes your flow. Um but that that particular one I think has has uh launched a thousand little ideas with our team. I think for me um this is where I start to lose my credibility as an engineer. Um Project Hail Mary is my uh my fiction book recommendation. Fantastic. Couldn't put it down. Awesome. Well, thank thanks very much for for this interesting conversation on what it actually takes to build an app that is a lot more complex on the inside than you know a lot of people assume who who download it and use it. This was great. Tons of Yeah, this was lots of fun. I hope you enjoyed this behind thescenes look into how Reddit redid the core parts of their native iOS and Android apps over the course of several years. Thanks very much to Lauren, Brandon, and Eric. and you can find all of them in the show notes linked below. For more details on mobile engineering challenges, you can find longer deep dives into pragmatic engineer as linked in the show notes. I also wrote a book titled Building Mobile Apps at Scale that contains more pointers on this topic. If you enjoyed this podcast, please do subscribe on your favorite podcast platform and on YouTube. This helps more people discover the podcast and a special thank you if you leave a rating. Thanks and see you in the next