Kill It with Fire: Begin Book Club

We had a great chat with Marianne Bellotti, the author of Kill It with Fire and were excited to learn more about her new project, Breaking up the Monolith.

Video

Transcript

Simon MacDonald: [0:20] Today we have author Marianne Bellotti to talk about her book, kill it with fire

Simon MacDonald: [0:12] just before we get started. I just want to reiterate that everything here today is covered under the Architect code of conduct.

Simon MacDonald: [0:19] If you don’t feel like reading all that legalese, it’s pretty simple, just be nice to each other, don’t be a jerk, that’s all that really comes down to.

Simon MacDonald: [0:30] so yes, today we have Marianne and just a quick biography.

Simon MacDonald: [0:35] She is an author, a relapsed anthropologist which I understand you can get a cream for and a software engineer.

Simon MacDonald: [0:42] She’s worked on large scale systems, legacy data infrastructure for the U.S. Government as part of the U. S. D. S. And I’m Canadian so I don’t know what U. S. D. S. Is, I guess that will be my first question and she currently runs platform and safety efforts at Rebellion Defense.

Simon MacDonald: [0:57] So welcome Marianne, thanks for coming.

Marianne Bellotti: [0:59] Thanks for having me started with what U. S. D. S. Is.

Marianne Bellotti: [1:9] U.S.D.S. Stands for United States digital services. And so in the aftermath of the healthcare dot gov of debacle.

Marianne Bellotti: [1:18] If you remember that the government tried to put up a website and that did not go well at all and they very quickly learned there’s something called site reliability engineering and like how all that works.

Marianne Bellotti: [1:30] there was an effort to what are the interesting things about that incident is that it happened at the same time there was a government shutdown going on in the United States.

Marianne Bellotti: [1:41] And so the rules about who can work and who can’t work are very strict during a government shutdown.

Marianne Bellotti: [1:46] And so a large they had this huge problem and like none of the people that would normally have fixed it could come and fix it.

Marianne Bellotti: [1:55] So instead they went out to Silicon Valley grabbed a bunch of people from google a bunch of people from like Netflix and like drag them over to D.C. And like had them come into these data centers and fixed, get the website standing back up and after they’ve done that the general consensus was hey that worked really well.

Marianne Bellotti: [2:15] And also there were a lot of things that we learned about that process from these people who would never come into government service.

Marianne Bellotti: [2:23] Uh and they learned a lot of interesting things about why the government works the way it did.

Marianne Bellotti: [2:27] So maybe we should stand up something a bit more permanent.

Marianne Bellotti: [2:30] That brings people from kind of more traditional technical private sector background into government technology and facilitates both the resolving of problems on critical systems but also the sharing of information and cross cultural exchange.

Marianne Bellotti: [2:45] And so I was at us yes for about 3.5 years and it was an absolutely completely game changing career experience that threw me in front of systems that were I thought I had worked in on old systems before I came to U.

Marianne Bellotti: [3:02] S. D. S. And then I realized that there are agencies that have computers that were built in the sixties that are still running actually like critical data processing jobs like every day.

Marianne Bellotti: [3:15] And that like blew my mind to smithereens. And so at that point I really learned what it’s like to work with some of these extremely complicated extremely old completely undocumented systems which was great fun.

Marianne Bellotti: [3:31] Which is why I ended up writing a book about it.

Simon MacDonald: [3:34] Yeah that’s awesome. And Like an anecdote from my own life. Again, I don’t think a lot of people understand how many government systems are particularly ancient but I did a co op work term in the 90s where I wrote a Damon that would get the weather data whether it was satellite radar, Doppler or whatever and transform it so that the meteorologists could actually give weather reports to Canadian military, the navy army, everything.

Simon MacDonald: [3:59] I met my boss in the mid 2000s and he’s like your daemon is still running. And I was just I was dumbfounded that this co op project I did over four months was still giving weather data like a decade later.

Marianne Bellotti: [4:15] It is amazing to me how often I hear things like that.

Marianne Bellotti: [4:19] Like I built this and it blows my mind that like 20-30 years later this is still what’s in production right?

Marianne Bellotti: [4:27] Like I thought this is good enough for right now. Certainly someone smarter than me will come along eventually and replace it with something good, right?

Marianne Bellotti: [4:35] And it’s like, no, like that’s never what happens in production.

Marianne Bellotti: [4:39] It’s like once it’s there basically sticks around forever.

Simon MacDonald: [4:48] Yeah, well Brian who’s on the call will often talk to us about the compounding value of software.Sometimes you can write software once and then you don’t have to change it for a super long time.

Simon MacDonald: [4:52] So you can imagine how much value you get out of stuff like that.

Simon MacDonald: [4:56] Yeah, but as tradition here on our begin book clubs, we like to do a little bit of a dramatic reading and I know we talked about it a little bit on email, but have you selected a passage to read from your book?

Marianne Bellotti: [5:08] I have, although I was really hoping, oh it’s getting grossed in a really intense conversation and then I won’t have to do the dramatic reading because I really, I really find it extremely difficult to read out loud, but I’m gonna I’m gonna go for you guys because I like honoring tradition.

Simon MacDonald: [5:23] You know, I kind of thought that’s what you were trying to do.

Simon MacDonald: [5:25] You like, let me steer the conversation forget all about the dramatic reading.

Simon MacDonald:[5:30] Nice try to make an attempt.

Marianne Bellotti: I had to make an attempt.

Simon MacDonald: [5:33] That’s not my first rodeo with these things.

Marianne Bellotti: So I pulled a passage out from Chapter five and kill it with fire.

Marianne Bellotti: [5:39] Chapter five is all about the idea of building momentum or losing momentum and roll the momentum plays in the success or failure of modernization efforts.

Marianne Bellotti: [5:49] And so it basically has a series of different types of things that are either momentum boosters or momentum killers and then kind of goes into a little bit more of a case study for each one.

Marianne Bellotti: [6:01] And this one, this one is about the history of failure, which I think will be broadly applicable to everybody who’s listening to us chatting right now.

Marianne Bellotti: [6:11] Everyone has been through this kind of experience with, with just technical debt in general.

Marianne Bellotti: [6:17] So here we go, momentum killer. A history of failure odds are good that the modernization effort you’re working on now is not the first attempt.

Marianne Bellotti: [6:28] Companies that successfully maintain their technology over time usually don’t need to engage in a big modernization project at all.

Marianne Bellotti: [6:35] They’re able to keep up through incremental change and regular maintenance.

Marianne Bellotti: [6:39] If you’re running a team tasked with just cleaning up debt and migrating onto a more suitable technology, it means the existing organization has failed to adapt.

Marianne Bellotti: [6:48] So your specific situation might have a history of failure that’s much deeper than slacking off on just regular maintenance.

Marianne Bellotti: [6:54] Is this even the first modernization project, if not each prior effort likely has left scar tissue on the organization that you need to consider the more false starts.

Marianne Bellotti: [7:05] A project has had, the harder it is to build momentum necessary to succeed.

Marianne Bellotti: [7:10] The first deliverables of modernization efforts have to take this history of failure into account.

Marianne Bellotti: [7:15] People aren’t pessimistic or uninspired by legacy modernization projects because they don’t care or don’t realize that modernization is important.

Marianne Bellotti: [7:23] They often feel that way because they are convinced that success is impossible.

Marianne Bellotti: [7:27] After experiencing a number of failures at the same time, I have yet to find a group of engineers who didn’t want to believe that they could reach a better state.

Marianne Bellotti: [7:36] It’s surprisingly easy to change people’s minds about the inevitability of failure when you demonstrate that success is possible.

Marianne Bellotti: [7:43] So inspired and motivated engineering teams run smoother and more productive modernization processes.

Marianne Bellotti: [7:49] So design your modernization strategy around front loading value. What changes will produce the most immediate positive impact?

Marianne Bellotti: [7:58] I once worked for an organization that was facing a major challenge around the breakup of its monolith.

Marianne Bellotti: [8:03] The organization wanted to build a standardized platform that production engineering teams could use to deploy services to production easily.

Marianne Bellotti: [8:10] A reasonable ambition. But the product itself was three monoliths crammed into a single VM.

Marianne Bellotti: [8:16] It was the monolith of monoliths, if you will at that time that it had been built that architecture fit the business case.

Marianne Bellotti: [8:24] But in the years that followed, the organization had seen explosive growth and by the time I got there, the architecture didn’t make sense anymore at all.

Marianne Bellotti: [8:31] This organization was facing two problems. First, the platform initiative and the monolith breakups were blocking each other.

Marianne Bellotti: [8:38] The product team did not want to break up their monolith into services.

Marianne Bellotti: [8:42] They could deploy on the platform, understandably. They didn’t want to put something on a release pipeline only to have it migrate off when the platform finally arrives.

Marianne Bellotti: [8:52] But the platform group on the other hand, cannot build the platform without requirements set by the product teams, they had to be able to build with the real needs of real services in mind services that did not exist because they hadn’t yet been broken off the monolith.

Marianne Bellotti: [9:06] The second problem was the organization actually tried both of these sides of the process before and failed at them multiple times and had tried to build a platform and migrated some small unimportant services that could be split off with minimum redesign.

Marianne Bellotti: [9:20] It had tried at least three times by my estimation each time losing momentum and failing to finish, the organization had also tried to break up the monolith several times.

Marianne Bellotti: [9:30] Each time it became overwhelmed by the complexity of the task splitting monoliths is rarely if ever only about copying and pasting some code into a different repository.

Marianne Bellotti: [9:40] When the software is designed to be coupled, engineers usually take advantage of that fact and build on the easy access that coupling provides.

Marianne Bellotti: [9:48] In this case that meant their testing suites had a high concentration of end to end tests over unit tests and then multiple components were accessing the same data store and sharing responsibilities over the same information when they’re tightly coupled monolith became decoupled services.

Marianne Bellotti: [10:03] The test would break and the plan on keeping that data consistent between services would need to be developed.

Marianne Bellotti: [10:10] Now facing their fourth attempt optimism was pretty low. Everybody wanted to see the project be successful but no one wanted to be the first team to invest work and only to be left holding the bag when the effort fell apart.

Marianne Bellotti: [10:22] Once again, prominent engineers on the platform group were asked to come up with a plan.

Marianne Bellotti: [10:27] They spent weeks collecting data and interviewing teams and eventually pitch the following compromise they would pull the three monoliths onto their own release channels with their own videos thereby ensuring that platform could support everything the product needed without running the product team without requiring the product team to split anything immediately.

Marianne Bellotti: [10:46] The problem with this plan is that it didn’t actually make anything any better.

Marianne Bellotti: [10:50] Now instead of one release cycle with an owner and an orderly schedule determining when code hit each region and in each environment, the organization, we have three re release cycles with no one actually owning them every deploy would have to be carefully coordinated against multiple teams.

Marianne Bellotti: [11:05] So the changes that actually hit production before one monolith early or too late.

Marianne Bellotti: [11:10] Um, this wasn’t gonna lower costs. Either commercial cloud providers charge per usage of time for each VM three separate set of VM’s meant the proposed plan would just double or triple the organizations hosting expenses.

Marianne Bellotti: [11:23] I wasn’t even sure it would get off the ground. My team had been working hard on redesigning a service that appeared to be fully separate from the get go from the monolith and moving on to the platform.

Marianne Bellotti: [11:35] And even in that situation we were finding all sorts of weird places where components were integrated in unexpected ways.

Marianne Bellotti: [11:42] So what was the value of putting three monoliths on separate release channels?

Marianne Bellotti: [11:46] I asked and when I asked that question, the engineers thought what I was asking is what was the value of breaking up the monolith?

Marianne Bellotti: [11:51] It took several conversations before I could get them to understand that I wasn’t questioning their goal.

Marianne Bellotti: [11:56] I was questioning their starting point. Starting with by tripling the number of VM’s would make updates more complicated for product teams and would increase the spending unnecessarily.

Marianne Bellotti: [12:05] Why would the organization continue to invest in the process of breaking up the monolith?

Marianne Bellotti: [12:10] If its first experience with that process made work harder and more expensive.

Marianne Bellotti: [12:14] The hard problems around legacy modernization are not technical problems are people problems, the technology is usually pretty straightforward, keeping people focused and motivated through the months or years it takes to finish the job is hard to do this.

Marianne Bellotti: [12:28] You need to provide significant value right away as soon as possible so that you overcome people’s natural skepticism and get them to buy it.

Marianne Bellotti: [12:36] The important word in the phrase, proof of concept is proof you need to be able to prove to people that success is possible and worth doing, the more an organization has failed at something, the more proof it needs that modernization to bring to the table if there when there’s a history of failure, that first step has to provide enough value to the momentum necessary to be successful.

Marianne Bellotti: [12:58] The obvious problem with that is that it means there’s a natural upper bound, there’s a point where the cynicism is so high and no single first step will ever provide enough value to prove the project will work and then what and then at that point we go into and what do you do in that scenario?

Marianne Bellotti: [13:17] So it’s a bit of a cliffhanger. You’re gonna have to buy the book and read to figure out what the solution.

Simon MacDonald: [13:21] Yeah. And I mean anybody who’s watching this recording later, I do highly recommend that you buy the book.

Simon MacDonald: [13:27] It. It’s fantastic. We all really enjoyed it. Now when you’re reading that passage, you know, some of the things that were coming to my mind three points and you don’t have to address them all.

Simon MacDonald:[13:37] It’s like, but to boil things down into those who do not learn from history are doomed to repeat it.

Simon MacDonald: [13:44] If you do not, you know, focus on the performance bottleneck, then you’re never gonna get anywhere and that the code is easy.

Simon MacDonald: [13:51] The people are hard, like if I could, if I could sum those things up, that would that would be what I would take from that.

Simon MacDonald: [13:56] And I remember thinking that when I was reading the book, that it was just like, yep, if you if the, you know, the manager who failed that project got promoted up to a director and then the new person comes in and doesn’t ask the engineers what went wrong, you’re gonna have it go wrong again.

Marianne Bellotti: [14:11] Yeah, A lot of times that people come in and suggest the same ideas over and over and over again and that just unnecessarily increases people’s cynicism about the ability for it to work.

Marianne Bellotti: [14:22] Like sometimes it’s actually the right strategy that’s gotten proposed over and over again.

Marianne Bellotti: [14:26] But there’s some, you know, internal political or timing or scheduling or context reason why it failed and it’s worth giving it a shot again.

Marianne Bellotti: [14:36] If you can make a reasonable case that that you’ve identified what that problem is and it’s no longer the context, but sometimes it failed for a really critical reason like that, it’s not gonna work at all.

Marianne Bellotti: [14:49] And no matter what the scenario is, it’s always better to know that going into the process than to just assume like, oh, well, obviously no one thought of this solution before, I am the first person who’s ever considered this path forward.

Simon MacDonald: [15:04] Yeah, it’s so great. It reminds me of that arrested development meme where it’s like, but it might just work for us.

Simon MacDonald: [15:11] And if the constraints haven’t changed, it’s probably not gonna work.

Simon MacDonald: [15:15] You’re gonna run into the same problems. But Francis, I’m so glad you were, you know, you got here and it looks like you got your hand up to ask questions.

Marianne Bellotti: [15:21] So just, you know, unmute and go for it. Yeah.

Francis Gulotta: [15:25] Thanks. Sorry. I was like, I say, I’ve never had novel thoughts joining an organization, but, but there’s always, there’s always so many good ideas just sitting on people’s desks thinking they don’t have permission to do it, you know?

Francis Gulotta: [15:38] And, and there is, um, there was definitely a surface level of those projects like, you know, we never enabled glinting, I wonder if anyone would be mad, maybe they would, I won’t do it, you know?

Francis Gulotta: [15:48] And, but then there’s often like deep architectural changes like boy, I know I could cut like cost 20% improved performance if we did it this way, you know, whatever.

Francis Gulotta: [15:58] Yeah. And it’s a learned, especially when you have this big legacy and it’s very difficult to change or understand the impact of changes like people give up fast and, and um, because, well, I don’t know who owns this or, you know, I don’t know like what the responsibility is, our ally have other work to do.

Francis Gulotta: [16:15] And um, so like in my, in my legacy, modern modernization projects in the past and has been solely just supporting people’s good ideas, like, no actually like I hope sponsor you get this out there, you know, it’s like you can totally sell this,

Marianne Bellotti: [16:32] That is that is a piece of advice I would give a lot when I was consulting when I was part of U.S.D.S. Whenever you’re in a situation where you’re almost like a visiting guest star in the organization and you don’t necessarily connect intuitively into the chain of command or have any ownership of your own.

Marianne Bellotti: [16:49] I would tell my team have a general idea of what you think the right solution is and then go and find the person who already agrees with you who works here and the older the system is, the more people there are that have had that same thought in the organization, right?

Marianne Bellotti: [17:07] And they’re like in government especially they’re all like screaming into the void just going like well someone listen to me and so be the person that comes there and listen to them And then that what your job becomes at that point is to use whatever credibility and social capital you have with their leadership to set them up for success and put them on the stage. 100% agree.

Simon MacDonald: [17:28] But I mean would you take credit for their ideas afterwards?

Marianne Bellotti: [17:43] The Mackenzie strategy which is like to go and figure out what their own people have been telling them for like the last five years, write it up in a report and then charge them millions of dollars for it and it’s I don’t think anybody is happy with that reality, but like when people have paid money to be told something, they inherently trust it more than when they’ve gotten that advice for free.

Marianne Bellotti: [18:09] And this is like the bane of every software engineer’s life because people have still like people watched this happen over and over again, is that you you’re very honest with your leadership about what the path or it is and then they hire this like fancy consultant who says the same thing and like now suddenly it’s like a thing,

Simon MacDonald: [18:35] I think that’s like a logical fallacy that that human beings have that the more familiar the person is giving the advice, the less likely they are to trust it. It’s like seems like you know experts that they don’t know who have never steered them wrong are more likely to be trusted.

Simon MacDonald: [18:41] I mean I know that when my dad has a software problem or a computer problem, he doesn’t come to me, he comes to like some of his other friends who are lawyers and it’s like okay cool, well I don’t have to spend, you know three hours on tech support tonight, so cool, I shouldn’t complain about that.

Simon MacDonald: [18:56] But speaking of logical fallacies early on in the book you do talk about the self serving bias and where people tend to overestimate how much their skill goes into a successful software project and underestimate the luck that’s involved in projects as well.

Simon MacDonald: [19:14] I was wondering if you could get into that a bit more when it comes to legacy systems.

Marianne Bellotti: [19:20] I think folks would be interested in that. Yeah, I mean I think that first of all as software and juniors, we always, we have a very strong grass is greener somewhere else kind of narrative.

Marianne Bellotti: [19:33] We always assume that like really effective systems must be built very elegantly and perfect.

Marianne Bellotti: [19:40] And one of the great parts about gaining more experience as a software inch here is going more places and realizing, nope, this is a hot pile of garbage to like over here as well.

Marianne Bellotti: [19:53] This company that I admired and respected and that makes billy, this is also a hot pile of garbage.

Marianne Bellotti: [19:58] I thought you guys knew what you were doing right, but it takes time for engineers to sort of develop that understanding that like a system can work and work really, really well and not conform to great architectural standards have all sorts of little problems with it and also have all sorts of little inconsistencies and um, failure points that are never realized right.

Marianne Bellotti: [20:21] And the more complex the system is, the more likely it is going to have these little inconsistencies in these little error points.

Marianne Bellotti: [20:27] And so a part of what makes the system successful is a strong amount of luck, right?

Marianne Bellotti: [20:35] Like a system needs to be designed in a way and that both suits It’s just kind of objective architectural best practices, but also the context in which it’s going to be used right?

Marianne Bellotti: [20:47] Both how the user person is gonna be interacting with the system the most is going to use it and what they’re going to use it for and what their assumptions are, but also how the organization is going to invest in it over its lifetime.

Marianne Bellotti: [21:00] And so those are things that are often not intuitively factored into the process of designing systems but definitely like push your luck principle in a favorable direction if you can sort of anticipate them.

Marianne Bellotti: [21:16] And I think that the luck factor becomes most deadly when we think about rebuilding systems either wholly rebuilding them from scratch or rewriting a part of it.

Marianne Bellotti: [21:29] because the natural tendency that that people have is to assume that if the system is successful, it’s because all of the edge cases were worked out, all the bugs were worked out, it’s like solid.

Marianne Bellotti: [21:45] And now we can not only just reproduce that functionality, But we can also add more functionality on top of that.

Marianne Bellotti: [21:53] We can pull out that wish list we’ve been holding onto for 20 years and start implementing some of that and that very quickly falls apart because you discover things about the system that like honestly either shouldn’t have worked or maybe never worked in the first place and now your basic underlying assumptions are completely shocked that never worked in the first place.

Marianne Bellotti: [22:13] Oh my goodness. Yeah. Isn’t that fun when you think to yourself?

Francis Gulotta: [22:18] Wait oh man okay so like the production is the spec and so now you gotta every bug is the spec and now your spec is ridiculous and you know like so can I make this product changes, can I you know how do I version this?

Marianne Bellotti: [22:38] Yeah I mean the most famous story of that is actually Microsoft with Microsoft.

Marianne Bellotti: [22:44] Excel? Lotus 123 made an error on how they calculated leap years and millennials right?

Marianne Bellotti: [22:54] You don’t normally have a leap year on like a century but you do have them a millennial.

Marianne Bellotti: [22:59] I think someone’s gonna call me out on getting that wrong. But anyway whatever it is they got it wrong right?

Simon MacDonald: [23:01] Any year divided by 100 should not be a leap year but any year divided by 400 should be a leap year.

Simon MacDonald: [23:10] And I think I learned that for some programming contest once upon a time and I’ve never lost it.

Marianne Bellotti: [23:16] Yeah so Lotus 123 made that mistake. Microsoft. Excel writes that bug into every iteration of Excel that comes out like need because they need backward compatibility with all of the spreadsheets that have never been produced on Microsoft.

Marianne Bellotti: [23:33] Excel and when Microsoft Excel came in it was making a play to get rid of, you know Lotus 1 to 3 so it wanted to have backwards compatibility with all the list.

Marianne Bellotti: [23:41] So it’s like generations of software engineers have now been like maintaining this bug in Excel.

Marianne Bellotti: [23:49] And like I love that story the first time an engineer told me that story I was like that is the most amazing piece of knowledge.

Simon MacDonald: [23:56] Anyone was never part of me. But it’s true I’ve seen that at companies I worked for as well, I worked for a telecommunications company in the early two thousand’s.

Simon MacDonald: [24:06] And Call Control was originally written in the seventies and there was a bug in the call of control and we tried to it went through the analog stage, it went through the digital stage in the two thousands we were getting into voice over IP. They were like oh we’ll fix this bug. Oh that was a bad idea because that that bug was now the spec just like Frances had mentioned so do not touch that bug that we had to big blocks around the codes like this is this is supposed to be this way. I know it looks wrong.

Francis Gulotta: [24:27] Yeah I’m in a similar thing where it’s like I cannot change existing uh I work for a web hosting company cannot change existing behavior of deployed websites, I can maybe change it for the future.

Francis Gulotta: [24:45] But uh but boy we have to talk about it but you know so it’s like if you’re using this feature and this here and I can’t tell you if you are, you know, and this educates, it’s just going to go away, like you’re gonna be completely different and never match the experience before it does now, but it’s gonna be different, you know, you know, and, and I could have, I could have a, I don’t know, I could be releasing a change like that a day.

Francis Gulotta: [25:10] Um, and so it’s like, how do we reversion it and we have to like figure out how to put the life cycle on it, you know?

Francis Gulotta: [25:18] Now I actually, I still have to do this forever, you know, and, and, and, but I can do the good thing in the future, you know, and, and it’s, I, I almost want, I’m a big proponent of slowly rewrite everything, you know, maybe stick a service in front of it just to like hide the fact that it’s, you know, the strangler pattern, but like I almost want to like completely just lock it in time, don’t ever fix it and start fresh and new and I’ve never had success with complete rewrites, but it makes it, but it’s like it might be better than documenting the undocumented.

Marianne Bellotti: [25:53] Yeah, yeah, I mean complete rewrites. There are places I don’t ever wanna and, and this is a really strong point that I make in the book, there are no silver bullets, but there are also no strategies that I feel like there’s never a scenario where this is the right thing to do.

Marianne Bellotti: [26:08] And so full rewrites, I actively discourage people from considering, but there are certain circumstances where in fact, I think that probably ends up being the right thing to do.

Marianne Bellotti: [26:18] And the example given the book is like front end stuff, like I’ve seen several full rewrites of like front end frameworks and like, you know, nowadays friends so inherently complicated, it’s like a full layer onto itself and like you really don’t, you’re gonna redo your front end.

Marianne Bellotti: [26:35] you really don’t want half of the website running on one front end while the other half is running on another front end.

Marianne Bellotti: [26:40] It’s not a place where it really makes much intuitive sense to do a page by page migration of the front end.

Marianne Bellotti: [26:46] So like that’s a great example where like I need to put those kind of things in the book because if I say absolutely never do a full rewrite, someone will come into an engineering meeting.

Marianne Bellotti: [26:55] We throw the book down and says, Marianne says, we have to do our front end migration piece by piece and I’m just like, no, that’s not a bad idea, right?

Kristofer Joseph: [27:05] Yeah. I think the thing that kept coming up to me was this inherent tension between product planning and developer experience, right?

Kristofer Joseph: [27:13] Like, and when I say developer experience, I mean like the developers that are currently working on the product no more than the planners most of the time, right?

Kristofer Joseph: [27:20] It’s like the product planner is going to be like, hey, we need these features, we need to keep moving this product forward and the developers, like there are reasons we did these things right?

Kristofer Joseph: [27:29] And I don’t really know what the net out is. I’ve worked on so many products that like features March ahead, there’s nothing you can do about it and your with the decisions that are made by people that aren’t in the code all day long, you know?

Kristofer Joseph: [27:41] And so I was trying to recommend that people who are senior developers move into products, like immediately I talk more about that.

Marianne Bellotti: [27:52] I feel like that’s, that’s the thing though, it’s the same thing with like a senior developers becoming engineering managers.

Marianne Bellotti: [27:59] Yes, there’s a huge value if you’re especially your highly technical company of also having your product will be highly technical so that people can speak the same language and like when you emphasize a trade off to them, they get it.

Marianne Bellotti: [28:12] But at the same time, the more time you spend not being a software engineer, the less that knowledge, you know, the state where that knowledge gets and then you get to these really difficult situations where you have someone who is not as technical as they very strongly believe they are and that’s worse than having a non technical person in that role is having a person that is not as technical as they think seem to think they are because then it just becomes impossible to sort of reason with them.

Marianne Bellotti: [28:42] I think that like the conclusion that I’ve come to after being through like several iterations of this kind of internal politicking and then seeing it in the federal government.

Marianne Bellotti: [28:55] And what’s really interesting about the dynamics of the federal government is that the government can’t go out of business, right?

Marianne Bellotti: [29:02] So like organizations and products and technologies, all of these things that sort of natural life cycles where the end of it doesn’t necessarily mean that you go out of business, but it does mean that you have you’re not the same company of the same organization.

Marianne Bellotti: [29:19] You were at the beginning of that cycle, right? You kind of transition to a different scale and a different sometimes a different culture and like a lot of the things that keep this behavior in check in engineering teams are that if you don’t keep it in check at some point you’re gonna go bankrupt.

Marianne Bellotti: [29:38] And so what was really interesting is to see technology built in an environment where that was not true, right?

Marianne Bellotti: [29:43] No matter how terrible the IRS Brand is the iris is not going out of business anytime soon, right?

Marianne Bellotti: [29:51] We need this in place, it’s sticking around right? Like there’s no you can pick any agency on the planet like they don’t shut down government agencies room at any at any time.

Marianne Bellotti: [30:02] So you have that kind of like natural forcing function pressure.

Marianne Bellotti: [30:08] It just doesn’t exist and as a result things are allowed to fester and like limp along that would never ever survive in the private sector and so you get really interesting technical problems in that kind of environment.

Marianne Bellotti: [30:25] I used to tell people it’s like the Ripley’s believe it or not of technology right?

Francis Gulotta: [30:31] How do you address it? How do you, how do you clean up inefficiencies in government?

Marianne Bellotti: [30:36] I mean, so there’s first a certain degree of patients and a certain degree of acceptance, right?

Marianne Bellotti: [30:42] First being very realistic on what you can actually get done.

Marianne Bellotti: [30:47] and the thing I would, the advice I would always give people is I always want people on my team to have one selfish goal doesn’t have to be a big selfish goal but it has to be something for you and just for you because if you come into that kind of experience of trying to work in that environment and do something good for people purely altruistic.

Marianne Bellotti: [31:10] Like you’re gonna have days where like either the bureaucracy or just the like absurdity of the environment, you’re in just kicks your butt, right?

Marianne Bellotti: [31:20] And people, if they don’t have something that’s just for them that’s attainable that they can sort of focus their attention on, they get really burned out very quickly.

Marianne Bellotti: [31:30] They get really like bitter and resentful and then they become more of a problem rather than the solution.

Marianne Bellotti: [31:36] So that was the advice that we gave people is, let’s talk about your career, like, do you want to learn a different piece of technology?

Marianne Bellotti: [31:42] Do you want to, like, what do you want to accomplish?

Marianne Bellotti: [31:44] Like, do you want to like filled up management skills? Like what is it for you that you’re like, if this, if you come away from this experience able to say, put this thing on your resume or have this thing going for you or made those network connections to like, you know, people in Congress or whatever your particular deal was, you will look back on your work and you will be happy and satisfied with your work, no matter what, That that was really a great way of keeping people kind of focused and invested in helping them kind of get through it.

Marianne Bellotti: [32:15] And then like when you’re operating at the scale that these systems operate in, people underestimate the amount of change you need to implement in order to make a huge difference.

Marianne Bellotti: [32:27] Like the story I would always tell people about this was some work we did very early on the refugee emissions system.

Marianne Bellotti: [32:34] This was during the Obama years. So back when refugees were great and we wanted them in the country as much as possible.

Marianne Bellotti: [32:42] Uh, and like the Obama administration had given the DHS this absurd goal of like how many people they wanted to be brought into the country within a timetable of a couple of months.

Marianne Bellotti: [32:55] It was just like a completely divorced from reality if you look at what the system was actually processing And the team that was working on that that system is part of us.

Marianne Bellotti: [33:05] Yes. What they did is they kind of mapped out the process and they look for bottlenecks and they look for a place where they could get a 10% improvement and a 5% improvement at 15% improvement.

Marianne Bellotti: [33:16] And those things don’t sound like big changes and none of them were.

Marianne Bellotti: [33:19] But at the end of this process, when they had gotten all these tiny, tiny little changes, I saw the person who was running that team on his way to the West Wing and he was going into this meeting to report out and he’s like, I’m really, really nervous about this meeting and I’m like, why?

Marianne Bellotti: [33:35] He goes well, because we’re gonna overshoot that insane number by about 700 people.

Marianne Bellotti: [33:42] And I’m trying to figure out a way of delivering that news that won’t result with the White House, putting a new, even more ridiculous number on the table like that with a shorter turnaround time.

Marianne Bellotti: [33:55] And I’m like, well man, that sounds like a really good problem to have good luck your meeting, right?

Marianne Bellotti: [33:59] So that people really, you know, it is unlikely in that kind of environment that you’re going to create a completely new system and completely reform these, these ancient machines, but that doesn’t mean you can’t have any impact at all.

Marianne Bellotti: [34:13] You can have a lot of really dramatic impact. It just doesn’t necessarily look like the, you know, sexy conference talk that you wanted to look like.

Francis Gulotta: [34:22] One question I had from the book around this is like, like you mentioned now is the process mapping, right?

Francis Gulotta: [34:29] How do you identify uh slow area especially and a lot of like these systems, it sounds like there’s a lot of people systems involved in the story you just told, but like, like there’s a lot that is not very observable and older systems and it’s very hard to get observable.

Francis Gulotta: [34:45] And I loved how you put like putting it, putting it into another service or putting an interface in there unless you observe it, you know?

Francis Gulotta: [34:51] And I thought that was fantastic advice, but when some of these things are complected as heck, you know, like it’s uh how do you approach that?

Marianne Bellotti: [35:00] So the first most basic way to approach it is figure out like take a user persona, some person who uses this system to do something and like kind of do a day in life, like track their behavior as they’re trying to do that thing through the system.

Marianne Bellotti: [35:16] Like where does that like packet of information go as it twists its way all around.

Marianne Bellotti: [35:22] And often in older systems it is a hybrid of computer systems and people systems sometimes actually manual.

Marianne Bellotti: [35:29] Like there were places in government where something would be submitted electronically and then printed out and scanned back in to a different computer system.

Marianne Bellotti: [35:39] Right? So that’s what I always find really interesting about talking about systems is that we are now at the point of digitization of society where we can no longer talk about computer systems as if they begin at the network boundary and you know like the database or what have you.

Marianne Bellotti: [35:58] Right, like they’re there, they reach out there are people involved in these systems.

Marianne Bellotti: [36:03] There are manual processes involved in this system that it’s a socio-technical environment rather than just like a purely technical system that we’re looking at.

Marianne Bellotti: [36:13] And so that’s the first most basic way of thinking about it is to pick a user or piece of data and to trace its path from beginning to end.

Marianne Bellotti: [36:22] And then once you have those steps figured out, then look at like how can you estimate the time each step takes.

Marianne Bellotti: [36:30] It’s really wonderful when you can do that in the traditional like movie monitoring this system and we know how fast this is.

Marianne Bellotti: [36:36] But sometimes it’s just kind of like talking to an analyst and going like about how long does it take something to get approved?

Marianne Bellotti: [36:42] Five days. Okay. Right, five days down. Right, so like that, that’s how you do it.

Marianne Bellotti: [36:47] I am, it’s funny you bring this up because like this is kind of the focus area that I’m at right now is thinking about a more structured framework for mapping out processes and systems.

Marianne Bellotti: [36:59] What is interesting about it sounds like really simple and I was actually surprised that the design community didn’t have really a framework approach to it because one of the problems that you find very quickly is that it really depends on your level of abstraction, right?

Marianne Bellotti: [37:20] Like you can treat a system as a black box on a process map or you can go into the system and look at every process instead that system that touches that exchange of data.

Marianne Bellotti: [37:32] Right? And like with really large models of system behavior, what will often cause problems for the person trying to reason and use the model to reason about it is that some part of the model is at one level of detail and another part of the model is a completely different level of detail.

Marianne Bellotti: [37:50] And another part of the model is that like yet another level of detail.

Marianne Bellotti: [37:53] So it’s the metaphor I use to explain this to people is like, it’s like if you said I have something that is this many inches and something that is this very centimeters and I’m just gonna take the numbers and ignore the unit of measurement and add them together and like that’s the total length of the thing that I have right?

Marianne Bellotti: [38:10] It’s like you can look at that and it’s an easy mistake that people make all the time.

Marianne Bellotti: [38:15] But as soon as you realize that there are different units of measurement, you understand that the result you’ve gotten at the end of it is complete bullshit.

Marianne Bellotti: [38:21] So I’ve been spending a lot of time thinking about how do we really approach this, this process of like doing architectural diagrams and representing systems in such a way that they have, we can want to highlight when they’re not consistent and to prevent uh inconsistencies like that, I don’t have the greatest answer for that.

Marianne Bellotti: [38:46] So stay tuned. I’m starting to sort of develop a kind of core concepts around it.

Marianne Bellotti: [38:52] I think one of the big problems that a lot of the modeling languages and approaches that we already have hit is that they primarily focus on expressing what systems look like.

Marianne Bellotti: [39:06] Like UML for example, will be like, here’s the database, here’s the server, here’s a network boundary, right?

Marianne Bellotti: [39:14] That’s what the system looks like. That’s not describing what it’s doing necessarily and what it’s doing.

Marianne Bellotti: [39:21] Its behavior is usually what you want to base your reasoning on.

Marianne Bellotti: [39:25] Right? Not like what components are here, but like what are those components actually doing?

Marianne Bellotti: [39:30] What are the state of those things in any one place?

Marianne Bellotti: [39:32] I could get very deep into the weeds on this, but I think we should move on.

Francis Gulotta: [39:37] You can listen all day to.

Simon MacDonald: Well, I’d like to drop in with a question because one of my friends, he wanted to be here for this.

Simon MacDonald: [39:46] Unfortunately he got pulled into another meeting. Um, so he’s actually struggling with updating some legacy systems right now.

Simon MacDonald: [39:53] The company that he joined as a large insurance company and the mainframes, everything was written in COBOL and they hired one of those fancy, you know, companies that would come in and tell you everything that’s wrong with your system and fix it up and they decided to run a trance pile er to change all that COBOL code into dot net code and you Yeah, okay.

Simon MacDonald: [40:11] So yeah, Francis already guess where this is going. You can imagine how well that that is going and I know that for us at Begin we are, we’re not huge fans of transpilation.

Simon MacDonald: [40:23] I mean in the, in the JavaScript world there’s, there’s typescript, which is, is great if you want to do type checking, but you end up debugging code that you didn’t write because what you, what you write and what is running on the system is completely different and it just adds a build step and, and I know that you do talk about, you know, transpilation in your book a little bit, especially around, you know, dev tools and python two and python three and your experience of going from COBOL to Java.

Simon MacDonald: [40:48] So if you can give him some words of encouragement, I would greatly appreciate it because he’ll be watching this later, tell him that his world is not over, He can get through this even if you have to lie.

Marianne Bellotti: [41:02] He can get through it. The problem is that like it’s gonna take awhile right?

Marianne Bellotti: [41:06] Like and so the thing that I think is the biggest warnings that sign about transpilation as a solution and legacy modernization is that it is attractive to people because they see it as like the easy button which is complete misunderstanding about what’s gonna go on right?

Marianne Bellotti: [41:24] Like it’s like you’re gonna do that, go over your eyes, open your expectations accurate about what you’re actually doing.

Marianne Bellotti: [41:31] This is not a quick fix, it’s not a cheat, like not there are no computers that rewrite code for you.

Marianne Bellotti: [41:37] Unfortunately if there were half of us would be in a different line of work right, they just don’t exist.

Marianne Bellotti: [41:44] So when you’re transpilation something, somebody is gonna have to go through that transpired code essentially line by line and make sure that the translation makes sense for what the actual code is trying to do.

Marianne Bellotti: [41:59] So most engineers would say let’s not do it because that’s just twice as much work as it would have been to just like do it the old fashioned way and they’re true and they’re 100% right about that.

Marianne Bellotti: [42:10] A lot of people in the management layer that make the decisions about what gets bought and what the strategy is, do not understand it.

Marianne Bellotti: [42:16] So the big struggle your friend is gonna have is that they have come in with this feeling that we’re gonna transpire it and then like maybe somebody will come through well, like, you know, fine tune it a little bit, but then it will be done and we’ll be done in like a weekend or like whatever, right?

Marianne Bellotti: [42:29] And like, hopefully they may have figured out that that’s not the case, right?

Marianne Bellotti: [42:35] but the big challenges, if they haven’t figured out that that’s not the case, and then the conversation becomes, well, maybe you’re not good enough to like just clean up the craft.

Marianne Bellotti: [42:45] So like, this is a place where one of the things that I talked about in the book and just generally with engineers is this idea of air cover, like who in that leadership level, if not the top layer of leadership, then the leadership, leadership level above you, directly above you.

Marianne Bellotti: [43:02] Like is there to support you and make sure you’re researched appropriately and that you can move forward and then your advocated for and to make sure that, you know what the answer to that question is before you engage in these issues.

Marianne Bellotti: [43:16] And then the other thing that I talk about in the book, that is just kind of an unfortunate reality of things is that you can’t actually help people that don’t want to be held.

Marianne Bellotti: [43:25] So you should always have your limits and your boundaries in terms of what you are willing to do and go through and what you aren’t willing to go through.

Marianne Bellotti: [43:34] I’ve walked away from projects because I loved the project. I love the potential of it.

Marianne Bellotti: [43:39] I saw what it could be but I was dealing with leadership or sometimes just the work as at its own that just wasn’t interested in what I had to say and I’m like all right, cool.

Marianne Bellotti: [43:53] I don’t know if you paid me money to come here, if you weren’t gonna listen to me but what?

Marianne Bellotti: [43:56] Whatever right? Like I’m not gonna waste any more of your time.

Marianne Bellotti: [43:59] I’m just gonna say like that’s the way it is best of luck and the way you’re going.

Marianne Bellotti: [44:03] That’s an unsatisfying answer for a lot of engineers. But I think when you have that those boundaries you have a better career and you do work that you feel better about.

Marianne Bellotti: [44:15] One of the best pieces of advice is advice I ever got was from Mikey Dickerson who told me I want you to know in your heart that you can quit on a Friday and he was my boss.

Marianne Bellotti: [44:27] So he’s talking about quitting the job that he gave me quit on a Friday and have a new job on Monday because you will make better decisions as an engineer if you believe that and I do not want engineers making decisions because they are scared they are going to get fired and that they will not be able to find another job afterwards because those decisions are bad decisions.

Simon MacDonald: [44:47] I think we need to get stickers made with that possibly an embroidered pillow because that is some really good advice.

Simon MacDonald: [44:53] Yeah, great advice. Yeah, I think there’s sorry Francis, go ahead.

Francis Gulotta: [44:59] No, I’ve got a I’ve got a new question for it.

Francis Gulotta: [45:03] All right. I didn’t really realize until this call, this was written for legacy modernization and not early stage startup to late stage startup modernization.

Francis Gulotta: [45:17] And um, in my head that’s what I was applying it to.

Francis Gulotta: [45:24] You know, not everything applied and I kinda understood, but what differences have you seen around those two worlds?

Francis Gulotta: [45:32] Because they are different worlds? They share a lot in common.

Francis Gulotta: [45:35] Often there’s an early monolith that does everything and there’s no spec and it’s lived forever and it’s and it’s hampering the ability to make change.

Francis Gulotta: [45:42] You know what, you know, we figured out our business model on it, but it’s hard and it should be easy and if it was easy boy, we could do so much more.

Francis Gulotta: [45:49] But like yeah, but there’s definitely it’s different organizations, there’s different forces at play and there’s different, you know, the people are different.

Marianne Bellotti: [45:58] I would go a step further and say that a lot of the advice in the book is also really good for building a completely new system from the ground up without any prior history whatsoever, right?

Marianne Bellotti: [46:09] Because the whole point of the book is about, let’s not get into like these petty little fights about who has the best technology and like whether you use the correct programming language, let’s optimize from attain ability.

Marianne Bellotti: [46:22] But as long as you can operate, you reach operational excellence and you can maintain your service with the resources you have, then you have the correct architecture and you are using the correct technology for it.

Marianne Bellotti: [46:32] There’s no magic language or magic database or like secret cloud environment that is going to guarantee your success.

Marianne Bellotti: [46:39] You have to be able to operate it with the people you’ve actually hired to operate it and you have to be able to maintain it at the level of resources and investment that you are willing to actually spend to maintain it.

Marianne Bellotti: [46:50] And so that is applicable no matter how old the system is like young, new, like startup, large come bearing corporation like that, broadly applicable all over the place in terms of like what’s different about the environment where they’re dealing with like uh, the mainframe versus the environment where they were dealing with like the five year old application that’s outgrown itself.

Marianne Bellotti: [47:16] Um, I think it’s the burden of history. Right? So like weirdly, it is exactly the thing that I, that I chose my excerpt on these kind of organizations with these older systems have more people who have died on the same hill and any time you try to go up the hill and you’re just passing dead bodies left and right.

Marianne Bellotti: [47:36] Like that’s a really intimidating thing, right Versus when you’re at like a five or six year old startup and like the monolith that was built or the thing that was built when it was like a three person engineering team just needs to be changed.

Marianne Bellotti: [47:50] You’re dealing with a certain amount of sentimentality, like maybe the people who built that thing in the first place still work there and our high level now and they’re like, don’t touch my baby and also don’t come into meetings and say that my baby isn’t beautiful, right?

Marianne Bellotti: [48:05] So you’re dealing with a certain degree of internal politics, but you have less of that history to like try to gather intelligence around and an account for.

Marianne Bellotti: [48:14] So there are like I would say that the differences are more in degrees of like what’s likely to come up and the intensity with which it comes up rather than like a full and complete difference in approaches and characteristics.

Francis Gulotta: [48:29] The amount of times I have to say it was great for the problem we had back then, you know, got us here.

Francis Gulotta: [48:36] I don’t know if it’s gonna get us there,

Marianne Bellotti: You know, and one of things I think is the most useful which like the most liberating for engineers to realize is that there is not possible to build a system that is perfect and last in that perfection for like 10, 20 years.

Marianne Bellotti: [48:52] I feel like that that I’ve taken to describing this as like the urban legend of software engineering.

Marianne Bellotti: [48:57] Everybody has the story about the mythical 10 X gray beard that showed up one day and re-architected the entire system and it was gorgeous and there was nothing like everything was just exactly the way it should be and perfect and that system survived for like you know 10, 20 a million orders of magnitude scaling and all that.

Marianne Bellotti: [49:18] But when you start talking to those people, it’s always first of all, no one ever really directly knows the legendary graveyard.

Marianne Bellotti: [49:23] It’s always like a friend of a friend or a guy who was at this company like a year before I joined or something like that.

Marianne Bellotti: [49:29] And then when you ask them well like is this part of the system like no one has touched it in like 10 years or as it scaled the answer to that has never know.

Marianne Bellotti: [49:37] So I’m like so in your mind it is this like legendary wizard that showed up one day and not the fact that over the course of X number of years you have been regularly maintaining it and updating and monitoring and like that has absolutely nothing to do with anything.

Marianne Bellotti: [49:53] So I mean people do, I watch it all the time.

Marianne Bellotti: [49:57] Like engineers Kind of like paint themselves into this corner because they think they’re trying to solve a puzzle of like all the various ways that something could scale and anticipating them all and having like a perfect solution for all of those scenarios from day one.

Marianne Bellotti: [50:11] And I’m like, that’s just a waste of time, right? Build what you can actually run and understand today and put some time and effort into watching how that situation changes and trying to make incremental change along the way. If you can do that, you’re golden.

Kristofer Joseph: [50:25] That that kind of made me think a little bit more about what Francis said earlier about the strangler pattern.

Kristofer Joseph: [50:33] And then it seemed like you were kind of saying that inexperience that you’ve had doesn’t work for everything.

Kristofer Joseph: [50:40] So it’s like how would you sell incrementally updating things without having some way to eject into like a different service or you know what I mean?

Marianne Bellotti: [50:49] Yeah, I think, I think the strangler pattern does work. The hesitation I have about that type of pattern sometimes is making sure you’re doing it in such a way that if you are forced to stop in the middle of the modernization effort, you’re not left with like duplicate interfaces.

Marianne Bellotti: [51:11] Like I was reading kind of somebody’s narrative modernization story the other day where they actually essentially built completely duplicate interfaces for something they were trying to modernize.

Marianne Bellotti: [51:26] And my first thought was like, oh, well you’re rolling the dice on that one, right?

Marianne Bellotti: [51:30] Because if you leave the deprecate ID interface up, people will use it, right?

Marianne Bellotti: [51:38] People will not stop using it right? And the longer you leave it up, the more people are gonna use it and the more people are gonna use the things that are using the delegated and like the harder it gets to like fully shut it down.

Marianne Bellotti: [51:49] So uh all strategies have their kind of pros and cons and they’re sort of risk profile and it really depends on how confident you feel about the mission that you’ve been given and the people of the air cover that you have in front of it and just thinking very thoughtfully about those patterns.

Marianne Bellotti: [52:08] And I do like to ask people like what happens if we don’t get to this point in the road map, right, what happens if they’re like our head of product is fired out from under us, New guy comes in, stops all work says we should do something different, like are we were better off now or we’re worse off now.

Francis Gulotta: [52:28] Right. Yeah, yeah, can share the success I’ve had with the strangler pattern, if it’s good for a point of reference, like The I joined, I joined a company, it was like 15 engineers, there were like 15 failing microservices and there was a new GraphQL server written by an intern who had just left written in a language nobody knew.

Francis Gulotta: [52:59] And so we had a new product to go live with that right around the time I got there and boy, we had a lot of downtime, what we ended up doing was wrapping everything with a new GraphQL server and that GraphQL server was just delegating to the other one at first, but also delegating to the microservices and we used we were able to get enough in there to shift our front ends over to it and then slowly service by service.

Francis Gulotta: [53:28] I think it’s giving up dollar bills to decommission named microservices to like removed their responsibilities, brought them into the main application and you know, and added capabilities to the main application to we built a monolith to, you know, to facilitate all the services needs, right.

Francis Gulotta: [53:47] Users and authentication was the hardest and I think that service Was about 30 grand a month.

Francis Gulotta: [53:53] And when I was able to turn off, like I stripped it of all responsibilities except for that being the source of truth for users, that one was great to turn off.

Francis Gulotta: [54:01] and like, you know, this is, this is also a lambda migration story.

Francis Gulotta: [54:05] I mean we went from a whole bunch of these microservices and a couple big services and we dropped our bill from I think it was like 45 grand a month to three grand a month and you know, by the time we were done and it was, it worked, it was hard, it was a slog and I had the cover to keep doing it until it was done.

Francis Gulotta: [54:24] I can’t tell you how many times I was told to well it’s good enough. Most things are better.

Marianne Bellotti: [54:27] You know, I’m looking forward to this conference talk where you talk about you took a set of microservices gradually, Shut them down until it was just a monolith because that that’s great.

Marianne Bellotti: [54:41] That’s great. But yeah, I mean like, I am known for being inherently skeptical and microservices a starting point.

Marianne Bellotti: [54:48] Not because I think microservices are bad. I like that pattern.

Marianne Bellotti: [54:51] I just think that most engineers way overdo it right way overdo it.

Marianne Bellotti: [54:56] Like they can’t, especially in the early stages of building something where you’re not quite clear what it is and who’s gonna use it and what it’s gonna do.

Marianne Bellotti: [55:04] How can you reason about separation of functionality and like what belongs with one another when you have absolutely no idea like what the product is gonna end up being.

Marianne Bellotti: [55:14] So, I think like this is I spent a lot of time talking about monoliths, uh an old technology because I want to improve some of the branding around the old technology so that people don’t categorically rule them out as options because any time people are categorically ruling things out without even stopping to consider them, You’re probably not going to make the best decision for the technology moving forward.

Simon MacDonald: [55:39] You just need a new marketing name for monolith. If you call, people will think it’s the hot new thing, they’ll jump right on it.

Simon MacDonald: [55:46] so we’ve kept you for about an hour here. So I’m gonna have to bring this plane in for a landing, I know that I could ask you about a billion more questions and it’s just been a really great talk.

Simon MacDonald: [55:56] I would like to sum things up with folks if you’re watching this after the fact just go by the book, it doesn’t matter if you’re modernizing legacy systems or they’re from the seventies or if they’re from the, I don’t know, 2017, it’s the everything becomes a legacy system at a certain point in time.

Simon MacDonald: [56:13] but also I wanted to throw this to you, I saw on twitter today you got something new called Breaking up the Monolith that you just literally just released today, is it?

Marianne Bellotti: [56:21] Yeah, so I’ve wanted to do this for a really long time.

Marianne Bellotti: [56:25] I love writing blog posts, I love writing books, I love doing conference talks.

Marianne Bellotti: [56:30] These are the typical mediums that I’ve been using to sort of like share my knowledge and like work with other engineers on different types of challenges, but they’re somewhat limited.

Marianne Bellotti: [56:42] And so I wanted to do something in the world of online courses for a while and we’re kind of soft launching today a set of mini courses around okay, how do you think strategically about architecture and like different ways of looking at architecture and figuring out what kind of changes you should be making two systems and like what kind of advantages you’re gonna get from making different changes and so the asked that I put out for people on twitter is that this is like a huge content development exercise for me, so I’m opening up one of the many courses, it’s specifically on the ratio of complexity to coupling.

Marianne Bellotti: [57:20] You’ve heard me geek out about this in the past, you’ll know that like the Charles parole, normal accident theory of systems, like the amount of complexity you have in the amount of coupling you have in the same system is a pretty strong indicator of like how much pain the system that is going to cause you long term.

Marianne Bellotti: [57:37] And so I really do a thorough deep dive into that concept from an engineer’s point of view.

Marianne Bellotti: [57:42] Like how do you look at a software system and think about complexity and coupling and like identify areas and then reason about like where you might be there too tightly coupled or too complex and then where do you go from there?

Marianne Bellotti: [57:53] And what I want people to do is sort of like give me lots of feedback on both.

Marianne Bellotti: [57:58] The format of the course and like what the next sections they want me to work on our and kind of like help me iterate on this because I do want to eventually build a community where people can have daily practice sort of thinking strategically about systems, practice system, thinking all of those things and so I’m really, really excited about kind of like opening this door and seeing like what happens.

Simon MacDonald: [58:23] Yeah, that sounds awesome, I can’t wait to check it out.

Simon MacDonald: [58:26] And for folks that want to follow Mariana twitter, she’s at bellmar, which is her twitter user name, not a suburb of Los Angeles.

Simon MacDonald: [58:35] Even though it sounds like that. And for us here and again, thank you all for joining the new breaking up, the monolith, the url will be in the blog post for people to be able to check out.

Simon MacDonald: [58:48] As for the book Club we’re gonna take August off, it’s gonna be a real busy month for Begin, so we’re gonna come back in September with another book club selection and if anybody has any suggestions, I should be able to confirm our September selection very soon, but if anybody has any other selections suggestions for books, please let us know because like we love talking to authors like Marianne and it was just it was just awesome to get a chance to talk with you today.

Marianne Bellotti: [59:14] Yeah, I had a great time. Thank you so much for inviting

Next Month

We are taking August off, see you in September!

Don’t miss the next book club meeting

Stay in touch by:

1. Joining the Architect Discord, where we will be hosting the book club video chat.
2. Follow the @begin Twitter account, where we will send out polls for future book club selections.

Or if you prefer emails join the book club newsletter. We promise that we will only use this mailing list for book club purposes like meet-up reminders and book club selections. We do not sell your personal data.