When is it time to make system changes?


Hi!

This week I'd like to shortly discuss a decision making process when it comes to making large system changes, here's a quote and a link to a nice blog post I've read yesterday:

We should always put off significant complexity increases as long as possible.

The post above tells the story of a team running a large monolith with one large database which they kept scaling up for years.

At one point, it reached 80% load with short bursts of downtime, but there was nowhere to grow in terms of scale.

This leaves the time with one of two basic options:

  1. Redesign the system (microservices? sharding? different DB?)
  2. Taking the less exciting path of debugging the source of struggle and slowly improving (code, queries, internal configs)

There's no right or wrong, but a very important statement made in the post and that I find myself reiterating at work is:

Making system changes will be expensive, but not because of technology, mainly for engineering hours and attention.

While the bold statement above is often overlooked, it's not only this should be considered when decided how to make progress, it should also be taken within the context of the situation:

  • Is the system on fire?
  • Do we expect fires to happen for some time?
  • Are we looking at growth? Exponential? Or gradual?

With a clear picture, it's easier to make wiser decisions that are contextual and informative.

This may be obvious to some, but in many cases even senior leaders tend to jump on the latest trend (I know I did) and let enthusiasm take over better judgment.

When I used to be in the consultant's shoes (but often one that had to follow his own advice and implement the changes), it was easier for me to view things through a different unbiased lens, which I try to keep in mind even today.

The takeaway here is, what would you do if you were a consultant? What's the context here? How would future-you look back at this decision a year from now?


With that in mind, have a great weekend! And here's a few interesting things I've discovered / read / tried over the past week:


You can find me on other channels as well:

📺 : www.youtube.com/@devopstoolbox

🎤 : https://topeaks.buzzsprout.com/

✍️ : https://omerxx.com/

ESPRESSO FRIDAYS

Every once in a while I send hand picked things I've learned. Kind of like your filter to the tech internet. No spam, I promise!

Read more from ESPRESSO FRIDAYS

Thank you to our sponsors who keep this newsletter free to the reader: Aikido is your no-nonsense DevSecOps platform. One central system that shows you what matters and how to fix it, from code to cloud. So you can get back to building. Try Aikido today! We’ve all been there – that heart-stopping moment when you realize you’ve accidentally exposed sensitive information. Today, I’m sharing a personal story and introducing a tool that could save you from a similar nightmare. The Nightmare...

Hi friends! Today we’re diving deep into improving your terminal history management. Exploring techniques that can transform your command line experience from frustrating to fluid. Whether you’re a CLI novice or a terminal titan, these methods will boost your productivity and smoothen your workflow. To do that, we’ll explore three levels of terminal command management, from basic to advanced. Not actually running on my phone 😅 1. Basic: Built-in (mostly unused) tooling: Even without...

Hi friends, Tmux is a fantastic tool for managing terminal sessions, but it has its limitations. One major drawback is the lack of a floating pane feature, which can make navigating between different panes cumbersome and inefficient. Me frustrated with Tmux lack of floating panes while Zellij is killing it... Most users workaround this by creating new Tmux windows or panes, or by using hidden splits to zoom in and out. These methods work but can be inefficient and require many keystrokes,...